This Feels Scripted: Zeek Scripting and Splunk

I originally planned to write this story as a follow-up to another blog that SURGe released for CVE-2022-3602 and CVE-2022-3786 (aka SpookySSL). That blog mentions that we weren’t able to test with any malicious payloads yet, and as things go…

After releasing that blog, we came across proof-of-concept exploits that weren’t detected by our searches. I’ll provide some updated searches to help you out later in this blog, but first I want to document my journey in case others find themselves in a similar situation.

SpookySSL, More Like SneakySSL

My tale begins with me briefly revisiting our original blog. We used Zeek in our searches to capture and log any SSL/TLS connections, along with the respective X.509 certificates. If your traffic is TLSv1.3 you won’t see any X.509 files, as those are encrypted as part of the newer protocol (we’ve added you to the visibility naughty list TLSv1.3!).

If you want to understand more details about the vulnerabilities and detections we wrote, please refer to the prior blog. In essence, we tried to search for evidence of malicious punycode within the Name Constraints and Subject Alternative Name (SAN) fields of the X.509 certificates.

When we tested the detections on packet captures that were generated by proof-of-concept exploits, we didn’t get the results we wanted.

Why is that?

What Isn't Logged Can’t Be Seen

By default, Zeek logs the SAN fields within X.509 certificates, but doesn’t log the Name Constraints extension at all. Why didn’t we see evidence of the SAN field? It would be good to understand if Zeek is detecting those fields. Zeek has the ability to log the entire X.509 certificate under a field called cert within the logs. Interrogating that value would be a good place to start. Loading this Zeek script creates a Base64 encoded version of the certificate within Zeek’s x509.log. This is off by default, as it will create much larger X.509 events in your logs, but it definitely helps in this instance.

How do you load Zeek scripts? One method is to add an @load directive into your local.zeek file. This file typically lives under $PREFIX/share/zeek/site/local.zeek ($PREFIX being the directory where you’ve installed Zeek).

Paste @load policy/protocols/ssl/log-certs-base64 into the local.zeek file, save it and then run $PREFIX/bin/zeekctl deploy to load the new configuration.

If everything starts up without issues, you can see the cert field created in any new X.509 entries. The image below is where we’d expect to also see the SAN field with our malicious puny code.

^{Fig 1. X509 log from Zeek (cert value continues off-screen)}

We could use that cert value to find out if we got the data we needed. CyberChef has a handy recipe to parse Base64 encoded certificates to human-readable format. Running our cert value through that recipe actually produced the value we expected to see in the logs as shown here:

^{Fig 2. CyberChef output of certificate extensions}

It's great that CyberChef displays what we want, but we really need that output to be logged in order to make use of it within Splunk.

Zeek uses OpenSSL for certificate parsing. So, what do we get when we try to analyze that certificate directly within OpenSSL? Saving that Base64 value to a file with the following top and bottom lines allows you to work with it in OpenSSL.

-----BEGIN CERTIFICATE-----

Base64 cert value goes here

-----END CERTIFICATE-----

My certificate is named puny.crt, so running the following OpenSSL command will parse out human-readable certificate info to analyze.

openssl x509 -in puny.crt -text -noout

Here is OpenSSL’s interpretation of the certificate extensions, as shown from CyberChef above.

^{Fig 3. OpenSSL output of certificate extensions}

It looks as though OpenSSL has difficulty working with the malicious puny code data within the X.509 SAN extension. The flow-on effect is that Zeek does not log this value for us to use in our detections. There are other methods to use within OpenSSL to extract this information, but I won’t get into that today. If you want to explore this further, do some searching on the asn1parse command that can be used within OpenSSL.

Where does that leave us? Well, our story has a choose-your-own-adventure plot twist!

Choice 1: Write a New Zeek Script

Spoiler alert! I chose this option because I really thought logging X.509 extensions would be a valuable thing when trying to hunt within certificate data (Zeek doesn’t log the extensions by default).

I only write Zeek scripts every 6-12 months and find myself quite rusty whenever I need to do it again. I usually turn to someone named Ben (you know who you are) and beg for help, or I rewatch what I believe is the best video out there to help you write a Zeek script.

As with most of my coding, there’s usually a lot that can be improved or made more elegant.  But I couldn’t find anyone else who’d written this, and I made it work, so there!

I was going to extend the base Zeek X.509 functionality to include my custom extension logging but decided to make things a bit more simple and just create an entirely new custom x509_extensions log. If you want to see my custom script you can get it here.

This script uses the base X.509 Zeek functions, but then looks for any extensions within certificates, and if found, logs the following fields as shown here in a Name Constraints extension event.

^{Fig 4. X.509 Name Constraints Extension Event}

I’ve included a few other fields here to help correlate with other Zeek data.

• fingerprint- Used to correlate back with the certificate in Zeek’s X.509 log
• uid- Used to correlate back with a number of other logs that Zeek creates for connections
• ts- Timestamp when the extensions were seen

How about the SAN extension from the same X.509 certificate?

^{Fig 5. X.509 SAN Extension Event}

Well, we can’t get lucky every time can we?  We still see the othername:<unsupported> value, but now that we have the Name Constraints extension logged we can write a detection to look for the offending punycode.

If you are now ingesting X.509 extension logs, here’s a quick search to detect the use of punycode within the Name Constraints extension.

index=main sourcetype="bro:x509_extensions:json" name="X509v3 Name Constraints"
| rex field=value "(?<punycode_detected>xn--.*)"
| search punycode_detected=*
| table  ts punycode_detected fingerprint uid

^{Fig 6. Detection of punycode}

Now that we’ve run through my science experiment, I want to show you another method of detecting this behavior.

Choice 2: Use a Zeek Detection

We always hear the phrase “why reinvent the wheel?” for good reason. I could have just waited for my good friend Ben (see above) to complete his work on this, but I usually can’t leave well enough alone.

Corelight released a Zeek package that can detect the same thing I am doing with my Splunk search above but does not require you to log the X.509 extensions. It also includes another script that helps to detect any servers running the vulnerable OpenSSL versions based on the HTTP Server header.

If you load that package and run the same packet capture from earlier, you’ll see the following event in Zeek’s Notice log.

^{Fig 7. Zeek Notice log entry}

What’s nice about this entry is that most of the relevant information is contained within the single event. If you are concerned about these recent CVEs for OpenSSL and you use Zeek, I highly recommend installing this package.

Conclusion

I can see most people leaning toward the Zeek detection package here, but I still see value in logging the extensions themselves. This goes a bit deeper into the realm of hunting vs. just detecting, but as we’ve learned from that great commercial: “Porque no las dos?”

Happy Splunking!

- Shannon Davis

^{Photo credit: Jeswin Thomas on Unsplash}