Digital forensics

Eyesight to the Blind – SSL Decryption for Network Monitoring [Updated 2019]

SSL and network monitoring aren't the most compatible of partners – even with the most sophisticated detection infrastructure in the world, you'll not derive many useful indicators from the barren randomness of encrypted traffic. Consider the plight of the Sguil sensor shown below:

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

The webserver's use of SSL means that network-based incident detection is problematic. No amount of tuning of the sensor's Snort instance will help it detect intrusion attempts – the only traffic it will see is HTTPS. Also, if an incident is detected by other means (e.g., customer notification, web server log file monitoring, etc.) the investigative value of Sguil's full packet capture is greatly diluted.

There are a number of ways to get back the visibility stolen by SSL, including the following. You could:

• Terminate the SSL "in front of" the webserver, perhaps on a reverse-proxying loadbalancer or web application firewall. You can then monitor the decrypted traffic between the loadbalancer and the webserver.
• Perform monitoring tasks on the webserver itself, perhaps by increasing the level of web and application logging.
• Give your existing sensor platforms the means to decrypt the SSL sessions.

Each approach has its pros and cons; this article will show you how to leverage the latter technique to restore the eyesight of your blind Sguil sensors.

SSL Decryption

We first need to understand a little about the mechanics of SSL decryption; you can read about it in depth here. In a nutshell there are two conditions that must be met before we can proceed:

1. The server must be using the RSA key exchange mechanism (see here, bottom of page, and here, section F.1.1.2). Fortunately, this is the most common form of key exchange for SSL based servers; if you're using DSA keypairs or the Diffie-Hellman key exchange mechanism you're probably out of luck.
2. You must have access to the server's private RSA key, and be able to copy it onto your Sguil sensor.

The latter point means that the only SSL decryption we're going to be able to pull off is decryption of traffic to and from servers that we own – we're not going to be able to magically decrypt arbitrary SSL traffic (darn!) However, this is quite adequate from the viewpoint of intrusion detection and network forensics.

Now, where did I leave my keys…?

Getting the server's private key onto the Sguil sensor could take a bit of work. The key might be stored in a .pem file somewhere on the server, making it child's play to copy it to the Sguil sensor. However, certain operating systems like Windows store certificates and keys in a "certificate store" instead of .pem files. In order to get the private key, we need to first export the certificate and key together as a PKCS12 file (read about that here). Once we've got the exported certificate, we can proceed to extract the private key using openssl like this:

C:>openssl pkcs12 -in c:myExportedCertificate.pfx –out c:myExportedCertificate.pem -nodes

When prompted, enter the password you gave when you exported the certificate and key. The file myExportedCertificate.pem will contain a block of text that looks like this:

-----BEGIN PRIVATE KEY-----

MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCvHJFIpFwXZJ0x

2DzEY2B4MDBBu/+jbfUriFI+GKh6Q5oEGTAARh3OAP+UMedNf2t8/MVJdEEAM7TQ

<snip>

EZq3TiCB3e+GSjVRorB0YGvDzHR1V098LEPOvPIKNMcLCC4lGQeTg+usZmtcx4VI

wGI70e4Byd+Lba0lUKY8UIU=

-----END PRIVATE KEY-----

Copy and paste this block into a file called myPrivateKey.pem and save it on your Sguil sensor.

Windows also has the concept of a non-exportable private key; a key stored in this way cannot normally be exported by the Windows certificate management tools (although tools like Jailbreak claim to be able to do it).

A word of warning

As noted in the article on decrypting SSL, your server's private key is a very sensitive thing indeed. With it, an attacker can decrypt the server's SSL traffic or use it together with the server's certificate in order to masquerade as a legitimate site. You must take extreme care with the handling of the private key – don't leave copies of it lying around in temporary directories, make very sure it doesn't end up on the Internet by accident, and make certain that the Sguil sensor you copy it to is as locked down as much as possible. Think of this key as the combination to a safe guarding your valuables—make very sure that you only share it with trusted sources. Leaking a private key is too high a price to pay for visibility into your SSL traffic, so take care!

Putting the key to work

Private key in hand, we could clearly point Wireshark at Sguil's full-content capture directories. This approach hardly lends itself to efficient network security monitoring; it would be much better if we could pipe our decrypts straight into Sguil in near real-time, leveraging the benefits of Snort, SANCP, PADS, and whatever else you've got running.

Dmitry Plashchynski's viewssld package is just the tool for the job. I've recently made some additions to the code to ease integration with Sguil and to produce richer decrypts. Conceptually, viewssld looks like this:

viewssld listens on a given interface via the usual libpcap for nominated SSL traffic. The captured frames are then passed on to Atomic Labs' libdssl library which handles the actual decryption (the latest version of libdssl at the time of writing is 2.1.1, available via svn here).

The output of the libdssl code is the decrypted packet contents only (no headers or anything like that), so libnet is used to build a "fake" traffic stream containing the decrypted SSL. This fake stream is pumped out through the output interface, where Sguil is listening.

Earlier versions of viewssld would only output the client->server traffic (e.g., HTTP requests). My modifications to viewssld include:

Outputting a fake 3-way TCP handshake for the decrypted traffic

Outputting the decrypted server replies as well as the client requests, keeping the (fake) TCP sequence numbers properly up to date. The sequence numbers and other TCP/IP header "furniture" are of course complete fiction – because the decrypted traffic is a different size to the encrypted traffic, it's not possible to use the "real" values. The decrypted stream may also be fragmented/segmented differently to the encrypted one, but again there's little you can do about this

Outputting a fake RST at the end of each session

Using an easily-searchable IP Ident number for all the fake frames so you can tell which ones were output by viewssld (54609 == 0xd551 == DSSL  )

Outputting the decrypted frames on a different port to which they were captured on (e.g. 443->80, 465->25, 993->143, etc.)

The first three points help to keep Sguil's SANCP instance happy, since viewssld's output is now a reasonably well-formed TCP stream with a defined beginning and end. Point four reminds us that although the decrypted traffic is available to Sguil, it is a complete fabrication – no such traffic ever existed in this form on the network. By tagging decrypted IP datagrams with an ident of 0xd551 you can sort the recorded facts from the decrypts.

Point five aids with integrating viewssld and Sguil as shown in the diagram below:

It goes like this:

1. Raw traffic (blue arrows) enters Sguil's usual monitoring interface.
2. This traffic is captured by both Sguil and viewssld.
3. viewssld decrypts what it can and pumps it back onto the interface it captured it from (red arrows). This is only made possible by using a different TCP port for the decrypted frames – if it used the same port it would re-enter viewssld again.
4. The fake stream of decrypted SSL is picked up by Sguil.

Sguil/SANCP/Snort/PADS/etc. therefore see both the encrypted and the decrypted traffic, as shown in the screenshot below. In the background you can see a SANCP query with two rows of results. The first row is the raw SSL traffic captured by Sguil on port 443; the second row shows Sguil capturing the corresponding fake decrypted stream output by viewssld on port 80. The two transcript windows tell the rest of the story:

I've tested viewssld with HTTPS, SMTPS, POP3S and IMAPS – all perform as expected. A sample viewssld.conf file for these four protocols might look like this:

# PID-file path (default: /var/run/viewssl.pid)

pid = /var/run/viewssl.pid

# daemonize? on/off (default: off)

daemon = on

# loglevel 0-10 (default: 0)

loglevel = 10

# SSL protocols to decrypt

[webserver]

src = eth1

dst = eth1

ip = 10.11.12.13

port = 443

key = /etc/viewssld/webserver.key

dsslport = 80

[mailserverSMTP]

src = eth1

dst = eth1

ip = 20.30.40.50

port = 465

key = /etc/viewssld/mailserver.key

dsslport = 25

[mailserverIMAP]

src = eth1

dst = eth1

ip = 20.30.40.50

port = 993

key = /etc/viewssld/mailserver.key

dsslport = 143

[mailserverPOP3]

src = eth1

dst = eth1

ip = 20.30.40.50

port = 995

key = /etc/viewssld/mailserver.key

dsslport = 110

Notice how the src and dst interfaces are the same in all cases, and that the non-SSL port is specified (dsslport=) as well as the SSL port (port=).

Another word of warning

There are a couple of things to consider when employing viewssld on a Sguil sensor:

• SSL decryption has a computational cost associated with it. If your sensor's CPU is normally very busy, the extra crypto load may push it over the edge.
• The decryption process is very sensitive to packet loss. If viewssld misses a packet, decryption of the stream will stop at that point; if it misses the SSL handshake at the start of an encrypted session, it won't decrypt anything at all.
• You're now storing both the encrypted and decrypted traffic, so there may be a disk utilisation impact.

Show me the money!

viewssld can be obtained from GitHub here. You'll need libpcap, libssl, libnet and libdssl installed to compile the code. Certain versions of libdssl don't seem to compile up a shared library, so you might need to statically link viewssld to libdssl.a if this is the case. If you get a link error when compiling like this one:

sensor:~/viewssld/src# make

gcc -g -O2 -lpcap -lssl -lnet -ldssl -o viewssld viewssld.o utils.o

viewssld.o: In function `proceed':

/user/viewssld/src/viewssld.c:339: undefined reference to `CapEnvCreate'

/user/viewssld/src/viewssld.c:341: undefined reference to `CapEnvSetSSL_ServerInfo'

/user/viewssld/src/viewssld.c:368: undefined reference to `CapEnvDestroy'

/user/viewssld/src/viewssld.c:354: undefined reference to `CapEnvSetSessionCallback'

/user/viewssld/src/viewssld.c:356: undefined reference to `CapEnvCapture'

viewssld.o: In function `error_callback_proc':

/user/viewssld/src/viewssld.c:773: undefined reference to `SessionToString'

viewssld.o: In function `session_event_handler':

/user/viewssld/src/viewssld.c:444: undefined reference to `SessionToString'

/user/viewssld/src/viewssld.c:462: undefined reference to `SessionSetCallback'

/user/viewssld/src/viewssld.c:464: undefined reference to `SessionSetMissingPacketCallback'

collect2: ld returned 1 exit status

make: *** [viewssld] Error 1

…you can fix it like this:

sensor:~/viewssld/src# gcc -g -O2 -lpcap -lssl -lnet -ldssl -o viewssld viewssld.o utils.o /usr/local/lib/libdssl.a

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Learn Digital Forensics

Build your skills with hands-on forensics training for computers, mobile devices, networks and more.

Start Learning

Get it up and running, and give your Sguil sensors sight beyond sight! For more info on SSL, check out our Cryptography 101 article.