Using Zeek for network analysis and detections
What is Zeek?
Zeek (formerly known as Bro) is an open-source network traffic analyzer. The tool sits on a sensor and observes network traffic. It is free, open-source software designed to extract hundreds of fields in network data in real-time. The tool has pre-built parsers for numerous protocols such as (HTTP, SSL, DNS, FTP etc.) and allows for the creation of custom parsers for protocols not yet supported. Zeek can detect anomalies, but not in the same fashion as a traditional IDS (like Suricata).
The tool mirrors (or SPAN) a router within your network to collect a copy of the traffic. It then takes that traffic and processes, parses and structures the network data based on protocols. The processed data is then stored into various log files (dns.log, http.log, con.log etc.). We will explore a few of these in the below sections to learn how they can be used. A common use of these files is to ingest them into a SIEM platform to craft detections.
Network data. Who cares?
Good network data is key when investigating security events and crafting good detections. To create a complete picture of what’s occurring in your network, you must first understand what systems and services are connecting to and how traffic flows through your organization’s IT environment. There are many different ways to collect network data. You can collect it via a firewall, netflow or other network analysis tools/technologies. However, a lot of the time, this data is incomplete or extremely expensive to capture. Enter Zeek.
Zeek is a great source of network data for hunting, monitoring and analysis. If configured correctly, it does not boggle down the network or overload security teams with useless data. It extracts fields in network data to provide teams with parsed and actionable information that can then be used to craft meaningful detections to better secure the network.
1. Get the latest version of Zeek that works best with the system and distro you plan to install it on (E.g., Ubuntu)
- You can find older versions of Zeek on various GitHub pages if there is a deprecated feature you wish to use
2. Before you install Zeek, make sure you have all the required dependencies
- You can install from a pre-built binary package
- You can install the code directly from the source
3. Next, configure the run time environment and define the local networks to monitor
4. Before you can run Zeek, you need to deploy the ZeekControl configurations
5. You can then check the Zeek logs in the below directory to see if Zeek is set up and configured properly. If you navigate to the below directory, you should start to see log files in the directory
Let’s take a look at the format of one of the most used Zeek log file. The conn.log as reproduced from ‘s Corelight Bright Ideas Blog.
|ts||time||Timestamp of the first packet|
|uid||string||Unique ID of the connection|
|id.orig_h||addr||Originating endpoint’s IP address (Orig)|
|id.orig_p||port||Originating endpoint’s TCP/UDP port (or ICMP code)|
|id.resp_h||addr||Responding endpoint’s IP address (Resp)|
|id.resp_p||port||Responding endpoint’s TCP/UDP port (or ICMP code)|
|proto||proto||Transport layer protocol of connection|
|service||string||Detected application protocol, if any|
|orig_bytes||count||Orig payload bytes; from sequence numbers if TCP|
|resp_bytes||count||Resp payload bytes; from sequence numbers if TCP|
|conn_state||string||Connection state ( see conn.log>conn_state)|
|local_orig||bool||Is Orig in Site::local_nets?|
|local_resp||bool||Is Resp in Site::local_nets?|
|missed_bytes||count||Number of bytes missing due to content gaps|
|history||string||Connection state history(see conn.log>history)|
|orig_pkts||count||Number of Orig packets|
|orig_ip_bytes||count||Number of Orig IP bytes (via IP total_length header field)|
|resp_pkts||count||Number of Resp packets|
|resp_ip_bytes||count||Number of Resp IP bytes (via IP total_length header field)|
|tunnel_parents||set||If tunneled, connection UID of encapsulating parent(s)|
This file stores all the logs Zeek created for connection attempts identified in the network. Drilling down into the fields and related descriptions, you can see that this entry shows the connections (TCP/UDP/ICMP) systems in your network are making. This is often a great place to start when learning to use Zeek data and investigating an event. Connection details can often help you baseline systems and identify abnormal or anomalous activity (such as a specific system reaching out to a system it’s never communicated with before).
Other files created when Zeek produces logs are the following:
- Files.log – information on different file analysis
- Dns.log – additional data on parsed DNS related activity captured
- Http.log – data on HTTP requests and replies captured by the parser
- Ssh.log – SSH connections captured during analysis
- Ftp.log – ALl FTP related activity detected by Zeek
Zeek produces numerous other log files. You can find more information on them in the Zeek documentation here.
Use-case: threat hunting
Zeek is an excellent tool for threat hunting. Many IDS (such as Suricata) are focused on signature and rule-based detections. Zeek can be used as a traditional IDS. However, Zeek can be used to focus more on specific network protocols for deeper analysis. The more data you have to work with when threat hunting, the better.
As we learned in the above sections of this article, Zeek stores metadata, which it captures into various files. The files can then be used by security analysts and threat hunters when performing investigations or hunting for bad activity in the network.
The tool can help analysts:
- Monitor network traffic flows for anomalous or malicious activity
- Profile specific protocols and services (e.g., DNS, HTTP, FTP) to build baselines that can be used to find deviations in patterns (hinting that something nefarious could be happening)
- Enrich their security tool data by adding a layer of analytics on their security tools stack
Zeek is an extremely powerful network analyzer. There are numerous use-cases for the tool ranging from network detection, forensic evidence captures, SIEM alert enrichment and much more. Zeek and an IDS like Suricata have similar methods to produce data and help security professionals defend their environment. They are often used in conjunction to provide a layered approach to security.
In the final article of this series, I will demonstrate how to convert PCAP files into Zeek logs and investigate the data.