Posted inNetworking

Network Security Monitoring with Zeek on Linux: Real-Time Protocol Analysis and Metadata Extraction

1. The Real-World Problem: Network Blind Spots and Storage Exhaustion

During a security audit across our core infrastructure, an anomalous outbound beacon triggered a low-priority netflow alert. Every fifteen minutes, an internal host pushed data to an unclassified IP (198.51.100.45) over TCP port 443. Our perimeter firewall logged the 5-tuple—source, destination, port, and byte count. That was all. We had no visibility into the SNI domain requested, the TLS ciphers negotiated, or the payload headers exchanged.

To investigate, we mirrored our 1 Gbps uplink and fired up tcpdump. The result was a logistical mess. In 48 hours, the sensor accumulated 620 GB of raw .pcap files across thirty rotation segments.

Opening a single 4 GB PCAP file took eight minutes and froze Wireshark on a 32 GB RAM workstation. Finding one DNS query across multiple split files burned an entire afternoon of bash scripting. When an incident is unfolding, analysts need immediate answers. They cannot afford to spend hours unpacking multi-gigabyte PCAPs just to identify a single hostname.

2. Root Cause Analysis: Why Traditional Packet Analysis Breaks Down

Our visibility bottleneck stemmed from treating all traffic data identically. Traditional monitoring approaches make flawed assumptions about packet storage, protocol depth, and alerting context:

  • Indiscriminate Payload Storage: Running raw packet capture collects everything—from 4K video streams to bulk file backups. Over 85% of modern enterprise traffic is encrypted with TLS 1.3. Storing those encrypted application bytes without session keys burns disk I/O and ballooned our SAN storage bills for zero investigative payoff.
  • Blindness to Application State: Stateful firewalls track TCP handshakes and sequence numbers, but they ignore reassembled protocol semantics. If an attacker runs a custom Cobalt Strike beacon over port 443, Layer 4 monitoring marks it as standard HTTPS simply because the port matches.
  • Alerts Stripped of Context: Traditional NIDS engines match packets against static regular expressions and emit isolated alert strings. When an alert fires, you still need to pull surrounding context manually. You need the preceding DNS lookup, the TLS certificate chain, and connection duration to distinguish a live compromise from a benign scanner.

3. Solutions Compared: Full PCAP vs. Signature IDS vs. Zeek

To balance operational cost against investigative fidelity, we bench-tested three passive network observation architectures on our 1 Gbps core uplink:

Monitoring Approach Storage Footprint (1 Gbps link, ~350 Mbps avg) Query Speed Protocol Context Maintenance Burden
Full PCAP (tcpdump / daemonlogger) Huge (~3.5 TB / day) Minutes to hours (BPF scans over split files) Raw bytes; reassembly required High; aggressive rolling purge scripts needed
Signature NIDS (Snort / Suricata) Low (~500 MB / day in alerts) Fast (indexed in SIEM) Sparse; metadata only captured on rule hit High; continuous rule tuning to curb false positives
Metadata Extraction (Zeek) Compact (15–30 GB / day) Sub-second with CLI tools (zeek-cut, ripgrep) Deep, stateful parsing (HTTP, DNS, TLS, SSH, SMB) Low; automated log rotation and predictable retention

Full packet capture still has a place for niche protocol debugging. Signature engines tell you when a known bad pattern appears. Zeek takes a different angle: it operates like an automated protocol auditor. It discards high-volume payload bodies while parsing and logging every protocol event into compact, structured text logs. You cut storage footprint by roughly 95% without sacrificing investigative depth.

4. The Best Approach: Deploying Zeek on Linux for Real-Time Metadata Extraction

The cleanest setup places Zeek on a dedicated sensor node running Debian 12 or Ubuntu 22.04 LTS. The host pulls traffic directly from a switch SPAN port or a passive TAP. For an environment pushing 500 Mbps to 1 Gbps of sustained traffic, a modest 4-core CPU host with 16 GB RAM and an NVMe log buffer handles the ingestion load with zero dropped packets.

Step 1: System Preparation and Interface Configuration

Dedicate a secondary physical interface (e.g., eth1 or enp3s0) exclusively to traffic capture. Strip this interface of all IP assignments so it remains unroutable and invisible on the wire. Next, disable network interface card (NIC) hardware offloading. When offloading features like GRO (Generic Receive Offload) or LRO stay enabled, the NIC driver merges packets before the kernel sees them. This breaks Zeek’s TCP stream reconstruction engine.

# Bring up interface in promiscuous mode with no IP
sudo ip link set dev eth1 up
sudo ip link set dev eth1 promisc on

# Strip TCP/UDP hardware offloading to preserve true packet boundaries
sudo ethtool -K eth1 rx off tx off gso off gro off tso off

To persist these NIC settings across server reboots, drop a script into /etc/networkd-dispatcher/routable.d/50-sniffer.sh or create a custom systemd service targeting the interface.

Step 2: Installing Zeek from the OpenSUSE Build Service

Distribution package repos often bundle outdated Zeek builds. Use the official OpenSUSE Build Service repositories to pull the current Long-Term Support (LTS) release:

# Install key management prerequisites
sudo apt-get update && sudo apt-get install -y curl gnupg2 lsb-release

# Import the OpenSUSE signing key
curl -fsSL https://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/Release.key | \
  gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null

# Add the official repository source
echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/ /' | \
  sudo tee /etc/apt/sources.list.d/security:zeek.list

# Update package lists and install Zeek LTS
sudo apt-get update
sudo apt-get install -y zeek-lts

# Export the Zeek binary directory to your default profile
echo 'export PATH="$PATH:/opt/zeek/bin"' >> ~/.bashrc
source ~/.bashrc

Step 3: Configuring Local Subnets and Sniffing Interfaces

Head to /opt/zeek/etc/ to configure core detection parameters.

Edit /opt/zeek/etc/networks.cfg to define your RFC 1918 subnets and DMZ boundaries. Zeek compares these definitions against source and destination IPs to tag connection directionality accurately:

10.0.0.0/8          Corporate_LAN
172.16.10.0/24      Production_DMZ
192.168.50.0/24     Management_VLAN

Next, bind Zeek to your sniffing NIC in /opt/zeek/etc/node.cfg. For a single-host sensor setup, update the standalone configuration stanza:

[zeek]
type=standalone
host=localhost
interface=eth1

Step 4: Managing the Engine with zeekctl

Zeek includes a dedicated management wrapper, zeekctl, for config validation, process control, and worker management:

# Validate configuration syntax
sudo /opt/zeek/bin/zeekctl check

# Deploy configuration and spin up background capture workers
sudo /opt/zeek/bin/zeekctl deploy

# Verify that the standalone instance is running
sudo /opt/zeek/bin/zeekctl status

If the status returns running, Zeek is actively dissecting traffic from eth1.

Step 5: Querying Extracted Metadata with zeek-cut

Zeek streams structured logs into /opt/zeek/logs/current/. By default, files are tab-delimited TSV tables with explicit field headers, though you can toggle JSON output if you pipe directly to Elasticsearch or OpenSearch.

Primary log streams to monitor include:

  • conn.log: Comprehensive record of every flow, tracking session duration, IP addresses, ports, protocol flags, and total transferred bytes.
  • dns.log: Full record of DNS queries and answers, including resource record types, query names, resolved addresses, and DNS server RCODEs.
  • ssl.log: Detailed TLS negotiation metrics, exposing Server Name Indication (SNI) strings, validated certificate subjects, issuer CNs, and cipher choices.
  • http.log: Unencrypted HTTP requests, capturing requested URIs, HTTP methods, host headers, status codes, and user-agent strings.

Skip visual GUI tools and query active traffic on the CLI with zeek-cut:

# Isolate top 10 queried external domain names
zeek-cut query answers < /opt/zeek/logs/current/dns.log | \
  sort | uniq -c | sort -nr | head -n 10

# Hunt for suspicious external SNI domains and legacy ciphers in TLS handshakes
zeek-cut id.orig_h id.resp_h server_name cipher < /opt/zeek/logs/current/ssl.log | \
  grep -v "yourdomain.com" | head -n 15

# Identify abnormally long TCP connections (sorted by duration in seconds)
zeek-cut id.orig_h id.resp_h duration proto service < /opt/zeek/logs/current/conn.log | \
  awk '$3 > 3600' | sort -k3 -nr | head -n 10

Step 6: Writing Custom Event-Driven Detection Scripts

Zeek is also a programmable network analysis language. Its event-driven architecture lets you write targeted logic executed whenever a protocol state machine changes. For example, if policy forbids TLS 1.0 and 1.1, you can flag non-compliant handshakes directly in notice.log.

Create /opt/zeek/share/zeek/site/detect-insecure-tls.zeek:

@load base/protocols/ssl
@load base/frameworks/notice

module SecurityMonitoring;

export {
    redef enum Notice::Type += {
        Insecure_TLS_Version
    };
}

event ssl_established(c: connection)
    {
    if ( ! c?$ssl )
        return;

    # Trigger notice if handshake settles on legacy TLS versions
    if ( c$ssl?$version && (c$ssl$version == "TLSv10" || c$ssl$version == "TLSv11") )
        {
        NOTICE([$note=Insecure_TLS_Version,
                $msg=fmt("Insecure TLS version: %s from %s to %s (SNI: %s)", 
                         c$ssl$version, c$id$orig_h, c$id$resp_h, 
                         c$ssl?$server_name ? c$ssl$server_name : "<none>"),
                $conn=c]);
        }
    }

Activate the script by appending an include line to your local site configuration:

echo '@load detect-insecure-tls.zeek' | sudo tee -a /opt/zeek/share/zeek/site/local.zeek
sudo /opt/zeek/bin/zeekctl deploy

Whenever an internal client triggers a deprecated handshake, Zeek writes a structured entry to /opt/zeek/logs/current/notice.log. The record includes the client IP, destination server, negotiated version, and requested SNI hostname.

Deploying Zeek on Linux gives your SOC high-fidelity protocol inspection while keeping storage overhead predictable. By separating actionable operational metadata from raw payload bytes, your analysts shift from wading through multi-gigabyte PCAPs to querying real-time transaction state in seconds.

Share:
Tags: