Skip to content

Real-Time Zeek Analytics

Rule Function

Zeek is an open source network security monitoring tool, consisting of a suite of monitoring apps divided into modules. This rule accepts syslog messages in JSON format comprised of Zeek log messages originating from these various modules. The rule then sets a set of basic user tags (indicated below) and composes a LogZilla event message consisting of certain of the incoming JSON data fields (Zeek data).

The Zeek modules (and whether they are processed by this rule) are:

Processed Module Description
conn TCP/UDP/ICMP connections
dce_rpc Distributed Computing Environment/RPC
dhcp DHCP leases
dnp3 DNP3 requests and replies
dns DNS activity
dpd Dynamic protocol detection
files File analysis results
ftp FTP activity
http HTTP requests and replies
irc IRC commands and responses
kerberos Kerberos
modbus Modbus commands and responses
modbus_register_change Tracks changes to Modbus holding registers
mysql MySQL
ntlm NT LAN Manager (NTLM)
ntp Network Time Protocol
radius RADIUS authentication attempts
rdp RDP
rfb Remote Framebuffer (RFB)
sip SIP
smb_cmd SMB commands
smb_files SMB files
smb_mapping SMB trees
smtp SMTP transactions
snmp SNMP messages
socks SOCKS proxy requests
ssh SSH connections
ssl SSL/TLS handshake info
stats Memory/event/packet/lag statistics
syslog Syslog messages
tunnel Tunneling protocol events
weird Unexpected network-level activity
x509 X.509 certificate info

Vendor Information

User Tags

There are many potential user tags available based on the Zeek data fields. The following table indicates both used and potential user tags:

Used Zeek Field Tag Name
domain Domain
id.orig_h SrcIP
id.orig_p SrcPort
id.resp_h DstIP
id.resp_p DstPort
operation Operation
rcode_name rCode Name
status_msg Status Message
dce_rpc Distributed Computing Environment/RPC
_node Zeek Node
_system_name" Zeek System Name
aa Zeek AA
action Action
actions Action
analyzer Threat
answers DNS Answer
assigned_addr DHCP Assigned IP
basic_constraints_ca Basic Constraints CA
certificate_issuer SSL Cert Issuer
certificate_key_alg SSL Key Alg
certificate_key_type SSL Key Type
certificate_sig_alg SSL Sig Alg
certificate_subject SSL Subject
cipher SSL Cipher
cipher_alg SSL Cipher Alg
client SSL Client
client_addr SSL Client Addr
client_cert_subject SSL Cert Subj
client_fqdn SSL Client FQDN
client_message Client Message
compression_alg Compression Alg
content_type Content Type
direction Direction
domain Domain
domainname Domain Name
dst Destination
endpoint Endpoint
error_msg Error Message
extracted Extracted
extracted_cutoff Extracted Cutoff
failure_reason Failure Reason
file_desc File Desc
file_mime_type Mime Type
forwardable Forwardable
helo SMTP Helo
host Host
host_key_alg Host Key Alg
host_name Host Name
host_p Host P
info_msg Info Message
is_orig Is Orig
issuer Issuer
kex_alg Key Exchange Alg
local_orig Local Orig
local_resp Local Resp
mac_alg MAC Alg
mailfrom Mail From
method Method
mime_type Mime Type
mode Mode
msg Message
msg_types Message Type
n Zeek N
name Name
named_pipe Named Pipe
native_file_system Native FS
num_exts Num Exts
orig_filenames Orig Filename
orig_mime_types Orig Mime Type
origin Origin
p Zeek P
password Password
path Path
peer Peer
peer_descr Peer Desc
precision Precision
prev_name Previous Name
proto Protocol
proxied Proxied
qclass qClass
qclass_name qClass Name
qtype_name qType Name
query Query
ra Zeek RA
rcptto Rcpt To
rd Zeek RD
ref_id Referer ID
referer Referer
remote_location_city City
remote_location_country_code Country Code
remote_location_region Region
renewable Renewable
reply_to Reply To
uest_from Request From
request_path Path
request_to Request To
request_type Request Type
resp_filenames File Name
resp_mime_types Mime Type
response_from Resp. From
response_path Resp. Path
response_to Resp. To
resumed Resumed
root_disp Root Disposition
san_dns SAN DNS
san_email SAN Email
san_ip SAN IP
san_uri SAN URI
seq Sequence
server Server
server_addr Server Addr.
server_cert_subject SSL Cert Subj
server_dns_computer_name DNS Name
server_message Server Message
server_name Server Name
server_nb_computer_name Netbios Name
server_tree_name Tree Name
service Service
share_type Share Type
source Source
stratum Stratum
sub Subject
subject Subject
success Success
tags Tags
tc Zeek TC
times_accessed Times Accessed
times_changed Times Changed
times_created Times Created
times_modified Times Modified
unparsed_version Unparsed Version
uri URI
user_agent User Agent
username User
validation_status Validation Status
version Version
version_addl Version Addl
version_major Version Major
version_minor Version Minor
version_minor2 Version Minor2
warning Warning
z Zeek Z

HC Tags

  • SrcIP
  • DstIP
  • Domain
  • Status Message

Log Examples

conn entry corresponding to a basic UDP packet communication

{
  "ts": 1591367999.305988,
  "uid": "CMdzit1AMNsmfAIiQc",
  "id.orig_h": "192.168.4.76",
  "id.orig_p": 36844,
  "id.resp_h": "192.168.4.1",
  "id.resp_p": 53,
  "proto": "udp",
  "service": "dns",
  "duration": 0.06685185432434082,
  "orig_bytes": 62,
  "resp_bytes": 141,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "Dd",
  "orig_pkts": 2,
  "orig_ip_bytes": 118,
  "resp_pkts": 2,
  "resp_ip_bytes": 197
}

ssl entry corresponding to a client initiating an SSL connection

{
  "ts": 1598377391.921726,
  "uid": "CsukF91Bx9mrqdEaH9",
  "id.orig_h": "192.168.4.49",
  "id.orig_p": 56718,
  "id.resp_h": "13.32.202.10",
  "id.resp_p": 443,
  "version": "TLSv12",
  "cipher": "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "curve": "secp256r1",
  "server_name": "www.taosecurity.com",
  "resumed": false,
  "next_protocol": "h2",
  "established": true,
  "cert_chain_fuids": [
    "F2XEvj1CahhdhtfvT4",
    "FZ7ygD3ERPfEVVohG9",
    "F7vklpOKI4yX9wmvh",
    "FAnbnR32nIIr2j9XV"
  ],
  "client_cert_chain_fuids": [],
  "subject": "CN=www.taosecurity.com",
  "issuer": "CN=Amazon,OU=Server CA 1B,O=Amazon,C=US"
}

ssh entry corresponding to an inbound SSH connection

{
  "ts": "2020-09-16T13:29:23.245216Z",
  "uid": "CzEmsljW9ooL0WnBd",
  "id.orig_h": "35.196.195.158",
  "id.orig_p": 53160,
  "id.resp_h": "192.168.4.37",
  "id.resp_p": 22,
  "version": 2,
  "auth_success": true,
  "auth_attempts": 1,
  "direction": "INBOUND",
  "client": "SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2",
  "server": "SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3",
  "cipher_alg": "chacha20-poly1305@openssh.com",
  "mac_alg": "umac-64-etm@openssh.com",
  "compression_alg": "none",
  "kex_alg": "curve25519-sha256",
  "host_key_alg": "ecdsa-sha2-nistp256",
  "host_key": "a3:41:03:32:1f:8c:8e:82:92:9f:62:8c:38:82:d3:74",
  "hasshVersion": "1.0",
  "hassh": "ec7378c1a92f5a8dde7e8b7a1ddf33d1",
  "hasshServer": "b12d2871a1189eff20364cf5333619ee",
  "cshka": "ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa",
  "hasshAlgorithms": "curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c;chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com;umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,zlib@openssh.com,zlib",
  "sshka": "ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519",
  "hasshServerAlgorithms": "curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1;chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com;umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,zlib@openssh.com"
}

Zeek Server-side configuration

Zeek configuration

The syslog-ng config in the section below expects the Zeek logs to be sent in JSON format. On the Zeek side, you may need to enable this log type.

  • edit /opt/zeek/share/zeek/site/local.zeek and set:
# Output in JSON format
@load policy/tuning/json-logs.zeek
  • Run zeekctl deploy to deploy the change

  • check /opt/zeek/logs/current/conn.log to make sure it's in JSON format, for example:

# tail -1 /opt/zeek/logs/current/conn.log
{"ts":1641949013.6772,"uid":"CX1l7X34hCbGkWGlB6","id.orig_h":"192.168.10.107","id.orig_p":36278,"id.resp_h":"192.168.10.255","id.resp_p":32412,"proto":"udp","conn_state":"OTH","local_orig":true,"local_resp":true,"missed_bytes":0,"history":"CC","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":0,"resp_ip_bytes":0}

Syslog-ng Configuration on Zeek

For use with Zeek log files this LogZilla rule requires that syslog be configured to read those Zeek log files and forward the log messages to LogZilla.

WARNING: You may also need to enable the syslog-ng-mod-extra package which provides the syslog-ng type output driver used in the zeek2logzilla.conf below. However, newer versions of syslog-ng have the module in the base package.

For example, if installing on a debian based system:

apt install syslog-ng-mod-extra

The syslog-ng configuration for the machine hosting the Zeek log files should be as follows:

# This is for your *zeek* server (not the LogZilla server)
# filename: /etc/syslog-ng/conf.d/zeek2logzilla.conf
# Zeek log format should look like:
# {"ts":1641946189.335886,"uid":"Ce6ul9J1tSYJNyRga","id.orig_h":"192.168.10.98","id.orig_p":755,"id.resp_h":"192.168.10.99","id.resp_p":2049,"proto":"tcp","conn_state":"OTH","local_orig":true,"local_resp":true,"missed_bytes":0,"history":"CC","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":0,"resp_ip_bytes":0}
#
# Global Options
options {
  flush_lines(100);
  threaded(yes);
  use_dns(yes);
  use_fqdn (no);
  keep_hostname (yes);
  dns-cache-size(2000);
  dns-cache-expire(87600);
};

# Define log sources
# WARNING: DO NOT USE the zeek symlinked directory
# (/opt/zeek/logs/current by default)
# If you do, then when zeek is restarted
# syslog-ng will try to follow/watch the old files
# and not the new ones
source s_zeek_logs {
    wildcard-file(
        base-dir("/opt/zeek/spool/zeek")
        filename-pattern("*.log")
        flags(no-parse)
    );
};

# Set destination (logzilla)
# REPLACE the host "logzilla" below
# with the actual hostname or IP of your LZ server
# test and make sure you can ping/reach the host
destination d_logzilla {
  syslog-ng(server("logzilla") port(514));
};

log {
  source(s_zeek_logs);
  parser { json-parser (prefix(".JSON.")); };
  rewrite { set("zeek" value(".JSON._source_type")); };
  rewrite { set("$(basename ${FILE_NAME})" value(".JSON._source")); };
  destination(d_logzilla);
  flags(flow-control);
};