Syslog Module
The Syslog Forwarder module allows forwarding all or specific matched events to a downstream log receiver.
Enable The Module
To enable the Forwarder Module paste the following command in your LogZilla server's console/ssh terminal:
Configure rule(s)
LogZilla allows for multiple forwarding rules to be separated into files for easier administration. For example:
Forward to host A
{
"window_size": 60,
"forwarders": [
{
"type": "syslog",
"target": "192.168.0.114:514",
"transport": "tcp",
"unsent_buffer_limit": 250000,
"protocol": "rfc5424",
"rules": [
{
"match": {
"field": "counter",
"op": "gt",
"value": 1
},
"rewrite": {
"message": "$MESSAGE LZ_Forwarded_For=\"$HOST\" LZ_dedupCount=\"$COUNTER\""
}
},
{
"match": {
"field": "counter",
"op": "le",
"value": 1
},
"rewrite": {
"message": "$MESSAGE LZ_Forwarded_For=\"$HOST\""
}
}
]
}
],
"fast_forward_first": true
}
Forward to host B
{
"window_size": 60,
"forwarders": [
{
"type": "syslog",
"target": "192.168.0.117:514",
"transport": "udp",
"unsent_buffer_limit": 250000,
"protocol": "bsd",
"rules": [
{
"match": {
"field": "counter",
"op": "gt",
"value": 1
},
"rewrite": {
"message": "$MESSAGE LZ_Forwarded_For=\"$HOST\" LZ_dedupCount=\"$COUNTER\""
}
},
{
"match": {
"field": "counter",
"op": "le",
"value": 1
},
"rewrite": {
"message": "$MESSAGE LZ_Forwarded_For=\"$HOST\""
}
}
]
}
],
"fast_forward_first": true
}
Forward to file
{
"window_size": 1,
"fast_forward_first": true,
"forwarders": [
{
"match": {
"field": "cisco_mnemonic",
"value": "BGP-5-ADJCHANGE"
},
"type": "file",
"target": "/var/log/logzilla/sec/simple.log",
"format": "tsv",
"separator": "\t",
"fields": [
"last_occurrence",
"host",
"message"
]
}
]
}
IMPORTANT: LZ_Forwarded_For
Downstream receivers such as Splunk (See Forwarding to Splunk) will need to know which host the event originated from. This rule adds a key/value pair for the downstream systems to parse and use as the original sending host. Otherwise, all events would appear to come from your local LogZilla server.
Module Options
window_size
is the time in seconds to keep each message while checking for its
duplicates. The higher number set here the better deduplication will
work, but will also delay how often the receiver gets events (every message is kept
for NN seconds before being forwarded to the defined target).
pre_match
An optional pre_match may also be specified to only forward events matching the defined criteria. If you want to deduplicate and forward all events do not include a match array here.
For example to only forward events containing foo bar=SOMETHING baz and only having severity of 1 or 2 (Alert and Critical):
{
"window_size" : 60,
"pre_match" : [
{ "field": "message", "op": "=~", "value": "foo bar=\\S+ baz" },
{ "field": "severity", "value": [1, 2] }
],
"forwarders"...
forwarders
This section defines all forwarders. Multiple forwarders and mixing Syslog and SNMP trap
destinations may be used. Every element of the forwarders table has a mandatory field type
which defines what type of forwarder it is - currently snmp and syslog are supported.
Other fields depend on the forwarder type:
For example, the following would forward to both an SNMP Trap receiver and a Syslog receiver:
"forwarders" : [
{
"type" : "snmp",
"target" : "snmp-server:162",
"trap_oid" : "1.3.6.1.4.1.2021.991",
"oid_prefix" : "1.3.6.1.4.1.9.9.41.1.2.3",
"oid_map" : [
{ "type" : "s", "oid" : ".1.2.0", "src" : "facility" },
{ "type" : "i", "oid" : ".1.3.0", "src" : "severity" },
{ "type" : "s", "oid" : ".1.4.0", "src" : "cisco_mnemonic" },
{ "type" : "s", "oid" : ".1.5.0", "src" : "message" },
{ "type" : "i", "oid" : ".1.99.0", "src" : "counter" }
]
},
{
"type" : "syslog",
"target" : "central-log-collector:514",
"transport": "tcp",
"protocol": "bsd"
"rules": [
{
"match": {
"field": "counter",
"op": "gt",
"value": 1
},
"rewrite": {
"message": "$MESSAGE LZ_dedupCount=$COUNTER",
}
}
]
}
]
target
this is host and port of the target syslog server.
transport
either tcp or udp.
unsent_buffer_limit
Maximum number of events (post predup) that will be buffered in case
destination is down. If the destination comes back up before overflow,
events will be forwarded in the original order. Otherwise, the buffer is emptied.
Defaults to 25000. Applies only to tcp transport.
Note that buffering is enabled after the forwarder realizes that the destination is
down, which might be significantly later depending on network communication.
protocol
either bsd for the classic (RFC3164) protocol or the newer rfc5424 protocol
octet_count
use the octet counting framing method for sending messages
rules
This allows you to use rules to specify any matches or rewrites to be applied
to the event before it is forwarded. In the example at the top, one rule matches and
adds a key/value pair for the source host that sent the event and a second rule for the
number of duplicate events detected
fast_forward_first - By default, each original event is forwarded immediately,
while duplicates are buffered for a time specified by window_size. This
option allows for downstream receivers to get any critical events while still benefiting
from deduplication. Using "fast_forward_first": false means that the original events
will be buffered and included in the duplicated event forwarded at the end of the window.
2. Add Your Config(s)
Add the file to LogZilla NEO using:
cp fwd-to-host-a.json /etc/logzilla/forwarder.d
cp fwd-to-host-b.json /etc/logzilla/forwarder.d
cp fwd-to-file.json /etc/logzilla/forwarder.d
3. Restart Processes
Restart the forwarder and parser
3a. Updating forwarding rules
Future changes to the forwarder will require a restart of only the Forwarder Module using