Skip to content

Syslog Module

The Syslog Forwarder module allows forwarding all or specific matched events to a downstream log receiver.

Enable The Module

To enable the Forwarder Module paste the following command in your LogZilla server's console/ssh terminal:

logzilla config FORWARDER_ENABLED 1

Configure rule(s)

LogZilla allows for multiple forwarding rules to be separated into files for easier administration. For example:

Forward to host A
{
  "window_size": 60,
  "forwarders": [
    {
      "type": "syslog",
      "target": "192.168.0.114:514",
      "transport": "tcp",
      "unsent_buffer_limit": 250000,
      "protocol": "rfc5424",
      "rules": [
        {
          "match": {
            "field": "counter",
            "op": "gt",
            "value": 1
          },
          "rewrite": {
            "message": "$MESSAGE LZ_Forwarded_For=\"$HOST\" LZ_dedupCount=\"$COUNTER\""
          }
        },
        {
          "match": {
            "field": "counter",
            "op": "le",
            "value": 1
          },
          "rewrite": {
            "message": "$MESSAGE LZ_Forwarded_For=\"$HOST\""
          }
        }
      ]
    }
  ],
  "fast_forward_first": true
}
Forward to host B
{
  "window_size": 60,
  "forwarders": [
    {
      "type": "syslog",
      "target": "192.168.0.117:514",
      "transport": "udp",
      "unsent_buffer_limit": 250000,
      "protocol": "bsd",
      "rules": [
        {
          "match": {
            "field": "counter",
            "op": "gt",
            "value": 1
          },
          "rewrite": {
            "message": "$MESSAGE LZ_Forwarded_For=\"$HOST\" LZ_dedupCount=\"$COUNTER\""
          }
        },
        {
          "match": {
            "field": "counter",
            "op": "le",
            "value": 1
          },
          "rewrite": {
            "message": "$MESSAGE LZ_Forwarded_For=\"$HOST\""
          }
        }
      ]
    }
  ],
  "fast_forward_first": true
}
Forward to file
{
  "window_size": 1,
  "fast_forward_first": true,
  "forwarders": [
    {
      "match": {
        "field": "cisco_mnemonic",
        "value": "BGP-5-ADJCHANGE"
      },
      "type": "file",
      "target": "/var/log/logzilla/sec/simple.log",
      "format": "tsv",
      "separator": "\t",
      "fields": [
        "last_occurrence",
        "host",
        "message"
      ]
    }
  ]
}
IMPORTANT: LZ_Forwarded_For

Downstream receivers such as Splunk (See Forwarding to Splunk) will need to know which host the event originated from. This rule adds a key/value pair for the downstream systems to parse and use as the original sending host. Otherwise, all events would appear to come from your local LogZilla server.

Module Options

window_size is the time in seconds to keep each message while checking for its duplicates. The higher number set here the better deduplication will work, but will also delay how often the receiver gets events (every message is kept for NN seconds before being forwarded to the defined target).

pre_match An optional pre_match may also be specified to only forward events matching the defined criteria. If you want to deduplicate and forward all events do not include a match array here.

For example to only forward events containing foo bar=SOMETHING baz and only having severity of 1 or 2 (Alert and Critical):

{
     "window_size" : 60,
     "pre_match" : [
         { "field": "message", "op": "=~", "value": "foo bar=\\S+ baz" },
         { "field": "severity", "value": [1, 2] }
     ],
     "forwarders"...

forwarders This section defines all forwarders. Multiple forwarders and mixing Syslog and SNMP trap destinations may be used. Every element of the forwarders table has a mandatory field type which defines what type of forwarder it is - currently snmp and syslog are supported. Other fields depend on the forwarder type:

For example, the following would forward to both an SNMP Trap receiver and a Syslog receiver:

"forwarders" : [
          {
             "type" : "snmp",
             "target" : "snmp-server:162",
             "trap_oid" : "1.3.6.1.4.1.2021.991",
             "oid_prefix" : "1.3.6.1.4.1.9.9.41.1.2.3",
             "oid_map" : [
                { "type" : "s", "oid" : ".1.2.0", "src" : "facility" },
                { "type" : "i", "oid" : ".1.3.0", "src" : "severity" },
                { "type" : "s", "oid" : ".1.4.0", "src" : "cisco_mnemonic" },
                { "type" : "s", "oid" : ".1.5.0", "src" : "message"  },
                { "type" : "i", "oid" : ".1.99.0", "src" : "counter" }
             ]
          },
          {
             "type" : "syslog",
             "target" : "central-log-collector:514",
             "transport": "tcp",
             "protocol": "bsd"
             "rules": [
                 {
                     "match": {
                         "field": "counter",
                         "op": "gt",
                         "value": 1
                     },
                     "rewrite": {
                         "message": "$MESSAGE LZ_dedupCount=$COUNTER",
                     }
                 }
             ]
          }
     ]

target this is host and port of the target syslog server.

transport either tcp or udp.

unsent_buffer_limit Maximum number of events (post predup) that will be buffered in case destination is down. If the destination comes back up before overflow, events will be forwarded in the original order. Otherwise, the buffer is emptied. Defaults to 25000. Applies only to tcp transport. Note that buffering is enabled after the forwarder realizes that the destination is down, which might be significantly later depending on network communication.

protocol either bsd for the classic (RFC3164) protocol or the newer rfc5424 protocol

octet_count use the octet counting framing method for sending messages

rules This allows you to use rules to specify any matches or rewrites to be applied to the event before it is forwarded. In the example at the top, one rule matches and adds a key/value pair for the source host that sent the event and a second rule for the number of duplicate events detected

fast_forward_first - By default, each original event is forwarded immediately, while duplicates are buffered for a time specified by window_size. This option allows for downstream receivers to get any critical events while still benefiting from deduplication. Using "fast_forward_first": false means that the original events will be buffered and included in the duplicated event forwarded at the end of the window.

2. Add Your Config(s)

Add the file to LogZilla NEO using:

cp fwd-to-host-a.json /etc/logzilla/forwarder.d
cp fwd-to-host-b.json /etc/logzilla/forwarder.d
cp fwd-to-file.json /etc/logzilla/forwarder.d

3. Restart Processes

Restart the forwarder and parser

logzilla restart -c forwardermodule 
logzilla restart -c parsermodule

3a. Updating forwarding rules

Future changes to the forwarder will require a restart of only the Forwarder Module using

logzilla restart -c forwardermodule