Page Comparison

...

• UDP
• TCP (you can also encrypt logs over with TLS)
• RELP (uses application-level acknowledgement for increased reliability over plain TCP)
• HTTP / HTTPS over the Elasticsearch API

...

There are 3 steps for configuring your rsyslog for Logsene:

1. Choose Configure one or more inputs. For example, set configure rsyslog to listen for upload local logs, remote logs over TCP and so on
2. Choose a protocol and an authentication method. For UDP, TCP, TLS and RELP you can authorize your public IP. However, we strongly recommend using the Logsene application's token, which works with all supported protocols
3. Configure the output. Based on the chosen protocol and authentication method, you'll have to configure the appropriate output plugin to send logs in the desired format

Configuring Inputs

First, make configure rsyslog to receive logs that you want 'd like to send upload to Logsene. The most common input module is imuxsock, which will take logs from your local /dev/log socket. To start listening on local logs, add this line at the beginning of your /etc/rsyslog.conf:

...

To tail a file, load the file input module, and  and optionally decide how often to look pool for changes. Then, for every file, specify its path and a few more related parameters, like whether you want multiline supportthis:

# add once
$ModLoad imfile                     # load the file input module
$InputFilePollInterval 1            # how often to check files for changes

# for every file you want to monitor
$InputFileName /var/log/jetty.log   # the file to monitor
$InputFileTag jetty:                # syslog tag attributed to those events. Yes, the trailing ":" should be there.
$InputFileStateFile jetty           # state file to remember where it left between restarts. This should not be the full path to a file, just a name.
$InputFileReadMode 2                # support indented multi-linemultiline logs (requires rsyslog 5.7.5+). For single-line logs use 0. For multi-linemultiline logs without indent support use 1.
$InputRunFileMonitor                # start monitoring this file

If you have issues with logrotate or other utilities that truncate or move organize the files you monitor, upgrade rsyslog to version 8.1.5 or later and the problems should go away. The file input module gets inotify support , and you also have a new configuration format, that's  at your disposal, which is easier to maintain:

# add once
module(load="imfile")

# for every file
input(type="imfile"
  File="/var/log/jetty.log"         # the file to monitor
  Tag="jetty:"                       # syslog tag attributed to those events
  ReadMode="2"                      # support indented multi-linemultiline logs (requires rsyslog 5.7.5+). For single-line logs use 0. For multi-linemultiline logs without indent support use 1.
)

...

To forward logs, you can use HTTP/HTTPS and authenticate by using your Logsene application token (recommended!). Alternatively, you can use UDP, TCP/TLS or RELP and authenticate either by using the Logsene application token, or by authorizing your public IP from in the Logsene UIapplication settings.

HTTP/HTTPS via the Elasticsearch API

The recommended method is to use the Elasticsearch API and to send logs over HTTP or HTTPS. This will give you maximum flexibility, reliability and encryption, if you need it.  This requires This requires rsyslog 6.4.0 or later, and the installation of the Elasticsearch output module. HTTPS is supported in rsyslog 8.2.0 or later (see info about rsyslog update above).  To To enable the Elasticsearch output module, install the rsyslog-elasticsearch package or use --enable-elasticsearch when compiling from sources.

Configuration

Before forwarding logs via the Elasticsearch API, define a template in /etc/rsyslog.conf that makes gives structure to your syslog messages by formatting them as JSON:

# define a template to specify which fields we send
template(name="LogseneFormat" type="list" option.json="on") {
  constant(value="{")
  constant(value="\"@timestamp\":\"")
  property(name="timereported" dateFormat="rfc3339")
  constant(value="\",\"message\":\"")
  property(name="msg")
  constant(value="\",\"host\":\"")
  property(name="hostname")
  constant(value="\",\"severity\":\"")
  property(name="syslogseverity-text")
  constant(value="\",\"facility\":\"")
  property(name="syslogfacility-text")
  constant(value="\",\"syslog-tag\":\"")
  property(name="syslogtag")
  constant(value="\",\"source\":\"")
  property(name="programname")
  constant(value="\"}")
}

UDP, TCP, TLS or RELP

You can send RFC-3164 or and RFC-5424 compatible syslog to Logsene via any of the following protocols:

• UDP: this is the fire-and-forget protocol that doesn't guarantee any reliability but is also the fastest and simplest implementation
• TCP: this provides better reliability than TCP, though messages might still be lost from the system buffers if the connection breaks and rsyslog is restarted
• TLS: RFC-5425 syslog over TLS is supported
• RELP: this is a reliable protocol, because it supports application-level acknowledgments

With Again, with all these protocols, you can either authenticate with your Logsene application token, or by registering your public IP.

...

UDP and TCP support is built in to into rsyslog. For TLS, you need to install the gtls driver, typically provided by the rsyslog-gnutls package , (or by adding --enable-gnutls when compiling rsyslog from sourcessource). For  For RELP, you'll need the RELP output module, normally usually provided by the rsyslog-relp package (or by adding --enable-relp when compiling from sourcessource).

Configuration

To use the If you chose to authorize using static IP address, instead of authenticating using Logsene application token (which is the recommended option), you don'll first have to get t need to make any configuration changes in this step. Instead, go to the Logsene web application and authorize the public IP (or multiple IPs) of the server(s) from where you send your logs.

To use the Logsene application token, you'll first have to obtain it from your list of Logsene applications. Then, in your /etc/rsyslog.conf, define a template in /etc/rsyslog.conf that  that forwards your syslog messages in CEE-formatted JSON over syslog, where you should put your token in the logsene-app-token field:

$template LogseneFormat,"<%PRI%>%TIMEREPORTED:::date-rfc3339% %HOSTNAME% %syslogtag%@cee: {\"logsene-app-token\": \"LOGSENE_APP_TOKEN_GOES_HERE\", \"message\": \"%msg:::json%\"}\n"

If you 're are using rsyslog version 7 or later, you can use the new configuration format to define the template. It's more verbose, but also easier to easier to maintain (e.g. add new fields if you want to, reformat messages):

 template(
  name="LogseneFormat"
  type="list"
) {
  # standard syslog fields
  constant(value="<")
    property(name="pri")
  constant(value=">")
  property(name="timereported" dateFormat="rfc3339")
  constant(value=" ")
    property(name="hostname")
  constant(value=" ")
  property(name="syslogtag")
  # CEE-formatted JSON message over syslog
  constant(value="@cee: {\"logsene-app-token\": \"LOGSENE_APP_TOKEN_GOES_HERE\", \"message\": \"")
  property(name="msg" format="json")                        # original syslog message goes in the "message" field
  constant(value="\"}\n")
}

If you prefer authorizing IPs to using tokens, you don't need to make any configuration change for this step. Instead, go to the Logsene UI and authorize the public IP (or IPs) from which you send your logs.

Configuring Outputs

The last step is about configuring the to configure an output module to send for sending your logs to Logsene. This depends on your chosen protocol.

...

To send your logs over HTTP, load the Elasticsearch output module and then point it to logsene-receiver.sematext.com on port 80. Make sure you replace LOGSENE_APP_TOKEN_GOES_HERE with your actual token:

...

If you want to encrypt logs, by sending them over HTTPS, you'll have to change serverport to "443" and add usehttps="on".

UDP

If you're using the Logsene application token for authentication, specify the LogseneFormat template in your action line. The host you'll connect to is logsene-receiver-syslog.sematext.com:

...

With TCP, the action line looks similar to UDP, except you'll have two @ signs. This is for when your using the

If you're authorizing using Logsene application token:

*.* @@logsene-receiver-syslog.sematext.com;LogseneFormat

This is for when you authorize IPsIf you use IP address for authorization:

*.* @@logsene-receiver-syslog.sematext.com;RSYSLOG_SyslogProtocol23Format

...

To set up syslog over TLS, you first set up need to configure the certificates:

mkdir /opt/rsyslog  # if it does not already exist
cd /opt/rsyslog
wget https://apps.sematext.com/cert/DigiCertCA.pem               # md5sum is fb30c5636d0108b2688d7e1ed59749ac
wget https://apps.sematext.com/cert/DigiCert_Global_Root_CA.pem  # md5sum is 3816293340b05c52bcbc99a4f00b1b04
cat {DigiCert_Global_Root_CA.pem,DigiCertCA.pem} > ca_bundle.pem

Then, configure the TLS driver like this:

$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /opt/rsyslog/ca_bundle.pem
 
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.sematext.com
$ActionSendStreamDriverMode 1

Finally, add the action line rsyslog action, which is the same as the TCP one, except you'll use port 10514. For

Again, there are two options, for authorizing with the Logsene application token:

*.* @@logsene-receiver-syslog.sematext.com:10514;LogseneFormat

And for authorizing using the IPsIP address(es):

*.* @@logsene-receiver-syslog.sematext.com:10514;RSYSLOG_SyslogProtocol23Format

If you prefer the new configuration format, you can find the whole complete TLS configuration below:

 global (
 defaultNetstreamDriver="gtls"
 defaultNetstreamDriverCAFile="/opt/rsyslog/ca_bundle.pem"
)
 
action(
  type="omfwd"
  target="logsene-receiver-syslog.sematext.com"
  port="10514"
  protocol="tcp"
  template="LogseneFormat"               # use "RSYSLOG_SyslogProtocol23Format" for IP authorization
  StreamDriverMode="1"
  StreamDriverPermittedPeers="*.sematext.com"
  StreamDriverAuthMode="x509/name"
)

...

To forward via RELP, load the RELP output module and then point it to logsene-receiver-syslog.sematext.com on port 20514.

As with TCP or UDP, specify the LogseneFormat template for authorizing with your Logsene application token:

$ModLoad omrelp
*.* :omrelp:logsene-receiver-syslog.sematext.com:20514;LogseneFormat

And the RFC-5424 predefined template for IP-based authorization:

...

From your syslog messages, Logsene will populate a number of special fields, such as the source and host. You can also configure rsyslog to add one or more tags to logs matching certain criteria. This is useful when you want to quickly identify a special kind of logs. For example, you could tag events that come to the "kernel" facility with a severity/level of "error" as both "kernel errors" and "urgent issues". Tagging information will be visible in the Logsene web application, which you can then use for filtering, sorting, ...

To achieve this, define a template similar to the ones described above, where you'd add a tags field. Then, you'd use a conditional to match those messages and send them to Logsene using the newly defined template:

$template kernelErrors,"<%PRI%>%TIMEREPORTED:::date-rfc3339% %HOSTNAME% %syslogtag%@cee: {\"logsene-app-token\": \"LOGSENE-APP-TOKEN-GOES-HERE\", \"message\": \"%msg:::json%\", \"tags\":[\"kernel errors\", \"urgent issues\"]}\n"

if $syslogfacility-text == 'kernel' and $syslogseverity-text == 'error' then @@(o)logsene-receiver-syslog.sematext.com;kernelErrors
&~

Notice the &~ statement - this prevents rsyslog from sending matched events twice (once with tags and once without). Make sure you place these conditionals before your main Logsene action (the one starting with *.* ).