In this guide, we are going to learn how to configure NXLog to forward system logs to Rsyslog server on Ubuntu 18.04. There are various NXLog log collection solutions. In this guide, we are going to configure the opensource version of NXLog.
Configure NXLog to Forward System Logs to Rsyslog Server on Ubuntu 18.04
NXLog is not available on the default Ubuntu 18.04 repositories. Hence, we are going to download the DEB package and install it with dpkg package manager. To download NXLog install binary, navigate to NXLog community edition downloads page and grab one for Ubuntu 18.04.
wget https://nxlog.co/system/files/products/files/348/nxlog-ce_2.10.2150_ubuntu_bionic_amd64.deb
You can also check our previuos articles on configuration of Rsyslog and Syslog by following the links below;
- Configure Rsyslog on Solaris 11.4 to Send logs to Remote Log Server
- Configure Syslog on Solaris 11.4 for Remote Logging
- How to Configure Remote Logging with Rsyslog on Ubuntu 18.04
Install NXLog on Ubuntu 18.04
Once you have downloaded the DEB binary, install it as shown below;
sudo apt install ./nxlog-ce_2.10.2150_ubuntu_bionic_amd64.deb
Configure NXLog to Forward Logs on Ubuntu 18.04
Now that NXLog CE has been installed, you need to configure it to forward logs to the remote Rsyslog server. The default configuration file for NXLog CE is /etc/nxlog/nxlog.conf.
NXLog can be configured to receive and read logs from different types of sources including;
- log data received over the network
- events stored in databases
- messages read from files
- data retrieved using executables
This guide focuses on configuring NXLog CE to receive, read and forward logs from system log files to a remote logging server. Therefore, make a backup of the original configuration so that you can make any adjustments that suit your environment.
mv /etc/nxlog/nxlog.conf /etc/nxlog/nxlog.conf.original
Create a new configuration file.
touch /etc/nxlog/nxlog.conf
The NXLog configuration file consists of global directives, module instances, and routes. To begin with, set the ROOT to the main directory of NXLog. This is a general
# Set the NXLog main directory define ROOT /etc/nxlog
Define the Global directives
There are quite a number of global directives that can be set. However, in its simplest, we will define the NXLog modules directory, directory to write the cached data, the logging level, NXLog PID file, NXLog working directory, NXLog log file etc. This can be done by using the following directives respectively; ModuleDir, CacheDir, LogLevel, PidFile, SpoolDir.
# Global Directives Moduledir /usr/lib/nxlog/modules CacheDir %ROOT%/data SpoolDir %ROOT%/data Pidfile /tmp/nxlog.pid LogFile /var/log/nxlog/nxlog.log
Some of the other important global directives include User and Group. NXLog runs as user nxlog by default with the limitation being this user cannot read the /var/log directory where most system logs are written to. To circumvent this, NXlog can be set to run as root by omitting the User option. However, it is more secure if you can add NXlog user to a group with permissions to read the log file.
Define Input Module Directives
Basically, we are going to define basic input modules directives that read various log files for sending to a remote log server.
# Define Input Modules
<Input in1>
Module im_file
File "/var/log/auth.log"
SavePos TRUE
ReadFromLast TRUE
</Input>
<Input in2>
Module im_file
File "/var/log/syslog"
SavePos TRUE
ReadFromLast TRUE
</Input>
The SavePos directive ensures that log file state is cached when NXLog exits. The log file will be read from that position when NXLog starts. This can however be turned off by using NoCache directive. The ReadFromLast directive ensures that NXLog reads the logs received after NXLog started.
Define the Processor Modules
Processor modules are used process logs between the Input and Output modules. This can be achieved by use of pm_buffer modules which supports both disk and memory log buffering. In this guide, we are going to set disk buffering. You also need to set the maximum size of the logs that can be buffered.
#Define Processor Modules <Processor buffer> Module pm_buffer MaxSize 512000 # Buffer logs upto 512MB Type disk # Disk buffering </Processor>
Define the Output Module
There are different types of output modules. We are going to set UDP as our output module. You can check about other modules here. You need to set the remote Host IP and Port.
# Define Output Modules
<Output udp>
Module om_udp
Host 192.168.43.208
Port 514
</Output>
Define the Route Directives
Define the data flow using the Path directive. More than one Input feeding logs into the route are comma separated. The list of Input modules is followed by an arrow (=>). Processor modules or Output modules follow after. Multiple Processors are separated by arrows. The syntax is;
Path INPUT1[, INPUT2...] => [PROCESSOR1 [=> PROCESSOR2...] =>]
OUTPUT1[, OUTPUT2...]
Hence, we can define this in our configuration file as;
# Route definition
<Route 1>
Path in1,in2 => buffer => udp
</Route>
That is all about our configuration in its simplest form. In general, it should look like;
# Set the NXLog main directory
define ROOT /etc/nxlog
# Global Directives
Moduledir /usr/lib/nxlog/modules
CacheDir %ROOT%/cache_dir
SpoolDir %ROOT%/spool_dir
Pidfile /tmp/nxlog.pid
LogFile /var/log/nxlog/nxlog.log
# Define Input Modules
<Input in1>
Module im_file
File "/var/log/auth.log"
SavePos TRUE
ReadFromLast TRUE
</Input>
<Input in2>
Module im_file
File "/var/log/syslog"
SavePos TRUE
ReadFromLast TRUE
</Input>
#Define Processor Modules
<Processor buffer>
Module pm_buffer
MaxSize 512000 # Buffer logs upto 512MB
Type disk # Disk buffering
</Processor>
# Define Output Modules
<Output udp>
Module om_udp
Host 192.168.43.188 # IP of Rsyslog Server
Port 514
</Output>
# Route Definition
<Route 1>
Path in1,in2 => buffer => udp
</Route>
Create the Cache and Spool Directories.
mkdir -p /etc/nxlog/{cache_dir,spool_dir}
Restart NXLog and set it to run on system boot.
systemctl restart nxlog systemctl enable nxlog
Check the status.
systemctl status nxlog
* nxlog.service - LSB: logging daemon Loaded: loaded (/etc/init.d/nxlog; generated) Active: active (running) since Sat 2019-03-16 19:23:19 EAT; 53s ago Docs: man:systemd-sysv-generator(8)
Confirm that you can receive logs on the remote server. As a POC, ssh into Ubuntu 18.04 server with nxlog running from a different server. Tail the logs on remove Rsyslog server and there you go.
tail -f /var/log/remotelogs/192.168.43.203.log 2019-03-16T19:32:40-04:00 u18svr sshd[21327]: Connection closed by 127.0.0.1 port 50630 [preauth] 2019-03-16T19:34:12-04:00 u18svr sshd[21335]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.43.149 user=amos 2019-03-16T19:34:14-04:00 u18svr sshd[21335]: Failed password for amos from 192.168.43.149 port 48224 ssh2 2019-03-16T19:34:18-04:00 u18svr sshd[21335]: Accepted password for amos from 192.168.43.149 port 48224 ssh2 2019-03-16T19:34:18-04:00 u18svr sshd[21335]: pam_unix(sshd:session): session opened for user amos by (uid=0) 2019-03-16T19:34:18-04:00 u18svr systemd-logind[581]: New session 26 of user amos. ...
That is all about how to configure NXLog to forward system logs to Rsyslog server on Ubuntu 18.04. Feel free to read more about NXLog on their reference manual.
Related Tutorials
Install and Configure NXLog CE on Ubuntu 20.04
Configure Rsyslog on Solaris 11.4 to Send logs to Remote Log Server
Configure Syslog on Solaris 11.4 for Remote Logging
How to Configure Remote Logging with Rsyslog on Ubuntu 18.04
