Monitor Linux Hosts using Nagios check_by_ssh Plugin

Welcome to our tutorial on how to monitor Linux Hosts using Nagios check_by_ssh plugin. This enables Nagios Server to monitor system metrics and services on remote Linux server using SSH.

We have covered various guides on Nagios Monitoring in our previous articles;

Nagios SNMP Monitoring of Linux Hosts on AlienVault USM/OSSIM

How to Install Nagios Plugins and NRPE agents on CentOS 7/RHEL 7/Fedora 29

Configure Nagios Availability Monitoring on AlienVault USM/OSSIM

How to Install Nagios Plugins From Source RHEL/CentOS/Oracle Linux

How to Install Nagios NRPE Agent on RHEL/CentOS/Oracle Linux

How to Install and Configure NSClient++ Nagios Agent on Windows System

How to Install and Configure Nagios Core From the Source Ubuntu 18.04

How to Install and Configure Nagios Core From repo Ubuntu 18.04

Monitoring Linux Hosts using Nagios check_by_ssh Plugin

In order to monitor Linux hosts using the Nagios check_by_ssh plugin, there are a few prerequisites that must be met.

• Create a Nagios user for monitoring on the host to be monitored.
• Configure passwordless SSH authentication as nagios user on host to monitor.
• Install Nagios plugins on the host to monitor.

Creating a Nagios user

Login to the host to monitor and create a user called nagios and set the login password.

useradd -m nagios

passwd nagios

SSH Public-key Authentication

In order to read the remote system metrics or run monitoring scripts, the Monitoring server has to login to the target host using nagios user created above. Hence, you need to configure passwordless authentication via the use of SSH keys. Want to read more about SSH configuration? Check our articles by following the links below;

• Allow/Deny Specific Users to Login via SSH on Ubuntu 18.04
• Configure SSH Public Key Authentication in Linux

Generate SSH Keys

Run the command below to generate SSH keys on the monitoring server, AlienVault OSSIM in my case. When prompted to enter the password, press enter to leave the password blank.

ssh-keygen

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/nagios
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/nagios.
Your public key has been saved in /root/.ssh/nagios.pub.
The key fingerprint is:
b3:7d:b2:26:8b:1e:84:c1:20:a8:53:1b:ab:32:20:ce root@alienvault
The key's randomart image is:
+---[RSA 2048]----+
|o .              |
|..oo             |
|.. +o            |
|= o  o           |
|=o  . . S        |
|+E   .   +       |
|..    . . o .    |
|       o. .+     |
|     .o .+.      |
+-----------------+

Note that I saved my SSH keys for connecting to nagios under the /root/.ssh/nagios file instead of the default, /root/.ssh/id_rsa. As a result, you need to tell SSH where to find the identity key file for the nagios user on the target host. This can be done by creating the identity configuration file under the /etc/ssh/ssh_config or .ssh/config in the user’s home directory. In this case, we are going to use the later where our user is root.

vim /root/.ssh/config

Add the lines below;

Host    192.168.43.176                   # Ip address of the remote host
        IdentityFile    /root/.ssh/nagios  # SSH key file

Copy the SSH public key to nagios user on the target host

ssh-copy-id -i /root/.ssh/nagios [email protected]

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password: Nagios User Password on Remote Host

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

To verify that you cannot be prompted for password when logging in as nagios, try to log in.

ssh '[email protected]'

You can also disable agent forwarding, port forwarding or tty allocation for the connections fromt the monitoring host to the nagios user. This can be done by specifying the following options in the authorized_keys file for the nagios user just before the keyword, ssh-rsa on the SSH key.

from "192.168.43.200",no-pty,no-agent-forwarding,no-port-forwarding

Where 192.168.43.200 is the IP of my Nagios Server (OSSIM) in this case. Check man sshd for further exaplanation of these options.

vim /home/nagios/.ssh/authorized_keys

from "192.168.43.200",no-pty,no-agent-forwarding,no-port-forwarding ssh-rsa AAAAB3NzaC1yc2EAA...StPEz root@alienvault

As a security measure, disable password login for nagios user since login will be made via the use of SSH keys.

vim /etc/ssh/sshd_config

Add the line below to the end of the SSHd configuration file.

Match   User    nagios
        PasswordAuthentication no

Reload SSH

systemctl reload ssh

You can learn more about disabling SSH password authentication for specific users in our previous guide.

• Disable SSH Password Login for Specific Users in Ubuntu 18.04

Test Monitoring with check_by_ssh

Most of the configurations are done and your remote server is ready for monitoring using check_by_ssh command. As an example, try to query the system uptime.

/usr/lib/nagios/plugins/check_by_ssh -H 192.168.43.176 -C "uptime" -l nagios
 16:00:46 up  2:13,  3 users,  load average: 0.07, 0.06, 0.01

Install Nagios Plugins

In order to utilize Nagios monitoring scripts on the target host, install the nagios plugins. If you are running an ubuntu system, install the plugins by running the command below;

apt install nagios-plugins

This installs the monitoring scripts to /usr/lib/nagios/plugins/.

For CentOS/Fedora or any RHEL based, follow the links below;

• How to Install Nagios Plugins From Source RHEL/CentOS/Oracle Linux
• How to Install Nagios Plugins and NRPE agents on CentOS 7/RHEL 7/Fedora 29

If you cannot install the plugins on the host for some reason, then you can copy the monitoring scripts from the Nagios server. For example;

scp /usr/lib/nagios/plugins/check_procs [email protected]:

To monitor the host with check_by_ssh plugin, run the command from the Nagios server (or your monitoring host) specifying the script to execute on the remote host. For example, to monitor disk usage (/ partition) and prints the usage in megabytes, run the command below.

/usr/lib/nagios/plugins/check_by_ssh -H 192.168.43.176 -C "/usr/lib/nagios/plugins/check_disk -w 80% -c 90% -p / -m" -l nagios
DISK OK - free space: / 99307 MB (94% inode=98%);| /=5402MB;22071;11035;0;110358

This assumes that the Nagios check_disk script is located under the default install location.

If you have the scripts in different directories, say home directory, execute the command as in below;

/usr/lib/nagios/plugins/check_by_ssh -H 192.168.43.176 -C "/home/nagios/check_procs -w 100 -c 200" -l nagios
PROCS WARNING: 152 processes | procs=152;100;200;0;

To alert of any process consumes more than 80% or 90% CPU;

/usr/lib/nagios/plugins/check_by_ssh -H 192.168.43.176 -C "/home/nagios/check_procs -w 80 -c 90 -m CPU" -l nagios
CPU OK: 152 processes | procs=152;;;0; procs_warn=0;;;0; procs_crit=0;;;0;

To make this easy for Nagios Monitoring, you need to create command definitions for various services or metrics you need to monitor.

In the command definition configuration file, you define the command as;

define command{
        command_name check_running_procs
        command_line /usr/lib/nagios/plugins/check_by_ssh -H $HOSTADDRESS$ -C "/home/nagios/check_procs -w $ARG1$ -c $ARG2$" -l nagios
}

The service definition will look like as shown below;

define service{
        use                     your-service_template
        hostgroup_name          <your group name> # can be a single host, host_name.
        service_description     Running Processes
        check_command           check_running_procs!100!200
        }

That is all on monitoring Linux hosts using Nagios check_by_ssh plugin.