Enroll Osquery Hosts on Fleet Manager

March 20, 2021

612

In this tutorial, you will learn how to add or enroll Osquery hosts on Fleet manager. Fleet is the most widely used open source osquery manager. Deploying osquery with Fleet enables programmable live queries, streaming logs, and effective management of osquery across 50,000+ servers, containers, and laptops. It’s especially useful for talking to multiple devices at the same time.“

In our previous guide, we learnt how to install Osquery Fleet manager on Ubuntu 20.04.

Install Fleet Osquery Manager on Ubuntu 20.04

Enroll Osquery Hosts on Fleet Manager

Install Osquery on hosts

Before you can add or enroll Osquery hosts on Fleet manager, you need to have installed Osquery on the remote hosts.

Below are some guides you can follow to install Osquery;

Install Osquery on Ubuntu 20.04

Install Osquery on Debian 10 Buster

Enroll Osquery Hosts on Fleet Manager

After you have installed Osquery on the hosts, you can then proceed to enroll them on Fleet manager.

Therefore, login to your Fleet manager web interface.

To add or enroll a host, click Add new host. This opens up such an wizard.

Download:

Enrollment secret: Provide an active enroll secret to allow osquery to authenticate with the Fleet server.

Server certificate: Provide the TLS certificate used by the Fleet server to enable secure connections from osquery.
Flag File: flags to control initialization, disable/enable features, and select plugins.

Once the files above are downloaded, copy them to the Osquery host.

For example, in my setup, i have copied the files to home directory of specific user account on my Ubuntu 20.04 server;

ls ~/ -1

flagfile.txt
fleet.pem
secret.txt

Navigate to the directory where the files above are stored on the osquery host, in case it is my users home directory.

cd ~/

Enroll Osquery host on Fleet manager by running the command below. (Before you can run the command below, ensure that the Osquery Fleet manager hostname that you generated the SSL certs from is resolvable from the host)

sudo osqueryd --flagfile=flagfile.txt --verbose

You will some system output.

...
I0320 16:53:40.973621  5419 init.cpp:340] osquery initialized [version=4.6.0]
I0320 16:53:40.993187  5419 system.cpp:342] Found stale process for osqueryd (4742)
I0320 16:53:40.993357  5419 system.cpp:374] Writing osqueryd pid (5419) to /var/run/osqueryd.pidfile
I0320 16:53:40.993547  5419 extensions.cpp:453] Could not autoload extensions: Cannot open file for reading: /etc/osquery/extensions.load
I0320 16:53:40.993702  5419 dispatcher.cpp:78] Adding new service: WatcherRunner (0x55de416f5f08) to thread: 139824272353024 (0x55de417114e0) in process 5419
I0320 16:53:40.994439  5420 watcher.cpp:613] osqueryd watcher (5419) executing worker (5421)
I0320 16:53:41.004231  5421 init.cpp:337] osquery worker initialized [watcher=5419]
I0320 16:53:41.004514  5421 dispatcher.cpp:78] Adding new service: WatcherWatcherRunner (0x55b07d7a9d98) to thread: 140124192265984 (0x55b07d79e8a0) in process 5421
I0320 16:53:41.004647  5421 rocksdb.cpp:132] Opening RocksDB handle: /var/osquery/osquery.db
I0320 16:53:42.638896  5421 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0x55b07d8a64c8) to thread: 140123990906624 (0x55b07d892450) in process 5421
I0320 16:53:42.639508  5421 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0x55b07d820758) to thread: 140123999299328 (0x55b07d79a060) in process 5421
I0320 16:53:42.640108  5421 tls_enroll.cpp:70] TLSEnrollPlugin requesting a node enroll key from: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/enroll
I0320 16:53:42.640694  5421 system.cpp:301] Using host identifier: f68ce9e5-caf4-4b2e-85c0-e1cd4304abb0
I0320 16:53:42.641921  5520 interface.cpp:270] Extension manager service starting: /var/osquery/osquery.em
I0320 16:53:42.642381  5421 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0320 16:53:42.662926  5421 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0320 16:53:42.664144  5421 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/enroll
I0320 16:53:43.371500  5421 auto_constructed_tables.cpp:97] Removing stale ATC entries
I0320 16:53:43.372932  5421 dispatcher.cpp:78] Adding new service: ConfigRefreshRunner (0x55b07d81d508) to thread: 140124007692032 (0x55b07d7a5680) in process 5421
I0320 16:53:43.374224  5421 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/config
...
...
I0320 17:00:07.842394  5521 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/config
I0320 17:00:08.261268  5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:09.360673  5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:13.452287  5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:15.192095  5522 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/log

The host now communicates with the Osquery Fleet manager and it should be enrolled.

Navigate to Osquery Fleet Manager and refresh the web interface. You should be able to see you host enrolled.

Running Osqueryd as a Service

To ensure a constant communication between the Osquery host and the Fleet manager, you need to run osqueryd as a service.

Therefore, stop the standalone process initiated above by pressing Ctrl+c.

Update the osqueryd service unit file configurations as follows.

First of all, let us move the secret, the certificate and the flag files to /etc/osquery directory;

sudo mv ~/{flagfile.txt,fleet.pem,secret.txt} /etc/osquery

Next, update the path to FLAG_FILE environment variable in the osqueryd service defaults file, /etc/default/osqueryd.

sudo vim /etc/default/osqueryd

Check the highlighted line. Replace the path accordingly.

FLAG_FILE="/etc/osquery/flagfile.txt"
CONFIG_FILE="/etc/osquery/osquery.conf"
LOCAL_PIDFILE="/var/osquery/osqueryd.pidfile"
PIDFILE="/var/run/osqueryd.pidfile"

Next, edit the flagfile.txt file and update the path to Secret and Certificate file.

sudo vim /etc/osquery/flagfile.txt

# Server
--tls_hostname=osquery.kifarunix-demo.com:8080
--tls_server_certs=/etc/osquery/fleet.pem

# Enrollment
--host_identifier=instance
--enroll_secret_path=/etc/osquery/secret.txt
--enroll_tls_endpoint=/api/v1/osquery/enroll

Save and exit the file after making the changes.

Reload systemd configurations;

sudo systemctl daemon-reload

Restart osqueryd service

sudo systemctl restart osqueryd.service

Checking the status;

systemctl status osqueryd

● osqueryd.service - The osquery Daemon
     Loaded: loaded (/lib/systemd/system/osqueryd.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-03-20 17:55:09 UTC; 19s ago
    Process: 6553 ExecStartPre=/bin/sh -c if [ ! -f $FLAG_FILE ]; then touch $FLAG_FILE; fi (code=exited, status=0/SUCCESS)
    Process: 6557 ExecStartPre=/bin/sh -c if [ -f $LOCAL_PIDFILE ]; then mv $LOCAL_PIDFILE $PIDFILE; fi (code=exited, status=0/SUCCESS)
   Main PID: 6566 (osqueryd)
      Tasks: 17 (limit: 2282)
     Memory: 9.7M
     CGroup: /system.slice/osqueryd.service
             ├─6566 /usr/bin/osqueryd --flagfile /etc/osquery/flagfile.txt --config_path /etc/osquery/osquery.conf
             └─6571 /usr/bin/osqueryd

Mar 20 17:55:09 ubuntu systemd[1]: Starting The osquery Daemon...
Mar 20 17:55:09 ubuntu systemd[1]: Started The osquery Daemon.
Mar 20 17:55:09 ubuntu osqueryd[6566]: osqueryd started [version=4.6.0]
Mar 20 17:55:12 ubuntu osqueryd[6571]: I0320 17:55:12.108310  6571 events.cpp:867] Event...

Enable the service to run on system boot;

sudo systemctl enable osqueryd.service

Also verify that osquery host status is online on the Fleet manager.

Querying Host from Fleet Osquery Manager

You can now query the host by clicking on the hostname and then Query

Set the name of the Query

Enter the SQL query e.g (select interface,address,mask from interface_addresses where interface NOT LIKE '%lo%';)
Save the query for future use if you want
Select the host to query