In this tutorial, you will learn how to add or enroll Osquery hosts on Fleet manager. Fleet is the most widely used open source osquery manager. Deploying osquery with Fleet enables programmable live queries, streaming logs, and effective management of osquery across 50,000+ servers, containers, and laptops. It’s especially useful for talking to multiple devices at the same time.“
In our previous guide, we learnt how to install Osquery Fleet manager on Ubuntu 20.04.
Install Fleet Osquery Manager on Ubuntu 20.04
Enroll Osquery Hosts on Fleet Manager
Install Osquery on hosts
Before you can add or enroll Osquery hosts on Fleet manager, you need to have installed Osquery on the remote hosts.
Below are some guides you can follow to install Osquery;
Install Osquery on Ubuntu 20.04
Install Osquery on Debian 10 Buster
Enroll Osquery Hosts on Fleet Manager
After you have installed Osquery on the hosts, you can then proceed to enroll them on Fleet manager.
Therefore, login to your Fleet manager web interface.
To add or enroll a host, click Add new host. This opens up such an wizard.
Download:
Enrollment secret
: Provide an active enroll secret to allow osquery to authenticate with the Fleet server.Server certificate
: Provide the TLS certificate used by the Fleet server to enable secure connections from osquery.Flag File
: flags to control initialization, disable/enable features, and select plugins.
Once the files above are downloaded, copy them to the Osquery host.
For example, in my setup, i have copied the files to home directory of specific user account on my Ubuntu 20.04 server;
ls ~/ -1
flagfile.txt
fleet.pem
secret.txt
Navigate to the directory where the files above are stored on the osquery host, in case it is my users home directory.
cd ~/
Enroll Osquery host on Fleet manager by running the command below. (Before you can run the command below, ensure that the Osquery Fleet manager hostname that you generated the SSL certs from is resolvable from the host)
sudo osqueryd --flagfile=flagfile.txt --verbose
You will some system output.
...
I0320 16:53:40.973621 5419 init.cpp:340] osquery initialized [version=4.6.0]
I0320 16:53:40.993187 5419 system.cpp:342] Found stale process for osqueryd (4742)
I0320 16:53:40.993357 5419 system.cpp:374] Writing osqueryd pid (5419) to /var/run/osqueryd.pidfile
I0320 16:53:40.993547 5419 extensions.cpp:453] Could not autoload extensions: Cannot open file for reading: /etc/osquery/extensions.load
I0320 16:53:40.993702 5419 dispatcher.cpp:78] Adding new service: WatcherRunner (0x55de416f5f08) to thread: 139824272353024 (0x55de417114e0) in process 5419
I0320 16:53:40.994439 5420 watcher.cpp:613] osqueryd watcher (5419) executing worker (5421)
I0320 16:53:41.004231 5421 init.cpp:337] osquery worker initialized [watcher=5419]
I0320 16:53:41.004514 5421 dispatcher.cpp:78] Adding new service: WatcherWatcherRunner (0x55b07d7a9d98) to thread: 140124192265984 (0x55b07d79e8a0) in process 5421
I0320 16:53:41.004647 5421 rocksdb.cpp:132] Opening RocksDB handle: /var/osquery/osquery.db
I0320 16:53:42.638896 5421 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0x55b07d8a64c8) to thread: 140123990906624 (0x55b07d892450) in process 5421
I0320 16:53:42.639508 5421 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0x55b07d820758) to thread: 140123999299328 (0x55b07d79a060) in process 5421
I0320 16:53:42.640108 5421 tls_enroll.cpp:70] TLSEnrollPlugin requesting a node enroll key from: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/enroll
I0320 16:53:42.640694 5421 system.cpp:301] Using host identifier: f68ce9e5-caf4-4b2e-85c0-e1cd4304abb0
I0320 16:53:42.641921 5520 interface.cpp:270] Extension manager service starting: /var/osquery/osquery.em
I0320 16:53:42.642381 5421 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0320 16:53:42.662926 5421 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0320 16:53:42.664144 5421 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/enroll
I0320 16:53:43.371500 5421 auto_constructed_tables.cpp:97] Removing stale ATC entries
I0320 16:53:43.372932 5421 dispatcher.cpp:78] Adding new service: ConfigRefreshRunner (0x55b07d81d508) to thread: 140124007692032 (0x55b07d7a5680) in process 5421
I0320 16:53:43.374224 5421 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/config
...
...
I0320 17:00:07.842394 5521 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/config
I0320 17:00:08.261268 5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:09.360673 5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:13.452287 5524 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0320 17:00:15.192095 5522 tls.cpp:254] TLS/HTTPS POST request to URI: https://osquery.kifarunix-demo.com:8080/api/v1/osquery/log
The host now communicates with the Osquery Fleet manager and it should be enrolled.
Navigate to Osquery Fleet Manager and refresh the web interface. You should be able to see you host enrolled.
Running Osqueryd as a Service
To ensure a constant communication between the Osquery host and the Fleet manager, you need to run osqueryd as a service.
Therefore, stop the standalone process initiated above by pressing Ctrl+c.
Update the osqueryd service unit file configurations as follows.
First of all, let us move the secret, the certificate and the flag files to /etc/osquery
directory;
sudo mv ~/{flagfile.txt,fleet.pem,secret.txt} /etc/osquery
Next, update the path to FLAG_FILE environment variable in the osqueryd service defaults file, /etc/default/osqueryd
.
sudo vim /etc/default/osqueryd
Check the highlighted line. Replace the path accordingly.
FLAG_FILE="/etc/osquery/flagfile.txt" CONFIG_FILE="/etc/osquery/osquery.conf" LOCAL_PIDFILE="/var/osquery/osqueryd.pidfile" PIDFILE="/var/run/osqueryd.pidfile"
Next, edit the flagfile.txt
file and update the path to Secret and Certificate file.
sudo vim /etc/osquery/flagfile.txt
# Server --tls_hostname=osquery.kifarunix-demo.com:8080 --tls_server_certs=/etc/osquery/fleet.pem # Enrollment --host_identifier=instance --enroll_secret_path=/etc/osquery/secret.txt --enroll_tls_endpoint=/api/v1/osquery/enroll
Save and exit the file after making the changes.
Reload systemd configurations;
sudo systemctl daemon-reload
Restart osqueryd service
sudo systemctl restart osqueryd.service
Checking the status;
systemctl status osqueryd
● osqueryd.service - The osquery Daemon
Loaded: loaded (/lib/systemd/system/osqueryd.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2021-03-20 17:55:09 UTC; 19s ago
Process: 6553 ExecStartPre=/bin/sh -c if [ ! -f $FLAG_FILE ]; then touch $FLAG_FILE; fi (code=exited, status=0/SUCCESS)
Process: 6557 ExecStartPre=/bin/sh -c if [ -f $LOCAL_PIDFILE ]; then mv $LOCAL_PIDFILE $PIDFILE; fi (code=exited, status=0/SUCCESS)
Main PID: 6566 (osqueryd)
Tasks: 17 (limit: 2282)
Memory: 9.7M
CGroup: /system.slice/osqueryd.service
├─6566 /usr/bin/osqueryd --flagfile /etc/osquery/flagfile.txt --config_path /etc/osquery/osquery.conf
└─6571 /usr/bin/osqueryd
Mar 20 17:55:09 ubuntu systemd[1]: Starting The osquery Daemon...
Mar 20 17:55:09 ubuntu systemd[1]: Started The osquery Daemon.
Mar 20 17:55:09 ubuntu osqueryd[6566]: osqueryd started [version=4.6.0]
Mar 20 17:55:12 ubuntu osqueryd[6571]: I0320 17:55:12.108310 6571 events.cpp:867] Event...
Enable the service to run on system boot;
sudo systemctl enable osqueryd.service
Also verify that osquery host status is online on the Fleet manager.
Querying Host from Fleet Osquery Manager
You can now query the host by clicking on the hostname and then Query
- Set the name of the Query
- Enter the SQL query e.g (
select interface,address,mask from interface_addresses where interface NOT LIKE '%lo%';
) - Save the query for future use if you want
- Select the host to query
- Run the query
Sample results of our query;
Very nice, isn’t it?
That is how simple it is to add or enroll Osquery hosts on Fleet manager.
You can add more hosts to the Fleet for easy management and monitoring.
That marks the end of our tutorial on how to enroll Osquery hosts on Fleet manager.