In this tutorial, you will learn how to add or enroll Osquery hosts on Fleet manager. Fleet is the most widely used open source osquery manager. Deploying osquery with Fleet enables programmable live queries, streaming logs, and effective management of osquery across 50,000+ servers, containers, and laptops. It’s especially useful for talking to multiple devices at the same time.“
In our previous guide, we learnt how to install Osquery Fleet manager on Ubuntu 20.04/22.04.
Install Fleet Osquery Manager on Ubuntu 20.04/Ubuntu 22.04
Enrolling Osquery Hosts on Fleet Manager
There are different ways in which you can enroll hosts on osquery fleet manager;
Enroll Hosts into osqueryFleet Manager using Fleet-Osquery Package
This is an easy way to enroll hosts into Fleet Osquery manager.
To begin with, install Fleetctl binary on the host;
Download the Fleetctl binary archive for your specific system from Fleet releases page.
wget https://github.com/fleetdm/fleet/releases/download/fleet-v4.20.1/fleetctl_v4.20.1_linux.zip -P /tmpExtract and place the fleetctl binary under /usr/local/bin;
sudo unzip -j /tmp/fleetctl_v4.20.1_linux.zip "fleetctl_v4.20.1_linux/fleetctl" -d /usr/local/bin/next, navigate to Fleet Manager UI > Hosts menu.
On Hosts menu, click Add hosts. Such a wizard opens up.
Depending on the Linux distribution, choose the correct package type from the available options.
For example, on RHEL based System, choose Linux RPM option. If you are not running a desktop based system, uncheck Include Fleet Desktop.
Copy the command and execute on the host to generate Fleet Osquery agent installer (This generates an RPM binary);
fleetctl package --type=rpm --fleet-url=https://fleet.kifarunix-demo.com:8080 \
--enroll-secret=wFULaNuzE0wuo3/z3jbZNV5ZD0Ku1ERJSample output;
Generating your osquery installer...
Success! You generated an osquery installer at /root/fleet-osquery-1.1.0.x86_64.rpm
To add this device to Fleet, double-click to open your installer.
To add other devices to Fleet, distribute this installer using Chef, Ansible, Jamf, or Puppet. Learn how: https://fleetdm.com/docs/using-fleet/adding-hosts
Execute the command below to install the osquery agent and enroll the host server into the Fleet.
sudo yum localinstall /root/fleet-osquery-1.1.0.x86_64.rpmThe package will create a systemd service called orbit. The configs file for this service are located under /opt/orbit/.
Before you can start the service, you can update the service flags with the Fleet server ssl certificate;
echo "--tls_server_certs=/etc/ssl/certs/fleet.cert" >> /opt/orbit/osquery.flagsStart and enable the orbit service to run on system boot;
systemctl enable --now orbitYou can check the status;
systemctl status orbitConfirm host enrollment on Fleet Manager dashboard;
Enroll Hosts into Osquery Fleet Manager using Osquery Package
Before you can add hosts to Osquery manager via this method, you need to have installed Osquery on the remote hosts.
Below are some guides you can follow to install Osquery;
Install Osquery on Ubuntu 20.04/22.04
Install Osquery on Debian 10 Buster
After you have installed Osquery on the hosts, you can then proceed to enroll them on Fleet manager.
You can begin by enrolling the Fleet Manager host server itself, if not already enrolled!
To add or enroll a host, navigate to Hosts menu and click Add hosts and click Advanced from the wizards that opens up.
Next, click Plain osquery drop down menu button;
Download:
Enrollment secret: Provide an active enroll secret to allow osquery to authenticate with the Fleet server.Server certificate: Provide the TLS certificate used by the Fleet server to enable secure connections from osquery.Flag File: flags to control initialization, disable/enable features, and select plugins.
Once the files above are downloaded, copy them to the remote Osquery host.
For example, in my setup, i have copied the files to home directory of specific user account on my Ubuntu 22.04 server;
ls ~/ -1flagfile.txt
fleet.pem
secret.txtNavigate to the directory where the files above are stored on the osquery host, in case it is my users home directory.
cd ~/Enroll Osquery host on Fleet manager by running the command below.
(Before you can run the command below, ensure that the Osquery Fleet manager hostname that you generated the SSL certs from is resolvable from the host)
sudo osqueryd --flagfile=flagfile.txt --verboseYou will some system output.
I0927 17:55:12.850069 55765 init.cpp:399] osquery initialized [version=5.5.1]
I0927 17:55:12.850409 55765 extensions.cpp:453] Could not autoload extensions: Cannot open file for reading: /etc/osquery/extensions.load
I0927 17:55:12.850607 55765 dispatcher.cpp:78] Adding new service: WatcherRunner (0x55ac173b62b8) to thread: 140233538664000 (0x55ac173ae9b0) in process 55765
I0927 17:55:12.851366 55766 watcher.cpp:680] osqueryd watcher (55765) executing worker (55767)
I0927 17:55:12.856755 55767 init.cpp:396] osquery worker initialized [watcher=55765]
I0927 17:55:12.856871 55767 dispatcher.cpp:78] Adding new service: WatcherWatcherRunner (0x55a4664d72b8) to thread: 140054961374784 (0x55a4664d05a0) in process 55767
I0927 17:55:12.856969 55767 rocksdb.cpp:132] Opening RocksDB handle: /var/osquery/osquery.db
I0927 17:55:12.913545 55767 dispatcher.cpp:78] Adding new service: ExtensionWatcher (0x55a466620098) to thread: 140054420764224 (0x55a466564bf0) in process 55767
I0927 17:55:12.913673 55767 dispatcher.cpp:78] Adding new service: ExtensionRunnerCore (0x55a46661f228) to thread: 140054429156928 (0x55a4664f2cb0) in process 55767
I0927 17:55:12.913722 55896 interface.cpp:299] Extension manager service starting: /var/osquery/osquery.em
I0927 17:55:12.913940 55767 auto_constructed_tables.cpp:99] Removing stale ATC entries
I0927 17:55:12.914314 55767 dispatcher.cpp:78] Adding new service: ConfigRefreshRunner (0x55a4665aaf38) to thread: 140054437549632 (0x55a466565e40) in process 55767
I0927 17:55:12.914551 55767 tls.cpp:255] TLS/HTTPS POST request to URI: https://fleet.kifarunix-demo.com:8080/api/v1/osquery/config
I0927 17:55:13.929601 55767 tls_enroll.cpp:81] TLSEnrollPlugin requesting a node enroll key from: https://fleet.kifarunix-demo.com:8080/api/osquery/enroll
I0927 17:55:13.931106 55767 system.cpp:237] Using host identifier: 2121d69f-6e3d-4204-806a-8e214b47b7cb
I0927 17:55:13.933938 55767 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0927 17:55:13.936805 55767 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0927 17:55:13.937805 55767 tls.cpp:255] TLS/HTTPS POST request to URI: https://fleet.kifarunix-demo.com:8080/api/osquery/enroll
I0927 17:55:14.005929 55767 tls.cpp:255] TLS/HTTPS POST request to URI: https://fleet.kifarunix-demo.com:8080/api/v1/osquery/config
W0927 17:55:14.025341 55767 options.cpp:106] The CLI only flag --logger_plugin set via config file will be ignored, please use a flagfile or pass it to the process at startup
I0927 17:55:14.047302 55767 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0927 17:55:14.048081 55767 smbios_tables.cpp:105] Reading SMBIOS from sysfs DMI node
I0927 17:55:14.048434 55767 dispatcher.cpp:78] Adding new service: TLSLogForwarder (0x55a46674b188) to thread: 140054840202816 (0x55a466764c90) in process 55767
I0927 17:55:14.048753 55900 tls.cpp:255] TLS/HTTPS POST request to URI: https://fleet.kifarunix-demo.com:8080/api/osquery/log
I0927 17:55:14.049381 55767 eventfactory.cpp:156] Event publisher not enabled: BPFEventPublisher: Publisher disabled via configuration
I0927 17:55:14.049559 55767 eventfactory.cpp:156] Event publisher not enabled: auditeventpublisher: Publisher disabled via configuration
I0927 17:55:14.049754 55767 eventfactory.cpp:156] Event publisher not enabled: inotify: Publisher disabled via configuration
I0927 17:55:14.049918 55767 eventfactory.cpp:156] Event publisher not enabled: syslog: Publisher disabled via configuration
I0927 17:55:14.050160 55767 events.cpp:70] Skipping subscriber: apparmor_events: Subscriber disabled via configuration
I0927 17:55:14.050382 55767 events.cpp:70] Skipping subscriber: process_file_events: Subscriber disabled via configuration
I0927 17:55:14.050551 55767 events.cpp:70] Skipping subscriber: seccomp_events: Seccomp subscriber disabled via configuration
I0927 17:55:14.050714 55767 events.cpp:70] Skipping subscriber: selinux_events: Subscriber disabled via configuration
I0927 17:55:14.050877 55767 events.cpp:70] Skipping subscriber: socket_events: Subscriber disabled via configuration
I0927 17:55:14.051129 55901 eventfactory.cpp:390] Starting event publisher run loop: udev
I0927 17:55:14.051138 55767 dispatcher.cpp:78] Adding new service: DistributedRunner (0x55a4666cc178) to thread: 140054815024704 (0x55a4666db480) in process 55767
I0927 17:55:14.051409 55767 dispatcher.cpp:78] Adding new service: SchedulerRunner (0x55a466620718) to thread: 140054806632000 (0x55a4666f6a20) in process 55767
I0927 17:55:14.051676 55902 tls.cpp:255] TLS/HTTPS POST request to URI: https://fleet.kifarunix-demo.com:8080/api/v1/osquery/distributed/read
I0927 17:55:14.063175 55902 distributed.cpp:151] Executing distributed query: fleet_detail_query_disk_space_unix:
SELECT (blocks_available * 100 / blocks) AS percent_disk_space_available,
round((blocks_available * blocks_size *10e-10),2) AS gigs_disk_space_available
...
The host now communicates with the Osquery Fleet manager and it should be enrolled.
Navigate to Osquery Fleet Manager and refresh the web interface. You should be able to see you host enrolled.
Running Osqueryd as a Service
To ensure a constant communication between the Osquery host and the Fleet manager, you need to run osqueryd as a service.
Therefore, stop the standalone process initiated above by pressing Ctrl+c.
Update the osqueryd service unit file configurations as follows.
First of all, let us move the secret, the certificate and the flag files to /etc/osquery directory;
sudo mv ~/{flagfile.txt,fleet.pem,secret.txt} /etc/osqueryNext, update the path to FLAG_FILE environment variable in the osqueryd service defaults file, /etc/default/osqueryd.
sudo vim /etc/default/osquerydCheck the highlighted line. Replace the path accordingly.
FLAG_FILE="/etc/osquery/flagfile.txt"
CONFIG_FILE="/etc/osquery/osquery.conf"
LOCAL_PIDFILE="/var/osquery/osqueryd.pidfile"
PIDFILE="/var/run/osqueryd.pidfile"Next, edit the flagfile.txt file and update the path to Secret and Certificate file.
sudo vim /etc/osquery/flagfile.txt
# Server
--tls_hostname=osquery.kifarunix-demo.com:8080
--tls_server_certs=/etc/osquery/fleet.pem
# Enrollment
--host_identifier=instance
--enroll_secret_path=/etc/osquery/secret.txt
--enroll_tls_endpoint=/api/v1/osquery/enroll
Save and exit the file after making the changes.
Reload systemd configurations;
sudo systemctl daemon-reloadRestart osqueryd service
sudo systemctl restart osqueryd.serviceChecking the status;
systemctl status osqueryd● osqueryd.service - The osquery Daemon
Loaded: loaded (/lib/systemd/system/osqueryd.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2022-09-27 18:00:09 UTC; 5s ago
Process: 56011 ExecStartPre=/bin/sh -c if [ ! -f $FLAG_FILE ]; then touch $FLAG_FILE; fi (code=exited, status=0/SUCCESS)
Process: 56012 ExecStartPre=/bin/sh -c if [ -f $LOCAL_PIDFILE ]; then mv $LOCAL_PIDFILE $PIDFILE; fi (code=exited, status=0/SUCCESS)
Main PID: 56013 (osqueryd)
Tasks: 17 (limit: 2241)
Memory: 8.5M
CPU: 111ms
CGroup: /system.slice/osqueryd.service
├─56013 /opt/osquery/bin/osqueryd --flagfile /etc/osquery/flagfile.txt --config_path /etc/osquery/osquery.conf
└─56015 /opt/osquery/bin/osqueryd "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ">
Sep 27 18:00:09 jellyfish systemd[1]: Starting The osquery Daemon...
Sep 27 18:00:09 jellyfish systemd[1]: Started The osquery Daemon.
Sep 27 18:00:09 jellyfish osqueryd[56013]: osqueryd started [version=5.5.1]
Enable the service to run on system boot;
sudo systemctl enable osqueryd.serviceAlso verify that osquery host status is online on the Fleet manager.
Querying Host from Fleet Osquery Manager
You can now query the host by clicking on the hostname of the host and then Query
Click Create custom query.
Enter the SQL query e.g (select interface,address,mask from interface_addresses where interface NOT LIKE '%lo%';)
You can either Execute or Save the query for future use if you want.
If you choose to execute, you will be prompted to select target hosts. Select the hosts to run the query against.
Run the query
Sample results of our query;
Very nice, isn’t it?
For other custom queries, choose a table you want to query from the right pane. You will see all the available options related to the respective table that can enable you to make specific queries;
You can add more hosts to the Fleet for easy management and monitoring.
Other Tutorials;
