In this guide, we are going to learn how to install Filebeat on Fedora 30/Fedora 29/CentOS 7. Filebeat is a lightweight shipper for collecting, forwarding and centralizing event log data. It is installed as an agent on the servers you are collecting logs from. It can forward the logs it is collecting to either Elasticsearch or Logstash for indexing.
To setup Elastic Stack, follow the link below.
Install Filebeat on Fedora 30/Fedora 29/CentOS 7
Assuming you have already setup Elastic Stack, proceed to install Filebeat to collect your system logs for processing. In this guide, we are going to configure Filebeat to collect system authentication logs for processing.
Update your system packages.
yum update yum upgrade
Next, install Filebeat on Fedora 30/Fedora 29/CentOS 7. Installation can be done using RPM binary or using YUM repos.
Install Filebeat 7 using RPM Repository
Import the repository signing GPG key.
sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
Next, install YUM Elastic repo.
cat > /etc/yum.repos.d/elastic-7.x.repo << EOF [elasticsearch-7.x] name=Elastic repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md EOF
yum install filebeat
Install Filebeat Using RPM Binary
Download the binary by executing the command below;
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.2.0-x86_64.rpm
yum localinstall filebeat-7.2.0-x86_64.rpm
Configure Filebeat 7 on Fedora 30/Fedora 29/CentOS 7
Next, configure Filebeat to sent event data to Logstash. Hence, edit the configuration file, /etc/filebeat/filebeat.yml, comment out Elasticsearch output and enable Logstash Output under the Outputs section as shown below;
... #================================ Outputs ===================================== # Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------ #output.elasticsearch: # Array of hosts to connect to. #hosts: ["localhost:9200"] # Optional protocol and basic auth credentials. #protocol: "https" #username: "elastic" #password: "changeme" #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["192.168.43.75:5044"]
Replace the IP address with IP address of the server under which Logstash is running and verify connection.
telnet 192.168.43.75 5044 Trying 192.168.43.75... Connected to 192.168.43.75. Escape character is '^]'.
Enable Filebeat System Module
In this setup, our Logstash was configured to process system authentication events. Hence, enable the System module which collects and parses logs created by the system logging service of common Unix/Linux based distributions. This module is disabled by default.
filebeat modules enable system Enabled system
Configure system module to read authentication logs only. Simply set the value of syslog to false.
... - module: system # Syslog syslog: enabled: false # Set custom paths for the log files. If left empty, # Filebeat will choose the paths depending on your OS. #var.paths: # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1. #var.convert_timezone: false # Authorization logs auth: enabled: true # Set custom paths for the log files. If left empty, # Filebeat will choose the paths depending on your OS. var.paths: ["/var/log/secure"] ...
Load the index template in Elasticsearch
If you are using Logstash as the event data process engine, you need to manually load the index template into Elasticsearch. Hence, ensure that there a connection to Elasticsearch before you can load the index template.
telnet 192.168.43.75 9200 Trying 192.168.43.75... Connected to 192.168.43.75. Escape character is '^]'.
If all is well., load the template.
filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["192.168.43.75:9200"]'
If you see the output, Index setup finished, template load was successful.
If the host doesn’t have direct connectivity to Elasticsearch, you can generate the index template, copy it to Elastic Stack Server and install it locally.
To generate the template;
filebeat export template > filebeat.template.json
To install the template on Elastic Stack server, copy it and run locally on Elastic Stack server.
curl -XPUT -H 'Content-Type: application/json' http://192.168.43.75:9200/_template/filebeat-7.0.2 [email protected]
Once you are done with that, start and enable Filebeat to run on system boot.
systemctl start filebeat systemctl enable filebeat
You can run Filebeat in debug mode using the command below;
filebeat -e -c filebeatconfig.yml
By default, /etc/filebeat/filebeat.yml is used. Hence, you can just run;
Verify Elasticsearch Index Data Reception
After the configuration above, simulate a failed and successful SSH authentication to the server on which Filebeat is running. Once that is done, login to Elastic stack server and verify data reception.
curl -X GET 192.168.43.75:9200/_cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .kibana_task_manager xQztoO5CRoygONVw6ujEVg 1 0 2 18 27.8kb 27.8kb yellow open ssh_auth-2019.07 f6lBK5osQemJEb1lUtwGEQ 1 1 41 0 118.9kb 118.9kb green open .kibana_1 1iR0TWklToSzoEBeZiE1Dg 1 0 3 1 43.2kb 43.2kb yellow open filebeat-7.2.0-2019.07.02-000001 nelIPqlOSfKzGidOOk5C4g 1 1 0 0 283b 283b
After that, proceed to the Kibana and Create Index Pattern. See our guide on setting up Elastic Stack 7 on Fedora 30/Fedora 29/CentOS 7.
You should now be able to see your SSH authentication events.
SSH successful Logins
SSH failed logins
Congratulations. That is all on how to install Filebeat on Fedora 30/Fedora 29/CentOS 7. Enjoy.