Easy way to configure Filebeat-Logstash SSL/TLS Connection

0
294

In this tutorial, we will show you an easy way to configure Filebeat-Logstash SSL/TLS Connection. In order to sent encrypted data from Filebeat to Logstash, you need to enable SSL/TLS mutual communication between them.

Easy way to configure Filebeat-Logstash SSL/TLS Connection

Before you can proceed, we assume that you already have installed and setup ELK stack as well the Filebeat on the end points from where you are collecting event data from.

Install and Setup ELK Stack

You can follow any of the guides below to install and setup Elastic Stack;

Install ELK Stack on Ubuntu 20.04

Installing ELK Stack on CentOS 8

Deploy a Single Node Elastic Stack Cluster on Docker Containers

Install and Setup Filebeat

Follow the links below to install and setup Filebeat;

Install and Configure Filebeat on CentOS 8

Install Filebeat on Fedora 30/Fedora 29/CentOS 7

Install and Configure Filebeat 7 on Ubuntu 18.04/Debian 9.8

Generate ELK Stack CA and Server Certificates

In this demo, we will be creating TLS certificates using elasticsearch-certutil.

elasticsearch-certutil is an Elastic Stack utility that simplifies the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack.

With elasticsearch-certutil, it is possible to generate the certificates for a specific node or multiple nodes. However, in this demo, since we are just running a single node Elastic Stack with all the components in place, then we will just generate the certificates for just this single node.

To silently generate the node certificates, create an YAML file to define you nodes distinguished names (can be hostname) and the node FQDN in the format shown below;

vim $HOME/instances.yml
instances:
  - name: 'elk'
    dns: [ 'elk.kifarunix-demo.com' ]

Once that is done, run the command below to generate the ELK Stack TLS Certificates.

/usr/share/elasticsearch/bin/elasticsearch-certutil cert --keep-ca-key --pem --in $HOME/instances.yml --out $HOME/elk-cert.zip --days 365

The command will create the CA key and certificate, the node key and certificate archived in a $HOME/elk-cert.zip file which is valid for an year.

Listing the contents of the archive file;

unzip -l $HOME/elk-cert.zip
Archive:  /root/elk-cert.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2020-10-16 17:48   ca/
     1200  2020-10-16 17:48   ca/ca.crt
     1675  2020-10-16 17:48   ca/ca.key
        0  2020-10-16 17:48   elk/
     1188  2020-10-16 17:48   elk/elk.crt
     1675  2020-10-16 17:48   elk/elk.key
---------                     -------
     5738                     6 files

Read more about the elasticsearch-certutil tool on Elasticsearch reference page.

Extract the certificate files to some directory. In the command below, we extract to my home directory.

unzip -d $HOME $HOME/elk-cert.zip

You should now have these files;

ls $HOME/ca/ -1
ca.crt
ca.key
ls $HOME/elk -1
elk.crt
elk.key

Be sure to keep you private keys as secure as possible.

Convert the Keys to Standard Elastic Beats PKCS#8 Key format

For Beat to connect to Logstash via TLS, you need to convert the generated node key to the PKCS#8 standard required for the Elastic Beat – Logstash communication over TLS;

openssl pkcs8 -in $HOME/elk/elk.key -topk8 -nocrypt -out $HOME/elk/elk.pkcs8.key

Configure Filebeat-Logstash SSL/TLS Connection

Next, copy the node certificate, $HOME/elk/elk.crt, and the Beats standard key, to the relevant configuration directory. In this setup, we install the certs/keys on the /etc/logstash directory;

cp $HOME/elk/{elk.pkcs8.key,elk.crt} /etc/logstash/

Configure Filebeat-Logstash SSL/TLS connection;

vim /etc/logstash/conf.d/test.conf
input {
  beats {
    port => 5044
    ssl => true
    ssl_key => '/etc/logstash/elk.pkcs8.key'
    ssl_certificate => '/etc/logstash/elk.crt'
  }
}
output {
 #  elasticsearch {
 #    hosts => ["https://localhost:9200"]
 #    manage_template => false
 #    index => "ssh_auth-%{+YYYY.MM}"
 #    cacert => "/etc/logstash/logstash.ca.crt"
 #}
 stdout { }
}

The lines highlighted above enables the Filebeat-Logstash connection.

Save and exit the configuration file.

Test Logstash Configuration

Before you can run Logstash, it is a good idea to check for any configuration errors;

/usr/share/logstash/bin/logstash --path.settings /etc/logstash -t

If all is well, you should see such lines from the command output;

...
Configuration OK
[2020-10-16T19:03:05,994][INFO ][logstash.runner ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

You can now run Logstash in debugging mode just to see if any error arises as per your Logstash configuration file;

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf --path.settings /etc/logstash/
...
[INFO ] 2020-10-16 19:07:34.788 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2020-10-16 19:07:34.899 [[main]<beats] Server - Starting server on port: 5044
[INFO ] 2020-10-16 19:07:35.212 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
...

If you see the line, Successfully started Logstash API endpoint, then you good to go.

Configure Filebeat for Logstash SSL/TLS communication

Assuming you have already installed Filebeat on a system you want to collect logs from, configure it for Logstash TLS communication as follows;

Copy the CA certificate generated above to the remote remote system.

scp $HOME/ca/ca.crt [email protected]:

Once you have copied the CA certificate to the remote host running filebeat, proceed to configure Filebeat-Logstash SSL/TLS communication.

Place the copied CA certificate at some relevant directory, e.g /etc/filebeat;

cp $HOME/ca.crt /etc/filebeat

Now configure Filebeat to use SSL/TLS by specifying the path to CA cert on the Logstash output;

output.logstash:
  hosts: ["elk.kifarunix-demo.com:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/ca.crt"]

See our sample Filebeat configuration file. Ensure that the Logstash hostname matches the FQDN used while creating the certificates.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/auth.log
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
output.logstash:
  hosts: ["elk.kifarunix-demo.com:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/ca.crt"]
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Save the configuration file.

Validate the Logstash server’s certificate

Before you can run Filebeat, you need to validate the Logstash server’s certificate trust.

curl -v --cacert /etc/filebeat/ca.crt https://elk.kifarunix-demo.com:5044

If the trust can be established between the Logstash and the Filebeat, the command should return an empty response from the server.

*   Trying 192.168.57.3:5044...
* TCP_NODELAY set
* Connected to elk.kifarunix-demo.com (192.168.57.3) port 5044 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/filebeat/ca.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=elk
*  start date: Oct 17 15:06:00 2020 GMT
*  expire date: Oct 15 15:06:00 2030 GMT
*  subjectAltName: host "elk.kifarunix-demo.com" matched cert's "elk.kifarunix-demo.com"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: elk.kifarunix-demo.com:5044
> User-Agent: curl/7.68.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS alert, close notify (256):
* Empty reply from server
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
curl: (52) Empty reply from server

If you used IP address whilst generating the TLS certs, then run verification as follows;

curl -v --cacert /etc/filebeat/ca.crt https://192.168.57.3:5044

Testing Filebeat Configuration

Filebeat in debugging mode to check if all is well.

filebeat -e

You should see the Filebeat starts to harvest the log files and connects to Logstash host.

...
2020-10-16T20:05:49.564Z	INFO	cfgfile/reload.go:224	Loading of config files completed.
2020-10-16T20:05:49.563Z	INFO	log/harvester.go:299	Harvester started for file: /var/log/auth.log
2020-10-16T20:05:52.543Z	INFO	[add_cloud_metadata]	add_cloud_metadata/add_cloud_metadata.go:89	add_cloud_metadata: hosting provider type not detected.
2020-10-16T20:05:53.544Z	INFO	[publisher_pipeline_output]	pipeline/output.go:143	Connecting to backoff(async(tcp://elk.kifarunix-demo.com:5044))
2020-10-16T20:05:53.547Z	INFO	[publisher]	pipeline/retry.go:219	retryer: send unwait signal to consumer
2020-10-16T20:05:53.549Z	INFO	[publisher]	pipeline/retry.go:223	  done
2020-10-16T20:05:53.624Z	INFO	[publisher_pipeline_output]	pipeline/output.go:151	Connection to backoff(async(tcp://elk.kifarunix-demo.com:5044)) established

If you were running Logstash in debugging mode, then you should be able to see the logs being populated to standard output;

...
{
    "@timestamp" => 2020-10-16T20:05:52.544Z,
         "input" => {
        "type" => "log"
    },
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
      "@version" => "1",
         "agent" => {
             "version" => "7.9.2",
                "name" => "elk.kifarunix-demo.com",
                "type" => "filebeat",
            "hostname" => "elk.kifarunix-demo.com",
        "ephemeral_id" => "1241500c-8f5f-401b-a9f9-1526e8651878",
                  "id" => "726660dc-4b6b-464f-b19b-62f343792a18"
    },
          "host" => {
        "containerized" => false,
         "architecture" => "x86_64",
                  "mac" => [
            [0] "08:00:27:5c:05:2a",
            [1] "08:00:27:7f:84:15"
        ],
                 "name" => "elk.kifarunix-demo.com",
             "hostname" => "elk.kifarunix-demo.com",
                   "os" => {
            "codename" => "focal",
             "version" => "20.04.1 LTS (Focal Fossa)",
                "name" => "Ubuntu",
            "platform" => "ubuntu",
              "family" => "debian",
              "kernel" => "5.4.0-51-generic"
        },
                   "ip" => [
            [0] "10.0.2.15",
            [1] "fe80::a00:27ff:fe5c:52a",
            [2] "192.168.57.3",
            [3] "fe80::a00:27ff:fe7f:8415"
        ],
                   "id" => "57e55f802e0648f885bfe16101cb8d55"
    },
           "log" => {
        "offset" => 6926,
          "file" => {
            "path" => "/var/log/auth.log"
        }
    },
           "ecs" => {
        "version" => "1.5.0"
    },
       "message" => "Oct 16 20:03:50 ubuntu20 sshd[8512]: Failed password for johndoe from 192.168.57.1 port 54196 ssh2"

Now stop both Filebeat and Logstash debugging modes and start and enable the services to start on boot;

systemctl enable --now logstash
systemctl enable --now filebeat

And that marks the end an easy way to configure Filebeat-Logstash SSL/TLS Connection. Enjoy.

Further Reading

Filebeat Reference: Secure communication with Logstash

Related Tutorials

Install Nextcloud with Nginx and SSL/TLS Certificates on CentOS 8

Configure Apache with SSL/TLS Certificates on CentOS 8

Configure Nginx with SSL/TLS certificates on CentOS 8

Monitor SSL/TLS Certificates Expiry with Nagios

LEAVE A REPLY

Please enter your comment!
Please enter your name here