Easily Configure Elasticsearch HTTPS Connection

|
Last Updated:
|
|

In this tutorial, you will learn how to easily configure Elasticsearch HTTPS Connection. You can configure your Beats; Filebeat, Metricbeat, Packetbeat, Logstash, Kibana, to securely connect to Elasticsearch via SSL/TLS mutual communication between them.

Easily Configure Elasticsearch HTTPS Connection

Install and Setup ELK Stack

Before you can proceed, we assume that you have already installed and setup ELK stack. Otherwise, you can follow any of the guides below to install and setup Elastic Stack;

Install ELK Stack on Linux

Generate ELK Stack CA and Server TLS Certificates

Elasticsearch 8.x Autogenerated CA and TLS Certificates

In our setup, we are running Elastic Stack 8.x. When you deploy Elasticsearch 8.x, authentication and authorization, TLS for the transport and HTTP layers is enabled and configured by default.

You can confirm this by checking the Elasticsearch 8.x configuration file, /etc/elasticsearch/elasticsearch.yml.

See sample configuration below;


#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 10-04-2023 06:16:19
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["debian11"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

The CA and self signed TLS certificates are generated by default and stored under /etc/elasticsearch/certs/.

ls -1 /etc/elasticsearch/certs/
http_ca.crt
http.p12
transport.p12

The certificates are in Public Key Cryptography Standard #12 (PKCS12) format. If you want to use this certificates with other endpoints such as Kibana, Logstash, Beats to secure connection connection with Elasticsearch, you will have to convert the PKCS12 certs into Privacy-Enhance Mail (PEM) format.

Beware that the certificates auto-generated usually have a validity period of 3 years.

With the above, you just have to configure your nodes to use HTTP and CA files as appropriately.

Generate ELK Stack CA and Server TLS Certificates

If you are using a version of Elastic stack other than Elastic 8.x, or maybe you upgraded from Elastic 7.x to Elastic 8.x and you just setup only HTTPS connection between Elasticsearch cluster nodes, then you need to use your own TLS certificates.

You can use commercial certificates or simply generate the self-signed ones!

On Elasticsearch, you can generate TLS certificates using elasticsearch-certutil command.

elasticsearch-certutil is an Elastic Stack utility that simplifies the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack.

With elasticsearch-certutil, it is possible to generate the certificates for a specific node or multiple nodes.

More IMPORTANTLY, if you have multiple Elasticsearch nodes or you plan to add more nodes to your cluster in the future, and you want to generate a single certificate that can be used with all of them, use --dns option with wildcard.

Create a directory to store the certificates if you don’t have one already.

[[ -d /etc/elasticsearch/certs ]] || mkdir /etc/elasticsearch/certs

Generate the Certificate Authority (CA). By default, the CA is generated in PKCS#12 format. Hence, to get them in the usual PEM format, pass --pem option.

/usr/share/elasticsearch/bin/elasticsearch-certutil ca \
--pem \
--days 3650 \
--out /etc/elasticsearch/certs/elkstack-ca.zip

The output zip file will contain individual files for the CA certificate and private key

unzip -l /etc/elasticsearch/certs/elkstack-ca.zip

Archive:  /etc/elasticsearch/certs/elkstack-ca.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2023-04-10 11:03   ca/
     1200  2023-04-10 11:03   ca/ca.crt
     1675  2023-04-10 11:03   ca/ca.key
---------                     -------
     2875                     3 files

Unzip the file to get the CA required to generate the TLS certs.

unzip -d /etc/elasticsearch/certs /etc/elasticsearch/certs/elkstack-ca.zip

Archive:  /etc/elasticsearch/certs/elkstack-ca.zip
   creating: /etc/elasticsearch/certs/ca/
  inflating: /etc/elasticsearch/certs/ca/ca.crt  
  inflating: /etc/elasticsearch/certs/ca/ca.key

You should now have the ca/ca.crt and ca/ca.key in the certs directory.

Be sure to keep you private keys as secure as possible.

Next, generate the certificates (in PEM format) using the CA generate above.

/usr/share/elasticsearch/bin/elasticsearch-certutil cert \
--name elkstack-certs \
--ca-cert /etc/elasticsearch/certs/ca/ca.crt \
--ca-key /etc/elasticsearch/certs/ca/ca.key \
--pem \
--dns '*.kifarunix-demo.com' \
--days 3650 \
--out /etc/elasticsearch/certtest/elkstack-certs.zip

Extract the certificate files.

unzip -d /etc/elasticsearch/certs /etc/elasticsearch/certs/elkstack-certs.zip

The certificate file and key will now be stored under /etc/elasticsearch/certs/elkstack-certs directory.

Read more about the elasticsearch-certutil tool on Elasticsearch reference page.

Configure Elasticsearch for HTTPS Connection

To ensure that any HTTP connection to Elasticsearch is encrypted with TLS, you need to configure Elasticsearch for the same.

The configuration options for enabling SSL/TLS encryption for HTTP traffic in Elasticsearch are;

  • xpack.security.http.ssl.enabled: set to true to enable SSL/TLS encryption for HTTP traffic.
  • xpack.security.http.ssl.key: specifies the path to the SSL/TLS private key file, in either PEM or PKCS format.
  • xpack.security.http.ssl.certificate: specifies the path to the SSL/TLS certificate file, in either PEM or PKCS format.
  • xpack.security.http.ssl.certificate_authorities: specifies the path to one or more SSL/TLS certificate authority (CA) certificate files, in either PEM or PKCS format.

Thus, to enable Elasticsearch HTTPS connection, add these following configuration options with proper paths to the files as shown below, on ALL the nodes in the cluster. You need to copy certs files to all nodes.


cat >> /etc/elasticsearch/elasticsearch.yml << 'EOL'
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elkstack-certs/elkstack-certs.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elkstack-certs/elkstack-certs.crt
xpack.security.http.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt
EOL

Note that the HTTP certificate files are different from Transport files.

Once you put the configurations in the file, save and exit and restart Elasticsearch;

systemctl restart elasticsearch

After this, any connection to Elasticsearch will required the CA certificate to connect. The endoint URL must be HTTPS.

For example, to get the nodes on the cluster, you can use such command as;

curl https://node02.kifarunix-demo.com:9200/_cat/nodes?v -u elastic --cacert /etc/elasticsearch/certs/ca/ca.crt

Note that, https://node.DOMAIN. The domain part is specified with --dns option above.

Also, specify the path to the CA cert file.

Sample output of the above command;


Enter host password for user 'elastic':
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.58.22            23          64   3    0.36    0.47     0.42 cdfhilmrstw -      node01
192.168.56.10            46          96  12    0.53    0.81     0.64 dim         -      node02
192.168.56.154           54          96   5    0.73    1.35     1.89 dim         *      node03

Configure Kibana Elasticsearch HTTPS connection

Since you have enabled Elasticsearch HTTPS connection, you need to configure every app that connects to Elasticsearch to use HTTPS connection to Elasticsearch.

All you need is just to copy the CA certificates files to each endpoint and make appropriate configurations.

In this setup, we have already copied the CA file into /etc/kibana/ca.crt;

ls -alh /etc/kibana/ca.crt
-rw-r--r-- 1 root kibana 1.2K Apr 10 12:26 /etc/kibana/ca.crt

Next, configure Kibana to connect to Elasticsearch via HTTPS by changing the URLs of the Elasticsearch instances and defining path to CA file.

In my setup, this is how it looks like in the beginning;

elasticsearch.hosts: ["http://192.168.58.22:9200"]

So, replace the URL accordingly.

sed -i.bak '/^elasticsearch.hosts/s/"[^"]*"/https:\/\/node01.kifarunix-demo.com:9200/' /etc/kibana/kibana.yml

Let’s confirm;

elasticsearch.hosts: [https://node01.kifarunix-demo.com:9200]
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/ca.crt" ]

Restart Kibana;

systemctl restart kibana

There you go. Now access Kibana and see if all is good!

Configure Logstash Elasticsearch HTTPS Connection

Next, copy the CA certificate to the relevant configuration directory on Logstash node.

In this setup, we install the certs/keys on the /etc/logstash directory;

Configure Logstash Elasticsearch HTTPS connection;

vim /etc/logstash/conf.d/test.conf

input {
  beats {
    port => 5044
  }
}
output {
   elasticsearch {
    hosts => ["https://node01.kifarunix-demo.com:9200"]
    cacert => "/etc/logstash/ca.crt"
    user => "${ES_USER}"
    password => "${ES_PASSWORD}"
 }
}

Save and exit the configuration file.

Before you can run Logstash, it is a good idea to check for any configuration errors;

/usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/test.conf -t

If all is well, you should see such lines from the command output;

Configuration OK

You can now run Logstash in debugging mode just to see if any error arises as per your Elasticsearch output configuration file;

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf --path.settings /etc/logstash/

Ensure that there is no error! if any fix it and proceed.

Configure Filebeat for Elasticsearch SSL/TLS communication

Assuming you have already installed Filebeat on a system you want to collect logs from, configure it for Elasticsearch TLS communication as follows;

Copy the CA certificate from the Elasticsearch cluster to the system where Filebeat is installed.

scp /path/ro/ca/ca.crt username@filebeat-host:

Once you have copied the CA certificate to the remote host running filebeat, proceed to configure Elasticsearch HTTPS communication.

Place the copied CA certificate at some relevant directory, e.g /etc/filebeat;

cp $HOME/ca.crt /etc/filebeat

Now configure Filebeat to use SSL/TLS by specifying the path to CA cert on the Elasticsearch output config section (note that, I also enabled basic authentication);

vim /etc/filebeat/filebeat.yml

output.elasticsearch:
  hosts: ["https://node01.kifarunix-demo.com:9200"]
  ssl.certificate_authorities: ["/etc/filebeat/ca.crt"]
  ssl.verification_mode: full
  username: "ES_USERNAME"
  password: "ES_PASSWORD"

Ensure that the Elasticsearch hostname matches the FQDN used while creating the certificates.

Save the configuration file and exit.

Validate the Elasticsearch server’s certificate: Before you can run Filebeat, you need to validate the Elasticsearch server’s certificate trust.

curl -v --cacert /etc/filebeat/ca.crt https://node01.kifarunix-demo.com:9200 -u elastic

Enter host password for user 'elastic':
*   Trying 192.168.58.22:9200...
* Connected to node01.kifarunix-demo.com (192.168.58.22) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/filebeat/ca.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=elkstack-certs
*  start date: Apr 10 08:18:16 2023 GMT
*  expire date: Apr  7 08:18:16 2033 GMT
*  subjectAltName: host "node01.kifarunix-demo.com" matched cert's "*.kifarunix-demo.com"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
* Server auth using Basic with user 'elastic'
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET / HTTP/1.1
> Host: node01.kifarunix-demo.com:9200
> Authorization: Basic ZWxhc3RpYzpzOWx1NW5zNk9JR21oSkUxelcxZw==
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 529
< 
{
  "name" : "node01",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "pE3v-1oSTfaiF3Dp2VbOjg",
  "version" : {
    "number" : "8.7.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "09520b59b6bc1057340b55750186466ea715e30e",
    "build_date" : "2023-03-27T16:31:09.816451435Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host node01.kifarunix-demo.com left intact

Testing Filebeat Configuration. Run Filebeat in debugging mode to check if all is well.

filebeat test config

You should get, Config OK, if all good.

Test connection to Elasticsearch;

filebeat test output

elasticsearch: https://node01.kifarunix-demo.com:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.58.22
    dial up... OK
  TLS...
    security... WARN server's certificate chain verification is disabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 8.7.0

Start the filebeat and log to stderr

filebeat -e

If there is any error, you will see on the console.

Otherwise, if you get such a line, all is good;

Connection to backoff(elasticsearch(https://URL:9200)) established

Press ctrl+c to cancel above and start filebeat

systemctl restart filebeat

This applies all Elastic Beats. configure them for Elasticsearch HTTPS connection the same way you have configured Filebeat.

And that marks the end an easy way to configure Elasticsearch HTTPS Connection. Enjoy.

Install Nextcloud with Nginx and SSL/TLS Certificates on CentOS 8

Configure Apache with SSL/TLS Certificates on CentOS 8

Configure Nginx with SSL/TLS certificates on CentOS 8

Monitor SSL/TLS Certificates Expiry with Nagios

SUPPORT US VIA A VIRTUAL CUP OF COFFEE

We're passionate about sharing our knowledge and experiences with you through our blog. If you appreciate our efforts, consider buying us a virtual coffee. Your support keeps us motivated and enables us to continually improve, ensuring that we can provide you with the best content possible. Thank you for being a coffee-fueled champion of our work!

Photo of author
Kifarunix
Linux Certified Engineer, with a passion for open-source technology and a strong understanding of Linux systems. With experience in system administration, troubleshooting, and automation, I am skilled in maintaining and optimizing Linux infrastructure.

1 thought on “Easily Configure Elasticsearch HTTPS Connection”

  1. Doesn’t this only describe a self signed cert? I don’t see steps for generating CSR and getting a cert from a CA?

    Reply

Leave a Comment