In this tutorial, you will learn how to easily configure Elasticsearch HTTPS Connection. You can configure your Beats; Filebeat, Metricbeat, Packetbeat, Logstash, Kibana, to securely connect to Elasticsearch via SSL/TLS mutual communication between them.
Table of Contents
Configuring Elasticsearch HTTPS Connection
Install and Setup ELK Stack
Before you can proceed, we assume that you have already installed and setup ELK stack. Otherwise, you can follow any of the guides below to install and setup Elastic Stack;
Generate ELK Stack CA and Server TLS Certificates
Elasticsearch 8.x Autogenerated CA and TLS Certificates
In our setup, we are running Elastic Stack 8.x. When you deploy Elasticsearch 8.x, authentication and authorization, TLS for the transport and HTTP layers is enabled and configured by default.
You can confirm this by checking the Elasticsearch 8.x configuration file, /etc/elasticsearch/elasticsearch.yml.
See sample configuration below;
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 10-04-2023 06:16:19
#
# --------------------------------------------------------------------------------
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["debian11"]
# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0
# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
The CA and self signed TLS certificates are generated by default and stored under /etc/elasticsearch/certs/.
ls -1 /etc/elasticsearch/certs/http_ca.crt
http.p12
transport.p12The certificates are in Public Key Cryptography Standard #12 (PKCS12) format. If you want to use this certificates with other endpoints such as Kibana, Logstash, Beats to secure connection connection with Elasticsearch, you will have to convert the PKCS12 certs into Privacy-Enhance Mail (PEM) format.
Beware that the certificates auto-generated usually have a validity period of 3 years.
With the above, you just have to configure your nodes to use HTTP and CA files as appropriately.
Generate ELK Stack CA and Server TLS Certificates
If you are using a version of Elastic stack other than Elastic 8.x, or maybe you upgraded from Elastic 7.x to Elastic 8.x and you just setup only HTTPS connection between Elasticsearch cluster nodes, then you need to use your own TLS certificates.
You can use commercial certificates or simply generate the self-signed ones!
On Elasticsearch, you can generate TLS certificates using elasticsearch-certutil command.
elasticsearch-certutil is an Elastic Stack utility that simplifies the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack.
With elasticsearch-certutil, it is possible to generate the certificates for a specific node or multiple nodes.
More IMPORTANTLY, if you have multiple Elasticsearch nodes or you plan to add more nodes to your cluster in the future, and you want to generate a single certificate that can be used with all of them, use --dns option with wildcard.
Create a directory to store the certificates if you don’t have one already.
[[ -d /etc/elasticsearch/certs ]] || mkdir /etc/elasticsearch/certsGenerate the Certificate Authority (CA). By default, the CA is generated in PKCS#12 format. Hence, to get them in the usual PEM format, pass --pem option.
/usr/share/elasticsearch/bin/elasticsearch-certutil ca \
--pem \
--days 3650 \
--out /etc/elasticsearch/certs/elkstack-ca.zipThe output zip file will contain individual files for the CA certificate and private key
unzip -l /etc/elasticsearch/certs/elkstack-ca.zip
Archive: /etc/elasticsearch/certs/elkstack-ca.zip
Length Date Time Name
--------- ---------- ----- ----
0 2023-04-10 11:03 ca/
1200 2023-04-10 11:03 ca/ca.crt
1675 2023-04-10 11:03 ca/ca.key
--------- -------
2875 3 files
Unzip the file to get the CA required to generate the TLS certs.
unzip -d /etc/elasticsearch/certs /etc/elasticsearch/certs/elkstack-ca.zip
Archive: /etc/elasticsearch/certs/elkstack-ca.zip
creating: /etc/elasticsearch/certs/ca/
inflating: /etc/elasticsearch/certs/ca/ca.crt
inflating: /etc/elasticsearch/certs/ca/ca.key
You should now have the ca/ca.crt and ca/ca.key in the certs directory.
Be sure to keep you private keys as secure as possible.
Next, generate the certificates (in PEM format) using the CA generate above.
/usr/share/elasticsearch/bin/elasticsearch-certutil cert \
--name elkstack-certs \
--ca-cert /etc/elasticsearch/certs/ca/ca.crt \
--ca-key /etc/elasticsearch/certs/ca/ca.key \
--pem \
--dns '*.kifarunix-demo.com' \
--days 3650 \
--out /etc/elasticsearch/certtest/elkstack-certs.zipExtract the certificate files.
unzip -d /etc/elasticsearch/certs /etc/elasticsearch/certs/elkstack-certs.zipThe certificate file and key will now be stored under /etc/elasticsearch/certs/elkstack-certs directory.
Read more about the elasticsearch-certutil tool on Elasticsearch reference page.
Configuring Elasticsearch for HTTPS Connection
To ensure that any HTTP connection to Elasticsearch is encrypted with TLS, you need to configure Elasticsearch for the same.
The configuration options for enabling SSL/TLS encryption for HTTP traffic in Elasticsearch are;
xpack.security.http.ssl.enabled: set totrueto enable SSL/TLS encryption for HTTP traffic.xpack.security.http.ssl.key: specifies the path to the SSL/TLS private key file, in either PEM or PKCS format.xpack.security.http.ssl.certificate: specifies the path to the SSL/TLS certificate file, in either PEM or PKCS format.xpack.security.http.ssl.certificate_authorities: specifies the path to one or more SSL/TLS certificate authority (CA) certificate files, in either PEM or PKCS format.
Thus, to enable Elasticsearch HTTPS connection, add these following configuration options with proper paths to the files as shown below, on ALL the nodes in the cluster. You need to copy certs files to all nodes.
cat >> /etc/elasticsearch/elasticsearch.yml << 'EOL'
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elkstack-certs/elkstack-certs.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elkstack-certs/elkstack-certs.crt
xpack.security.http.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt
EOL
Note that the HTTP certificate files are different from Transport files.
Once you put the configurations in the file, save and exit and restart Elasticsearch;
systemctl restart elasticsearchAfter this, any connection to Elasticsearch will required the CA certificate to connect. The endoint URL must be HTTPS.
For example, to get the nodes on the cluster, you can use such command as;
curl https://node02.kifarunix-demo.com:9200/_cat/nodes?v -u elastic --cacert /etc/elasticsearch/certs/ca/ca.crtNote that, https://node.DOMAIN. The domain part is specified with --dns option above.
Also, specify the path to the CA cert file.
Sample output of the above command;
Enter host password for user 'elastic':
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.58.22 23 64 3 0.36 0.47 0.42 cdfhilmrstw - node01
192.168.56.10 46 96 12 0.53 0.81 0.64 dim - node02
192.168.56.154 54 96 5 0.73 1.35 1.89 dim * node03
Configure Kibana Elasticsearch HTTPS connection
Since you have enabled Elasticsearch HTTPS connection, you need to configure every app that connects to Elasticsearch to use HTTPS connection to Elasticsearch.
All you need is just to copy the CA certificates files to each endpoint and make appropriate configurations.
In this setup, we have already copied the CA file into /etc/kibana/ca.crt;
ls -alh /etc/kibana/ca.crt-rw-r--r-- 1 root kibana 1.2K Apr 10 12:26 /etc/kibana/ca.crtNext, configure Kibana to connect to Elasticsearch via HTTPS by changing the URLs of the Elasticsearch instances and defining path to CA file.
In my setup, this is how it looks like in the beginning;
elasticsearch.hosts: ["http://192.168.58.22:9200"]So, replace the URL accordingly.
sed -i.bak '/^elasticsearch.hosts/s/"[^"]*"/https:\/\/node01.kifarunix-demo.com:9200/' /etc/kibana/kibana.ymlLet’s confirm;
elasticsearch.hosts: [https://node01.kifarunix-demo.com:9200]
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/ca.crt" ]Restart Kibana;
systemctl restart kibanaThere you go. Now access Kibana and see if all is good!
Configure Logstash Elasticsearch HTTPS Connection
Next, copy the CA certificate to the relevant configuration directory on Logstash node.
In this setup, we install the certs/keys on the /etc/logstash directory;
Configure Logstash Elasticsearch HTTPS connection;
vim /etc/logstash/conf.d/test.conf
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["https://node01.kifarunix-demo.com:9200"]
cacert => "/etc/logstash/ca.crt"
user => "${ES_USER}"
password => "${ES_PASSWORD}"
}
}
Save and exit the configuration file.
Before you can run Logstash, it is a good idea to check for any configuration errors;
/usr/share/logstash/bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/test.conf -tIf all is well, you should see such lines from the command output;
Configuration OKYou can now run Logstash in debugging mode just to see if any error arises as per your Elasticsearch output configuration file;
/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf --path.settings /etc/logstash/Ensure that there is no error! if any fix it and proceed.
Configure Filebeat for Elasticsearch SSL/TLS communication
Assuming you have already installed Filebeat on a system you want to collect logs from, configure it for Elasticsearch TLS communication as follows;
Copy the CA certificate from the Elasticsearch cluster to the system where Filebeat is installed.
scp /path/ro/ca/ca.crt username@filebeat-host:Once you have copied the CA certificate to the remote host running filebeat, proceed to configure Elasticsearch HTTPS communication.
Place the copied CA certificate at some relevant directory, e.g /etc/filebeat;
cp $HOME/ca.crt /etc/filebeatNow configure Filebeat to use SSL/TLS by specifying the path to CA cert on the Elasticsearch output config section (note that, I also enabled basic authentication);
vim /etc/filebeat/filebeat.yml
output.elasticsearch:
hosts: ["https://node01.kifarunix-demo.com:9200"]
ssl.certificate_authorities: ["/etc/filebeat/ca.crt"]
ssl.verification_mode: full
username: "ES_USERNAME"
password: "ES_PASSWORD"
Ensure that the Elasticsearch hostname matches the FQDN used while creating the certificates.
Save the configuration file and exit.
Validate the Elasticsearch server’s certificate: Before you can run Filebeat, you need to validate the Elasticsearch server’s certificate trust.
curl -v --cacert /etc/filebeat/ca.crt https://node01.kifarunix-demo.com:9200 -u elastic
Enter host password for user 'elastic':
* Trying 192.168.58.22:9200...
* Connected to node01.kifarunix-demo.com (192.168.58.22) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/filebeat/ca.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=elkstack-certs
* start date: Apr 10 08:18:16 2023 GMT
* expire date: Apr 7 08:18:16 2033 GMT
* subjectAltName: host "node01.kifarunix-demo.com" matched cert's "*.kifarunix-demo.com"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
* Server auth using Basic with user 'elastic'
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET / HTTP/1.1
> Host: node01.kifarunix-demo.com:9200
> Authorization: Basic ZWxhc3RpYzpzOWx1NW5zNk9JR21oSkUxelcxZw==
> User-Agent: curl/7.81.0
> Accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 529
<
{
"name" : "node01",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "pE3v-1oSTfaiF3Dp2VbOjg",
"version" : {
"number" : "8.7.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "09520b59b6bc1057340b55750186466ea715e30e",
"build_date" : "2023-03-27T16:31:09.816451435Z",
"build_snapshot" : false,
"lucene_version" : "9.5.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
* Connection #0 to host node01.kifarunix-demo.com left intact
Testing Filebeat Configuration. Run Filebeat in debugging mode to check if all is well.
filebeat test configYou should get, Config OK, if all good.
Test connection to Elasticsearch;
filebeat test output
elasticsearch: https://node01.kifarunix-demo.com:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 192.168.58.22
dial up... OK
TLS...
security... WARN server's certificate chain verification is disabled
handshake... OK
TLS version: TLSv1.3
dial up... OK
talk to server... OK
version: 8.7.0
Start the filebeat and log to stderr
filebeat -eIf there is any error, you will see on the console.
Otherwise, if you get such a line, all is good;
Connection to backoff(elasticsearch(https://URL:9200)) establishedPress ctrl+c to cancel above and start filebeat
systemctl restart filebeatThis applies all Elastic Beats. configure them for Elasticsearch HTTPS connection the same way you have configured Filebeat.
And that marks the end an easy way to configure Elasticsearch HTTPS Connection. Enjoy.
Related Tutorials
Install Nextcloud with Nginx and SSL/TLS Certificates on CentOS 8
Configure Apache with SSL/TLS Certificates on CentOS 8
Doesn’t this only describe a self signed cert? I don’t see steps for generating CSR and getting a cert from a CA?