Last updated on July 16th, 2022 at 01:44 pm

In this tutorial, you will learn how to install ELK stack 8.x on Ubuntu systems. Elastic/ELK stack 8.0 has been released making it another major version release after Elastic 7.0. Elastic 8.0 comes with a lot of improvements including;

• compatibility with 7.x REST API
• security features enabled and configured by default (HTTPS and Authentication)
• Better protection for system indices
• New k-nearest neighbor (kNN) API
• Read more on the Elastic 8.0 release highlights page.

Install ELK Stack 8.x on Ubuntu

In order to demonstrate how you can install ELK Stack 8.x on Ubuntu, we will be using Ubuntu 20.04 LTS system.

You can run Elastic Stack 8.x as a docker container or run it on your system as a package. We will be installing it as a package.

Set system Hostname

Begin by setting your system hostname;

hostnamectl set-hostname elk.kifarunix-demo.com

Update DNS records locally on hosts file if you dont have DNS server;

echo "192.168.59.23 elk.kifarunix-demo.com elk" >> /etc/hosts

Install Elastic Stack 8.x Repositories

To install Elastic Stack 8.x on Ubuntu, you need to install Elastic Stack 8.x repositories as follows.

Install Elastic stack 8.x repository signing key.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Install the Elastic Stack 8.x repository;

echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" > /etc/apt/sources.list.d/elastic-8.x.list

Run system update;

apt update

Install ELK Stack 8.x on Ubuntu

Elastic stack is made up of various opensource tools; Elasticsearch, Logstash, Kibana, and Beats.

In this setup, I will only install the Kibana and Elasticsearch. Since I do not require any special data processing on data, Logstash wont be necessary.

Install Elasticsearch 8.0 on Ubuntu

You can install Elasticsearch 8.0 automatically from Elastic repos installed above by executing the command below;

apt install elasticsearch -y

During the installation, the Security features will be enabled by default;

• Authentication and authorization are enabled.
• TLS for the transport and HTTP layers is enabled and configured.
• Elastic super user account (elastic) and its password is created.

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  elasticsearch
0 upgraded, 1 newly installed, 0 to remove and 157 not upgraded.
Need to get 515 MB of archives.
After this operation, 1100 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 elasticsearch amd64 8.0.0 [515 MB]
Fetched 515 MB in 3min 50s (2242 kB/s)                                                                                                                                     
Selecting previously unselected package elasticsearch.
(Reading database ... 131793 files and directories currently installed.)
Preparing to unpack .../elasticsearch_8.0.0_amd64.deb ...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Unpacking elasticsearch (8.0.0) ...
Setting up elasticsearch (8.0.0) ...
--------------------------- Security autoconfiguration information ------------------------------
Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : Dq3a-zWIgGz0_F*mpazU
If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token '
after creating an enrollment token on your existing cluster.
You can complete the following actions at any time:
Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.
Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.
Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.
-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

Configure Elasticsearch 8.0 on Ubuntu

Since we are running a basic setup single node cluster, we will go with the default settings.

If you check the Elasticsearch configuration file,/etc/elasticsearch/elasticsearch.yml, you will see the security setting enabled;

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 16-02-2022 12:17:24
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ubuntu20"]

# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]

# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

Without comment lines, this is how the default Elasticsearch 8.0 configuration looks like;

grep -Ev '^#|^$' /etc/elasticsearch/elasticsearch.yml

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["elk.kifarunix-demo.com"]
http.host: [_local_, _site_]

Of course you can update the cluster name, node name, the host and the port details.

Adjust Elasticsearch JVM Settings

Next, configure JVM heap size to no more than half the size of your memory. In this case, our test server has 2G RAM and the heap size is set to 512M for both maximum and minimum sizes.

vim /etc/elasticsearch/jvm.options

...
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms512m
-Xmx512m
...

Save and exit the file.

Running Elasticsearch

Start and enable Elasticsearchto run on system boot;

systemctl daemon-reload

systemctl enable --now elasticsearch

To check the status;

systemctl status elasticsearch

● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-02-16 13:38:43 UTC; 25s ago
       Docs: https://www.elastic.co
   Main PID: 5153 (java)
      Tasks: 50 (limit: 3486)
     Memory: 757.3M
     CGroup: /system.slice/elasticsearch.service
             ├─5153 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Djava.security.manager=a>
             └─5463 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Feb 16 13:38:21 ubuntu20 systemd[1]: Starting Elasticsearch...
Feb 16 13:38:43 ubuntu20 systemd[1]: Started Elasticsearch.

You can as well verify ES status using curl command. Replace the IP accordingly.

curl https://192.168.59.23:9200 --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic

When prompted, enter the Elasticsearch password generated above.

If you get such an output, then all is well.

{
  "name" : "elk.kifarunix-demo.com",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "ntbGI_QMQmSja7leLSTWMw",
  "version" : {
    "number" : "8.0.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "1b6a7ece17463df5ff54a3e1302d825889aa1161",
    "build_date" : "2022-02-03T16:47:57.507843096Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Also, check the ports are opened;

ss -altnp | grep 9200

LISTEN   0        4096       [::ffff:192.168.59.23]:9200                *:*      users:(("java",pid=9566,fd=372))                                               
LISTEN   0        4096           [::ffff:10.0.2.15]:9200                *:*      users:(("java",pid=9566,fd=371))                                               
LISTEN   0        4096           [::ffff:127.0.0.1]:9200                *:*      users:(("java",pid=9566,fd=370))                                               
LISTEN   0        4096                        [::1]:9200             [::]:*      users:(("java",pid=9566,fd=369))

Install Kibana 8.0 on Ubuntu 22.04

Since we already setup Elastic repos, simply install Kibana 8.0 by running the command;

apt install kibana

Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  kibana
0 upgraded, 1 newly installed, 0 to remove and 157 not upgraded.
Need to get 281 MB of archives.
After this operation, 761 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 kibana amd64 8.0.0 [281 MB]
Fetched 281 MB in 2min 3s (2282 kB/s)                                                                                                                                      
Selecting previously unselected package kibana.
(Reading database ... 71317 files and directories currently installed.)
Preparing to unpack .../kibana_8.0.0_amd64.deb ...
Unpacking kibana (8.0.0) ...
Setting up kibana (8.0.0) ...
Creating kibana group... OK
Creating kibana user... OK
Created Kibana keystore in /etc/kibana/kibana.keystore

Kibana is set to run on localhost:5601 by default. To allow external access, edit the configuration file and replace the value of server.host with an interface IP.

vim /etc/kibana/kibana.yml

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

...
# To allow connections from remote users, set this parameter to a non-loopback address.
#server.host: "localhost"
server.host: "192.168.59.23"

Those are the only changes we will make for now.

Next, generate an enrollment token for Kibana using the command below;

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

eyJ2ZXIiOiI4LjAuMCIsImFkciI6WyIxOTIuMTY4LjU5LjE0OjkyMDAiLCIxMC4wLjIuMTU6OTIwMCIsIjE5Mi4xNjguNTkuMjM6OTIwMCJdLCJmZ3IiOiIxZjM2YmY2YzY5YjZiODZhNGFjOTJmZWZiMTBmNjFhZTIyNTljN2ZkZThmN2Q2OTBiNDdmZTg5NGIzYWVhYWZlIiwia2V5IjoiLV9ELUJYOEJYdU9CcVFmb081bWk6ZU15dU1WVGpRTHV5Ql8zTHdlanlQQSJ9

Also generate Kibana Encryption keys;

• xpack.encryptedSavedObjects.encryptionKey: Used to encrypt stored objects such as dashboards and visualizations
• xpack.reporting.encryptionKey: Used to encrypt saved reports
• xpack.security.encryptionKey: Used to encrypt session information

These can be generated using the command below;

/usr/share/kibana/bin/kibana-encryption-keys generate

xpack.encryptedSavedObjects.encryptionKey: 706c88e045c127e21b81c902425cdb54
xpack.reporting.encryptionKey: d67296d7d4958bdd1594e965e6b97ab9
xpack.security.encryptionKey: d496d7cb6a5983c213f7902767069744

Insert these lines into Kibana config file, kibana.yml.

echo -e "xpack.encryptedSavedObjects.encryptionKey: 706c88e045c127e21b81c902425cdb54
xpack.reporting.encryptionKey: d67296d7d4958bdd1594e965e6b97ab9
xpack.security.encryptionKey: d496d7cb6a5983c213f7902767069744" >> /etc/kibana/kibana.yml

With all comment lines removed, this is how our Kibana configuration looks like;

grep -Ev '^#|^$' /etc/kibana/kibana.yml

server.port: 5601
server.host: "192.168.59.23"
elasticsearch.hosts: ["http://localhost:9200"]
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid
xpack.encryptedSavedObjects.encryptionKey: 706c88e045c127e21b81c902425cdb54
xpack.reporting.encryptionKey: d67296d7d4958bdd1594e965e6b97ab9
xpack.security.encryptionKey: d496d7cb6a5983c213f7902767069744

If you need to secure Kibana 8.0 by proxying it with Nginx, you can check how to on our previous by following the link below;

Configure Nginx with SSL to Proxy Kibana

Running Kibana

Once the installation is done, start and enable Kibana 8.0 to run on system boot.

systemctl enable --now kibana

Confirm Kibana status;

systemctl status kibana

● kibana.service - Kibana
     Loaded: loaded (/lib/systemd/system/kibana.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-02-17 05:49:45 UTC; 36s ago
       Docs: https://www.elastic.co
   Main PID: 10214 (node)
      Tasks: 11 (limit: 3486)
     Memory: 219.1M
     CGroup: /system.slice/kibana.service
             └─10214 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist

Feb 17 05:49:45 elk.kifarunix-demo.com systemd[1]: Started Kibana.
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: [2022-02-17T05:49:53.013+00:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: [2022-02-17T05:49:53.106+00:00][INFO ][http.server.Preboot] http server running at http://192.168.59.23:5601
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: [2022-02-17T05:49:53.136+00:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: [2022-02-17T05:49:53.139+00:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch >
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: [2022-02-17T05:49:53.175+00:00][INFO ][root] Holding setup until preboot stage is completed.
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: i Kibana has not been configured.
Feb 17 05:49:53 elk.kifarunix-demo.com kibana[10214]: Go to http://192.168.59.23:5601/?code=639753 to get started.

From the status output, you will see such lines;

Kibana has not been configured.

Go to http://192.168.59.23:5601/?code=639753 to get started.

You will to copy go to the provided URL on browser to complete the setup.

Access Kibana 8.0 Dashboard

You can now access Kibana 8.0 from your browser using the url provided above, http://192.168.59.23:5601/?code=639753. It could be different for you.

If UFW is running, Open Kibana 8.0 port;

ufw allow 5601/tcp

Upon accessing Kibana 8.0 interface, on the welcome page, you will be required to configure Elastic to get started. So just copy the Kibana token generated using the /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana, command and paste on the box.

Once you past the token, click Configure Elastic. It will then save the settings, configure and restart Elasticsearch.

You are then taken to Login page. Login using the generated Elastic user credentials.

On the welcome page, click Explore on my own to proceed to Kibana 8.0 dashboard.

And this is how you can install ELK Stack 8.x on Ubuntu. You can now explore further.

Other Tutorials

Install Logstash 8 on Ubuntu/Debian

Configure Logstash Elasticsearch Basic Authentication

Configure Filebeat-Elasticsearch Authentication

How to Enable Basic Authentication on ELK Stack