Install Elastic Stack 7 on Ubuntu 18.04/Debian 9.8

This tutorial will guide on how to install Elastic Stack 7 on Ubuntu 18.04/Debian 9.8. We will be discussing the installation and configuration of each component of Elastic Stack 7. Talking of components, Elastic Stack, previously known as ELK stack, comprises of four opensource major components, Elasticsearch, Kibana, Logstash and Beats. These components can be used to collect, parse, store, search, analyze and visualize different types of logs collected from different types sources. To break down a bit;

• Elasticsearch is a search and analytics engine
• Kibana is a data visualization and dash-boarding tool that enables you to analyze data stored on Elasticsearch.
• Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then stashes it on search analytics engine like Elasticsearch
• Beats on the other hand are the log shippers that collects logs from different endpoints and sends them to either Logstash or directly to Elasticsearch.

Install Elastic Stack 7 on Ubuntu 18.04/Debian 9.8

We will run the installation of Elastic Stack components in the following order. Such an order ensures that each component depend is in place.

• Install Elasticsearch
• Install Kibana
• Install Logstash
• Install Beats

Installing Elasticsearch 7

Installation of Elasticsearch 7.0 on Ubuntu 18.04/Debian 9.8 has been discussed in our previous guide. Therefore check it out by following the link below.

Install Elasticsearch 7 on Ubuntu 18.04/Debian 9.8

Install Kibana 7.x on Ubuntu 18.04/Debian 9.8

Once the installation of Elasticsearch is done, proceed to install Kibana. Kibana is not available on the default Ubuntu/Debian repos. But since we are working on a single node Elastic stack, we already created the Elastic Stack APT repos while installing Elasticsearch. Hence, you can simply run the command below to install Kibana 7.x.

apt install kibana

Once the installation is done, start and enable Kibana to run on system boot.

systemctl daemon-reload

systemctl enable --now kibana

Kibana is set to run on localhost:5601 by default. Therefore, to add some layer of security, you can install and configure Nginx to proxy the connection to Kibana via a publicly accessible interface IP. If you choose to use Nginx instead of exposing Kibana, you can proceed as follows;

Install Nginx

apt install nginx

Configure Nginx with SSL to Proxy Kibana

To configure Nginx with SSL to Proxy connection to Kibana, you need to generate the SSL/TLS certificates and create Nginx configuration file to define Kibana settings. In this guide, we are using self-signed certificate. You can as well obtain a trusted CA certificate from your preferred provider.

Generate Self-signed SSL/TLS certificates

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/kibana-selfsigned.key -out /etc/ssl/certs/kibana-selfsigned.crt

Also, create Deffie-Hellman group.

openssl dhparam -out /etc/nginx/dhparam.pem 2048

Create Kibana Nginx configuration. You can use the recommendations from the Cipherli.st while configuring SSL.

vim /etc/nginx/sites-available/kibana

server {
	listen 80;
	server_name elk.example.com;
	return 301 https://$host$request_uri;
}
server {
	listen 443 ssl;
	server_name elk.example.com;

	root /var/www/html;
	index index.html index.htm index.nginx-debian.html;

    	ssl_certificate /etc/ssl/certs/kibana-selfsigned.crt;
	ssl_certificate_key /etc/ssl/private/kibana-selfsigned.key;

	ssl_protocols TLSv1.2 TLSv1.3;
	ssl_prefer_server_ciphers on; 
	ssl_dhparam /etc/nginx/dhparam.pem;
	ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;
	ssl_ecdh_curve secp384r1;
	ssl_session_timeout  10m;
	ssl_session_cache shared:SSL:10m;
	resolver 192.168.42.129 8.8.8.8 valid=300s;
	resolver_timeout 5s; 
	add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
	add_header X-Frame-Options DENY;
	add_header X-Content-Type-Options nosniff;
	add_header X-XSS-Protection "1; mode=block";

	access_log  /var/log/nginx/kibana_access.log;
	error_log  /var/log/nginx/kibana_error.log;

	auth_basic "Authentication Required";
	auth_basic_user_file /etc/nginx/kibana.users;

	location / {
	        proxy_pass http://localhost:5601;
        	proxy_http_version 1.1;
	        proxy_set_header Upgrade $http_upgrade;
        	proxy_set_header Connection 'upgrade';
	        proxy_set_header Host $host;
        	proxy_cache_bypass $http_upgrade;
	}
}

Configure Nginx Authentication

To configure Nginx User authentication, you need to create users and their password. These authentication details will be saved in the file, /etc/nginx/kibana.users, specified by auth_basic_user_file parameter in the Nginx configuration file. You can use openssl command to generate the authentication credentials as shown below. Replace the USERNAME and PASSWORD accordingly;

printf "USERNAME:$(openssl passwd -crypt PASSWORD)\n" > /etc/nginx/kibana.users

Active Kibana Nginx Configuration

Next, run the commands below to enable Kibana Nginx configuration, verify the Nginx syntax and reload it if everything is okay.

ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/

nginx -t
 nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
 nginx: configuration file /etc/nginx/nginx.conf test is successful

systemctl reload nginx

If UFW is running, allow Nginx connections, both HTTP and HTTPS.

ufw allow 'Nginx Full'

Access Kibana Dashboard

You should now be able to access Kibana dashboard via the server fully qualified hostname, https://elk.example.com in this case. Accept the risk of using the self-signed certificate and proceed. Before you can access the Kibana dashboard, you will be required to provide the authentication credentials set above.

After authentication, you will land on Kibana dashboard. Since we don’t have data yet, you will see the screen below. Click Explore My Own to proceed to Kibana dashboard.

Kibana is now configured. The next step is to install Logstash, the data processing engine and Filebeat, the data shippers. See the installation links below.

Install and Configure Logstash 7 on Ubuntu 18/Debian 9.8

Install and Configure Filebeat 7 on Ubuntu 18.04/Debian 9.8

Reference;

Kibana User Guide