Install and Setup Prometheus on Ubuntu 20.04

|
Last Updated:
|
|

In this guide, we are going to learn how to install Prometheus on Ubuntu 20.04. Prometheus is an open-source systems and service monitoring system. It collects metrics from configured targets via HTTP calls at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some conditions are met.

Some of the main features of Prometheus include;

  • a multi-dimensional data model with time series data identified by metric name and key/value pairs
  • PromQL, a flexible query language to leverage this dimensionality
  • no reliance on distributed storage; single server nodes are autonomous
  • time series collection happens via a pull model over HTTP
  • pushing time series is supported via an intermediary gateway
  • targets are discovered via service discovery or static configuration
  • multiple modes of graphing and dashboarding support

Installing Prometheus on Ubuntu 20.04

Prometheus is made up various components;

  • The main Prometheus server which scrapes and stores time series data
  • Client libraries for instrumenting application code
  • Push gateway for supporting short-lived jobs
  • Exporters for exporting existing metrics from third-party systems as Prometheus metrics in cases where it is not feasible to instrument a given system with Prometheus metrics directly for example in services like HAProxy, StatsD, Graphite, etc.
  • Alertmanager to handle alerts.

Prometheus is available on the default Ubuntu 20.04 repos;

apt-cache policy prometheus
prometheus:
  Installed: (none)
  Candidate: 2.15.2+ds-2
  Version table:
     2.15.2+ds-2 500
        500 http://ke.archive.ubuntu.com/ubuntu focal/universe amd64 Packages

However, as you might have noted, the Prometheus release versions provided by the default focal repos is not up-to-date. The version 2.18 is current stable release version of Prometheus.

Install Prometheus Using Pre-compiled Binaries on Ubuntu 20.04

To ensure that you are installing the latest release version of Prometheus, use the pre-compiled binaries which can be downloaded directly from Prometheus downloads section.

Prerequisites

Before you can go ahead and install Prometheus using pre-compiled binaries on Ubuntu 20.04;

Create Prometheus System User and Group

Run the command below to create Prometheus system user and group;

useradd -M -r -s /bin/false prometheus
Create Prometheus Directories

Next, you need to create the directories that will be used to store Prometheus configurations files and other data.

mkdir /etc/prometheus /var/lib/prometheus
Download Prometheus Binary

Next, navigate to Prometheus downloads section and grab the latest version of Prometheus. You simply use wget to download the Prometheus binary for Linux as shown below;

wget https://github.com/prometheus/prometheus/releases/download/v2.18.1/prometheus-2.18.1.linux-amd64.tar.gz

Verify the integrity of the binary file downloaded by calculating the SHA256 hash.

sha256sum prometheus-2.18.1.linux-amd64.tar.gz
5fcc35b78bd0a1b84afae6de94248a4bea3cdb4daf0d54a37b5491cb86b014d7 prometheus-2.18.1.linux-amd64.tar.gz

Compare the resulting hash with the hash provided on the downloads page. Ensure that they match.

Install Prometheus on Ubuntu 20.04

Extract the downloaded Prometheus binary;

tar xzf prometheus-2.18.1.linux-amd64.tar.gz

Copy the prometheus and promtool binaries under the extracted Prometheus archive folder to /usr/local/bin directory.

cp prometheus-2.18.1.linux-amd64/{prometheus,promtool} /usr/local/bin/

After copying, set the user and group ownership of these binaries to prometheus.

chown prometheus:prometheus /usr/local/bin/{prometheus,promtool}

Next, copy the consoles and console_libraries directories to Prometheus configuration directory, /etc/prometheus.

cp -r prometheus-2.18.1.linux-amd64/{consoles,console_libraries} /etc/prometheus/

Create Prometheus Configuration file

A sample Prometheus configuration file is available on the extracted archive folder. To make our work easier, just copy it to Prometheus configuration directory.

cp prometheus-2.18.1.linux-amd64/prometheus.yml /etc/prometheus/

Modify the configuration file to suit your needs. In this case, we just go with the defaults.

vim /etc/prometheus/prometheus.yml
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

In the default configuration there is only a single job, called prometheus , which scrapes the time series data exposed by the Prometheus server. The job contains a single, statically configured, target, the localhost on port 9090.

Next, set the user and group ownership of Prometheus configuration directory, /etc/prometheus to prometheus.

chown -R prometheus:prometheus /etc/prometheus

Once that is done, similarly set the user and group ownership of Prometheus data directory, /var/lib/prometheus/ to prometheus.

chown prometheus:prometheus /var/lib/prometheus

Running Prometheus

At the very least, Prometheus is now set and is ready to run. However, at this point we do not have the Prometheus service configuration file and hence, we can run it as shown below;

prometheus --config.file=/etc/prometheus/prometheus.yml
...
level=info ts=2020-05-29T08:32:47.317Z caller=main.go:678 msg="Starting TSDB …"
level=info ts=2020-05-29T08:32:47.320Z caller=head.go:575 component=tsdb msg="Replaying WAL, this may take awhile"
level=info ts=2020-05-29T08:32:47.320Z caller=web.go:523 component=web msg="Start listening for connections" address=0.0.0.0:9090
...
level=info ts=2020-05-29T08:32:47.326Z caller=main.go:695 msg="TSDB started"
level=info ts=2020-05-29T08:32:47.326Z caller=main.go:799 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-05-29T08:32:47.731Z caller=main.go:827 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-05-29T08:32:47.731Z caller=main.go:646 msg="Server is ready to receive web requests."

Accessing Prometheus from Web Interface

Open Prometheus port on firewall (UFW) if it is running. By default, it listens on TCP port 9090.

ufw allow 9090/tcp

Prometheus is now ready to receive web requests. You can access it from the browser using the address, http://server-IP-or-Hostname:9090.

prometheus web

To check the status of your node, navigate to Status > Targets.

Prometheus targets on Ubuntu 20.04

To view scraped metrics, navigate to, http://<server_IP>:9090/metrics

To check memory statistics, for example, free memory available, select the go_memstats_frees_total query and click execute and see the result on the console tab.

memstats

To view the graph for the memory statistics, click Graph tab.

memstats graph

Create Prometheus Systemd Service File

To run Prometheus as a service, you can create a systemd service configuration file as shown below;

vim /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus Time Series Collection and Processing Server
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Next, reload systemd configuration files and start and enable Prometheus to run on system boot.

systemctl daemon-reload
systemctl enable --now prometheus

To check the status of Prometheus service;

systemctl status prometheus
● prometheus.service - Prometheus Time Series Collection and Processing Server
     Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-05-29 08:56:32 UTC; 18s ago
   Main PID: 105588 (prometheus)
      Tasks: 6 (limit: 2281)
     Memory: 17.1M
     CGroup: /system.slice/prometheus.service
             └─105588 /usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/promethe>

May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.655Z caller=main.go:678 msg="Starting TSDB ..."
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.657Z caller=head.go:575 component=tsdb msg="Replaying WAL, this may tak>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.657Z caller=web.go:523 component=web msg="Start listening for connectio>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.660Z caller=head.go:624 component=tsdb msg="WAL segment loaded" segment>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.660Z caller=head.go:627 component=tsdb msg="WAL replay completed" durat>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.662Z caller=main.go:694 fs_type=EXT4_SUPER_MAGIC
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.662Z caller=main.go:695 msg="TSDB started"
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.662Z caller=main.go:799 msg="Loading configuration file" filename=/etc/>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.710Z caller=main.go:827 msg="Completed loading of configuration file" f>
May 29 08:56:33 freeradius.kifarunix-demo.com prometheus[105588]: level=info ts=2020-05-29T08:56:33.711Z caller=main.go:646 msg="Server is ready to receive web requests."

Further Reading;

Prometheus Getting Started

Other Related Tutorials

Monitor Linux System Metrics with Prometheus Node Exporter

Install Graylog 3.0 on CentOS 7

Monitor Squid Access Logs with Graylog Server

Monitor Squid logs with Grafana and Graylog

SUPPORT US VIA A VIRTUAL CUP OF COFFEE

We're passionate about sharing our knowledge and experiences with you through our blog. If you appreciate our efforts, consider buying us a virtual coffee. Your support keeps us motivated and enables us to continually improve, ensuring that we can provide you with the best content possible. Thank you for being a coffee-fueled champion of our work!

Photo of author
gen_too
Co-founder of Kifarunix.com, Linux Tips and Tutorials. Linux/Unix admin and author at Kifarunix.com.

1 thought on “Install and Setup Prometheus on Ubuntu 20.04”

  1. root@ip-172-27-184-101:~# prometheus –config.file=/etc/prometheus/prometheus.yml
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:302 msg=”No time or size retention was set so using the default time retention” duration=15d
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:337 msg=”Starting Prometheus” version=”(version=2.18.1, branch=HEAD, revision=ecee9c8abfd118f139014cb1b174b08db3f342cf)”
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:338 build_context=”(go=go1.14.2, user=root@2117a9e64a7e, date=20200507-16:51:47)”
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:339 host_details=”(Linux 5.4.0-1045-aws #47~18.04.1-Ubuntu SMP Tue Apr 13 15:58:14 UTC 2021 x86_64 ip-172-27-184-101 (none))”
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:340 fd_limits=”(soft=1024, hard=1048576)”
    level=info ts=2022-11-08T03:58:45.126Z caller=main.go:341 vm_limits=”(soft=unlimited, hard=unlimited)”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:678 msg=”Starting TSDB …”
    level=info ts=2022-11-08T03:58:45.128Z caller=web.go:523 component=web msg=”Start listening for connections” address=0.0.0.0:9090
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:547 msg=”Stopping scrape discovery manager…”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:561 msg=”Stopping notify discovery manager…”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:583 msg=”Stopping scrape manager…”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:557 msg=”Notify discovery manager stopped”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:577 msg=”Scrape manager stopped”
    level=info ts=2022-11-08T03:58:45.128Z caller=repair.go:59 component=tsdb msg=”Found healthy block” mint=1667796969062 maxt=1667822400000 ulid=01GH9GHZJ5JBBVJ02DFE1MGXE6
    level=info ts=2022-11-08T03:58:45.128Z caller=manager.go:882 component=”rule manager” msg=”Stopping rule manager…”
    level=info ts=2022-11-08T03:58:45.128Z caller=manager.go:892 component=”rule manager” msg=”Rule manager stopped”
    level=info ts=2022-11-08T03:58:45.128Z caller=notifier.go:601 component=notifier msg=”Stopping notification manager…”
    level=info ts=2022-11-08T03:58:45.128Z caller=repair.go:59 component=tsdb msg=”Found healthy block” mint=1667822400000 maxt=1667844000000 ulid=01GH9Y9E10N1RW7H7PHWN7HV7Q
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:749 msg=”Notifier manager stopped”
    level=info ts=2022-11-08T03:58:45.128Z caller=main.go:543 msg=”Scrape discovery manager stopped”
    level=info ts=2022-11-08T03:58:45.128Z caller=repair.go:59 component=tsdb msg=”Found healthy block” mint=1667865600000 maxt=1667872800000 ulid=01GHAJWKQ9BQJPHNQQXMB6SHQX
    level=info ts=2022-11-08T03:58:45.128Z caller=repair.go:59 component=tsdb msg=”Found healthy block” mint=1667844000000 maxt=1667865600000 ulid=01GHAJWKRJYFFG9TPK0JC7CTJG
    level=error ts=2022-11-08T03:58:45.129Z caller=main.go:758 err=”error starting web server: listen tcp 0.0.0.0:9090: bind: address already in use”

    Getting this error, is there any way to solve it??

    We have a openvpn server for that before we need to test it in a jump server, we have to install openvpn service, Prometheus and openvpn_exporter and then test it Is it working fine or not??

    Reply

Leave a Comment