Install Arkime (Moloch) Full Packet Capture tool on Debian 11

Welcome to our tutorial on how to install Arkime (Moloch) Full Packet Capture tool on Debian. Arkime, formerly known as Moloch “is a large scale, open source, indexed packet capture and search system“.

Using Ubuntu system? Follow the link below;

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

According to its Github repository page, some of the features of Arkime tool include;

• It stores and indexes network traffic in standard PCAP format, providing fast, indexed access.
• Provides an intuitive web interface for PCAP browsing, searching, and exporting.
• Exposes APIs which allow for PCAP data and JSON formatted session data to be downloaded and consumed directly.
• Stores and exports all packets in standard PCAP format, allowing you to also use your favorite PCAP ingesting tools, such as wireshark, during your analysis workflow.

Install Arkime on Debian

You can install Arkime on Debian by either:

• using prebuilt binary packages or
• simply build it from the source.

Install Arkime using Prebuilt Binary on Debian

Download Arkime Binary Installer

There is no binary installer for Debian as of this writing. As a result, we will be using the installer for Ubuntu system, and preferably Ubuntu 20.04.

Thus, navigate to the downloads page and grab the binary installer for Ubuntu.

You can as well grab the link to the binary installer and pull it using curl or wget command.

For example, the command below downloads the current stable release version of Arkime binary installer for Ubuntu 20.04;

wget https://s3.amazonaws.com/files.molo.ch/builds/ubuntu-20.04/arkime_3.1.1-1_amd64.deb

Run System Update

Update your system package cache;

apt update

Installing Arkime Debian

Next, install Arkime using the downloaded binary installer.

apt install ./arkime_3.1.1-1_amd64.deb

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'arkime' instead of './arkime_3.1.1-1_amd64.deb'
The following additional packages will be installed:
  ethtool libauthen-sasl-perl libclone-perl libcommon-sense-perl libdata-dump-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl libhtml-form-perl
  libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl libhttp-message-perl
  libhttp-negotiate-perl libio-html-perl libio-socket-ssl-perl libjson-perl libjson-xs-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmailtools-perl
  libnet-http-perl libnet-smtp-ssl-perl libtimedate-perl libtry-tiny-perl libtypes-serialiser-perl liburi-perl libwww-perl libwww-robotrules-perl libyaml-0-2 libyaml-dev
Suggested packages:
  libdigest-hmac-perl libgssapi-perl libcrypt-ssleay-perl libauthen-ntlm-perl libyaml-doc
The following NEW packages will be installed:
  arkime ethtool libauthen-sasl-perl libclone-perl libcommon-sense-perl libdata-dump-perl libencode-locale-perl libfile-listing-perl libfont-afm-perl libhtml-form-perl
  libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libhttp-cookies-perl libhttp-daemon-perl libhttp-date-perl libhttp-message-perl
  libhttp-negotiate-perl libio-html-perl libio-socket-ssl-perl libjson-perl libjson-xs-perl liblwp-mediatypes-perl liblwp-protocol-https-perl libmailtools-perl
  libnet-http-perl libnet-smtp-ssl-perl libtimedate-perl libtry-tiny-perl libtypes-serialiser-perl liburi-perl libwww-perl libwww-robotrules-perl libyaml-0-2 libyaml-dev
0 upgraded, 36 newly installed, 0 to remove and 29 not upgraded.
Need to get 1,931 kB/101 MB of archives.
After this operation, 330 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

Install Elasticsearch on Debian

Arkime uses Elasticsearch as a search and indexing engine. Ensure you provide as much RAM for Elasticsearch.

NOTE: It is recommended that you run Elasticsearch on a different node apart from the one running Arkime. This is because Arkime capture and viewer will fail to start if Elasticsearch takes time to start. You can however update the Arkime capture and viewer services to run after ES service if you want to run everything on the same host.

Therefore, install Elasticsearch by running the command below;

Import the Elastic stack PGP repository signing Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor > /etc/apt/trusted.gpg.d/elastic.gpg

Install Elasticsearch APT repository;

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list

Update package cache and install Elasticsearch;

apt update

apt install elasticsearch -y

Install Arkime by building it from the source

If you want, you can as well install Arkime by building it from the source. Check the installation page for instructions.

Configure Arkime (Moloch) on Debian

Configuring Arkime

Once the installation is done, run the script below to configure Arkime (Moloch);

Answer the script prompts accordingly;

opt/arkime/bin/Configure

Select an interface to monitor;

Found interfaces: lo;enp0s3;enp0s8
Semicolon ';' seperated list of interfaces to monitor [eth1] enp0s8

Choose whether to install Elasticsearch automatically or you want to install manually yourself.

(We have already installed Elasticsearch, hence choose no).

Install Elasticsearch server locally for demo, must have at least 3G of memory, NOT recommended for production use (yes or no) [no] no [or SIMPLY PRESS ENTER]

Set Elasticsearch server URL, localhost:9200 in this setup. Just press Enter to accept the defaults.

Elasticsearch server URL [http://localhost:9200] ENTER

Set encryption password. Be sure to replace the password.

Password to encrypt S2S and other things [no-default] changeme

The configuration of Arkime then runs.

...
Moloch - Creating configuration files
Installing systemd start files, use systemctl
Moloch - Installing /etc/logrotate.d/moloch to rotate files after 7 days
Moloch - Installing /etc/security/limits.d/99-moloch.conf to make core and memlock unlimited
Download GEO files? (yes or no) [yes] yes
Moloch - Downloading GEO files
...

...
2021-11-12 20:46:36 URL:https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv [23328/23328] -> "/tmp/tmp.O4R9DwNay1" [1]
2021-11-12 20:46:37 URL:https://raw.githubusercontent.com/wireshark/wireshark/master/manuf [1920087/1920087] -> "/tmp/tmp.pa0Tq0aSb2" [1]

Arkime - Configured - Now continue with step 4 in /opt/arkime/README.txt

 4) The Configure script can install elasticsearch for you or you can install yourself
      systemctl start elasticsearch.service
 5) Initialize/Upgrade Elasticsearch Arkime configuration
  a) If this is the first install, or want to delete all data
      /opt/arkime/db/db.pl http://ESHOST:9200 init
  b) If this is an update to a moloch/arkime package
      /opt/arkime/db/db.pl http://ESHOST:9200 upgrade
 6) Add an admin user if a new install or after an init
      /opt/arkime/bin/arkime_add_user.sh admin "Admin User" THEPASSWORD --admin
 7) Start everything
      systemctl start arkimecapture.service
      systemctl start arkimeviewer.service
 8) Look at log files for errors
      /opt/arkime/logs/viewer.log
      /opt/arkime/logs/capture.log
 9) Visit http://arkimeHOST:8005 with your favorite browser.
      user: admin
      password: THEPASSWORD from step #6

If you want IP -> Geo/ASN to work, you need to setup a maxmind account and the geoipupdate program.
See https://arkime.com/faq#maxmind

Any configuration changes can be made to /opt/arkime/etc/config.ini
See https://arkime.com/faq#moloch-is-not-working for issues

Additional information can be found at:
  * https://arkime.com/faq
  * https://arkime.com/settings

Running Elasticsearch

Start and enable Elasticsearch to run on system boot;

systemctl enable --now elasticsearch

Verify if Elasticsearch is running;

curl http://localhost:9200

{
  "name" : "debian11",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "tm5rhTHyTleSIwP6NMZBjA",
  "version" : {
    "number" : "7.15.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "93d5a7f6192e8a1a12e154a2b81bf6fa7309da0c",
    "build_date" : "2021-11-04T14:04:42.515624022Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Initialize Elasticsearch Moloch configuration

Run the command below to initialize Elasticsearch Arkime/Moloch configuration.

/opt/arkime/db/db.pl http://localhost:9200 init

It is STRONGLY recommended that you stop ALL Arkime captures and viewers before proceeding.  Use 'db.pl http://localhost:9200 backup' to backup db first.

There is 1 elastic search data node, if you expect more please fix first before proceeding.

This is a fresh Arkime install
Erasing
Creating
Finished

Create Arkime/Moloch Admin User Account

You can use the /opt/arkime/bin/arkime_add_user.sh script to create Arkime/Moloch user account;

/opt/arkime/bin/arkime_add_user.sh -h

addUser.js []    []

Options:
  --admin               Has admin privileges
  --apionly             Can only use api, not web pages
  --email               Can do email searches
  --expression    Forced user expression
  --remove              Can remove data (scrub, delete tags)
  --webauth             Can auth using the web auth header or password
  --webauthonly         Can auth using the web auth header only, password ignored
  --packetSearch        Can create a packet search job (hunt)
  --createOnly          Only create the user if it doesn't exist

Config Options:
  -c       Config file to use
  -n         Node name section to use in config file
  --insecure            Disable certificate verification for https calls

Run the command below to create Arkime/Moloch admin user account.

Replace the username and password accordingly.

/opt/arkime/bin/arkime_add_user.sh admin "Arkime SuperAdmin" changeme --admin

Running Arkime Services

Arkime is made up of 3 components:

• capture – A threaded C application that monitors network traffic, writes PCAP formatted files to disk, parses the captured packets, and sends metadata (SPI data) to elasticsearch.
• viewer – A node.js application that runs per capture machine. It handles the web interface and transfer of PCAP files.
• elasticsearch – The search database technology powering Arkime.

We already started Elasticsearch.

Now start and enable Moloch Capture and viewer services to run on system boot;

systemctl enable --now arkimecapture

systemctl enable --now arkimeviewer

Check the status;

systemctl status arkimecapture

● arkimecapture.service - Arkime Capture
     Loaded: loaded (/etc/systemd/system/arkimecapture.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-11-12 21:02:08 EAT; 27s ago
   Main PID: 4125 (sh)
      Tasks: 2 (limit: 1133)
     Memory: 30.2M
        CPU: 389ms
     CGroup: /system.slice/arkimecapture.service
             ├─4125 /bin/sh -c /opt/arkime/bin/capture -c /opt/arkime/etc/config.ini  >> /opt/arkime/logs/capture.log 2>&1
             └─4126 /opt/arkime/bin/capture -c /opt/arkime/etc/config.ini

Nov 12 21:02:07 debian11 systemd[1]: Starting Arkime Capture...
Nov 12 21:02:08 debian11 systemd[1]: Started Arkime Capture.

systemctl status arkimeviewer

● arkimeviewer.service - Arkime Viewer
     Loaded: loaded (/etc/systemd/system/arkimeviewer.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-11-12 21:02:33 EAT; 48s ago
   Main PID: 4147 (sh)
      Tasks: 8 (limit: 1133)
     Memory: 42.1M
        CPU: 2.457s
     CGroup: /system.slice/arkimeviewer.service
             ├─4147 /bin/sh -c /opt/arkime/bin/node viewer.js -c /opt/arkime/etc/config.ini  >> /opt/arkime/logs/viewer.log 2>&1
             └─4148 /opt/arkime/bin/node viewer.js -c /opt/arkime/etc/config.ini

Nov 12 21:02:33 debian11 systemd[1]: Started Arkime Viewer.

At this point, if you reboot your server, Arkime capture and viewer services may fail to start, due to slow starting of elasticsearch service.

Here is a temporary solution. Configure these services to start only when Elasticsearch starts. You may miss the traffic capture during the period when Elasticsearch is starting.

Add these lines;

After=network.target elasticsearch.service
Requires=network.target elasticsearch.service

You can use sed to update these services;

sed -i 's/network.target/network.target elasticsearch.service/' /etc/systemd/system/arkimecapture.service /etc/systemd/system/arkimeviewer.service

sed -i '/After=/a Requires=network.target elasticsearch.service' /etc/systemd/system/arkimecapture.service /etc/systemd/system/arkimeviewer.service

systemctl daemon-reload

This will ensure that Arkime capture and viewer will start only after Elasticsearch.

Log Files

You can find Arkime/Moloch logs and Elasticsearch logs on the log files;

/opt/arkime/logs/viewer.log

/opt/arkime/logs/capture.log

/var/log/elasticsearch/*

Adjusting Arkime/Moloch configurations;

if you ever want to update Arkime configs, check the configuration file /opt/arkime/etc/config.ini.

Accessing Arkime Web Interface

Moloch is listening on port 8005/tcp by default.

ss -altnp | grep 8005

LISTEN 0      511                     *:8005             *:*    users:(("node",pid=1021,fd=26))

If UFW is running, open this port on it to allow external access.

ufw allow 8005/tcp

You can then access Arkime/Moloch using the URL, http://ARKIMEHOST:8005 with your favorite browser.

Accessing Arkime with SSL/TLS

If you want to use SSL/TLS serts, update the lines below by uncommenting them and then specify the full paths to the files;

vim /opt/arkime/etc/config.ini

...
# Cert file to use, comment out to use http instead
#certFile=/opt/arkime/etc/arkime.cert
certFile=/opt/arkime/etc/arkime.cert
...
# Private key file to use, comment out to use http instead
#keyFile=/opt/arkime/etc/arkime.key
keyFile=/opt/arkime/etc/arkime.key
...

Next, restart Arkime viewer;

systemctl restart arkimeviewer

You can then access your Arkime using the url: https://ARKIMEHOST-DOMAIN-NAME:8005

You will be prompted to enter the basic user authentication credentials you create above. We didnt enable SSL in our case (screenshot below).

Upon successful authentication, you land on Arkime Web interface.

And that is how simple it is to install Arkime on Debian.

Reference

Arkime Installation README.txt

Arkime Demo (Credentials: arkime:arkime)

Arkime Demo

Other Tutorials

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu