Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

0
641
Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

Welcome to our tutorial on how to install Arkime (Moloch) Full Packet Capture tool on Ubuntu. Arkime, formerly Moloch “is a large scale, open source, indexed packet capture and search system“.

According to its Github repository page, some of the features of Arkime tool include;

  • It stores and indexes network traffic in standard PCAP format, providing fast, indexed access.
  • Provides an intuitive web interface for PCAP browsing, searching, and exporting.
  • Exposes APIs which allow for PCAP data and JSON formatted session data to be downloaded and consumed directly.
  • Stores and exports all packets in standard PCAP format, allowing you to also use your favorite PCAP ingesting tools, such as wireshark, during your analysis workflow.

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

You can either install Arkime (Moloch) Full Packet Capture tool on Ubuntu using prebuilt binary packages or simply build it from the source yourself.

Installing Arkime using Prebuilt Binary on Ubuntu

Download Arkime Binary Installer

In order to install Arkime using the prebuilt binary on Ubuntu, navigate to the downloads page and grab the binary installer for your Ubuntu flavour, which in my setup is Ubuntu 20.04.

You can as well grab the link to the binary installer and pull it using curl or wget command. For example, the command below downloads the current stable release version of Arkime binary installer for Ubuntu 20.04;

wget https://s3.amazonaws.com/files.molo.ch/builds/ubuntu-20.04/moloch_2.7.1-1_amd64.deb

Run System Update

Update your system package cache;

apt update

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

Next, install Arkime (Moloch) Full Packet Capture tool on Ubuntu using the downloaded binary installer.

apt install ./moloch_2.7.1-1_amd64.deb

If you want, you can as well build Arkime by building it from the source. Check the installation page for instructions.

Install Elasticsearch on Ubuntu

Arkime uses Elasticsearch as a search and indexing engine. Therefore, install Elasticsearch by running the command below;

Import the Elastic stack PGP repository signing Key

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch --no-check-certificate | sudo apt-key add -

Install Elasticsearch APT repository;

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Update package cache and install Elasticsearch;

apt update
apt install elasticsearch

Configure Elasticsearch JVM options depending on the size of your memory;

vim /etc/elasticsearch/jvm.options
...
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms512m
-Xmx512m
...

Save and exit the configuration file.

Configure Arkime (Moloch) on Ubuntu

Configuring Arkime

Once the installation is done, run the script below to configure Arkime (Moloch);

Answer the script prompts accordingly;

/data/moloch/bin/Configure

Select an interface to monitor;

Found interfaces: lo;enp0s3;enp0s8
Semicolon ';' seperated list of interfaces to monitor [eth1] enp0s8

Choose whether to install Elasticsearch automatically or you want to install manually yourself (We have already installed Elasticsearch, hence choose no).

Install Elasticsearch server locally for demo, must have at least 3G of memory, NOT recommended for production use (yes or no) [no] no

Set Elasticsearch server URL, localhost:9200 in this setup. Just press Enter to accept the defaults.

Elasticsearch server URL [http://localhost:9200] ENTER

Set encryption password. Be sure to replace the password.

Password to encrypt S2S and other things [no-default] changeme

The configuration of Arkime then runs.

...
Moloch - Creating configuration files
Installing systemd start files, use systemctl
Moloch - Installing /etc/logrotate.d/moloch to rotate files after 7 days
Moloch - Installing /etc/security/limits.d/99-moloch.conf to make core and memlock unlimited
Download GEO files? (yes or no) [yes] yes
Moloch - Downloading GEO files
...

Running Elasticsearch

Start and enable Elasticsearch to run on system boot;

systemctl enable --now elasticsearch

Verify if Elasticsearch is running;

curl http://localhost:9200
{
  "name" : "ubuntu20",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "bc-twCeeSsSmgjYT6HlkXA",
  "version" : {
    "number" : "7.11.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
    "build_date" : "2021-02-15T13:44:09.394032Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Initialize Elasticsearch Moloch configuration

Run the command below to initialize Elasticsearch Arkime/Moloch configuration.

/data/moloch/db/db.pl http://localhost:9200 init

Create Arkime/Moloch Admin User Account

You can use the /data/moloch/bin/moloch_add_user.sh script to create Arkime/Moloch user account;

/data/moloch/bin/moloch_add_user.sh --help
addUser.js [<config options>] <user id> <user friendly name> <password> [<options>]

Options:
  --admin               Has admin privileges
  --apionly             Can only use api, not web pages
  --email               Can do email searches
  --expression  <expr>  Forced user expression
  --remove              Can remove data (scrub, delete tags)
  --webauth             Can auth using the web auth header or password
  --webauthonly         Can auth using the web auth header only, password ignored
  --packetSearch        Can create a packet search job (hunt)

Config Options:
  -c <config file>      Config file to use
  -n <node name>        Node name section to use in config file
  --insecure            Allow insecure HTTPS

Run the command below to create Arkime/Moloch admin user account. Replace the username and password accordingly.

/data/moloch/bin/moloch_add_user.sh admin "Moloch SuperAdmin" changeme --admin

Running Arkime Services

Arkime is made up of 3 components:

  • capture – A threaded C application that monitors network traffic, writes PCAP formatted files to disk, parses the captured packets, and sends metadata (SPI data) to elasticsearch.
  • viewer – A node.js application that runs per capture machine. It handles the web interface and transfer of PCAP files.
  • elasticsearch – The search database technology powering Arkime.

We already started Elasticsearch.

Now start and enable Moloch Capture and viewer services to run on system boot;

systemctl enable --now molochcapture
systemctl enable --now molochviewer

Check the status;

systemctl status molochcapture
● molochcapture.service - Moloch Capture
     Loaded: loaded (/etc/systemd/system/molochcapture.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-02-23 18:57:16 UTC; 1min 23s ago
   Main PID: 4002 (sh)
      Tasks: 7 (limit: 4620)
     Memory: 224.9M
     CGroup: /system.slice/molochcapture.service
             ├─4002 /bin/sh -c /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini  >> /data/moloch/logs/capture.log 2>&1
             └─4003 /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini

Feb 23 18:57:15 ubuntu20 systemd[1]: Starting Moloch Capture...
Feb 23 18:57:16 ubuntu20 systemd[1]: Started Moloch Capture.
systemctl status molochviewer
● molochviewer.service - Moloch Viewer
     Loaded: loaded (/etc/systemd/system/molochviewer.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2021-02-23 18:57:40 UTC; 14s ago
   Main PID: 4054 (sh)
      Tasks: 12 (limit: 4620)
     Memory: 51.5M
     CGroup: /system.slice/molochviewer.service
             ├─4054 /bin/sh -c /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini  >> /data/moloch/logs/viewer.log 2>&1
             └─4055 /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini

Feb 23 18:57:40 ubuntu20 systemd[1]: Started Moloch Viewer.

Log Files

You can find Arkime/Moloch logs and Elasticsearch logs on the log files;

/data/moloch/logs/viewer.log
/data/moloch/logs/capture.log
/var/log/elasticsearch/*
Adjusting Arkime/Moloch configurations;

if you ever want to update Arkime/Moloch configs, check the configuration file /data/moloch/etc/config.ini.

Accessing Arkime/Moloch Web Interface

Moloch is listening on port 8005/tcp by default.

If UFW is running, open this port on it to allow external access.

ufw allow 8005/tcp

You can then access Arkime/Moloch using the URL, http://MOLOCHHOST:8005 with your favorite browser.

You will be prompted to enter the basic user authentication credentials you create above.

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

Upon successful authentication, you land on Arkime Web interface.

Install Arkime (Moloch) Full Packet Capture tool on Ubuntu
Install Arkime (Moloch) Full Packet Capture tool on Ubuntu

And that is how simple it is to install Arkime (Moloch) Full Packet Capture tool on Ubuntu.

Reference

Arkime Installation README.txt

Further Reading

Arkime Settings page

Arkime Faqs

LEAVE A REPLY

Please enter your comment!
Please enter your name here