Welcome to our guide on how to install ModSecurity 3 with Nginx on Debian 12. Modsecurity is an open source, cross platform web application firewall (WAF) which provides a robust event-based programming language which protects web applications against a wide range of attacks such as SQL injection, Cross-site Scripting (XSS), Local File Include, Remote File Include e.tc. It also allows for HTTP traffic monitoring, logging and real-time analysis.

Installing ModSecurity 3 with Nginx on Debian 12

Install Required Build Tools and Dependencies

To install ModSecurity 3 with Nginx, you need to build it from source.

You will also need to build Nginx from the source code in order to enable suppport for ModSecurity.

Hence, you need to install some required build tools and dependencies for a successful build.

apt install g++ \
	flex \
	bison \
	curl \
	apache2-dev \
	doxygen \
	libyajl-dev \
	ssdeep \
	liblua5.2-dev \
	libgeoip-dev \
	libtool \
	dh-autoreconf \
	libcurl4-gnutls-dev \
	libxml2 \
	libpcre3-dev \
	libxml2-dev \
	zlib1g \
	zlib1g-dev \
	libssl-dev \
	git -y

Compile and Install ModSecurity on Debian 12

Download ModSecurity Source Code

Navigate to ModSecurity releases page and download the source code for ModSecurity 3. You can simply use wget to pull it.

VER=3.0.10

wget https://github.com/SpiderLabs/ModSecurity/releases/download/v${VER}/modsecurity-v${VER}.tar.gz

Extract the ModSecurity source code.

tar xzf modsecurity-v${VER}.tar.gz

Configure ModSecurity 3 Source Code on Debian 12

Navigate to ModSecurity 3.x source directory and configure it;

cd modsecurity-v${VER}

./build.sh

You can safely ignore the fatal: not a git repository (or any of the parent directories): .git error and the obsolete warning messages.

./configure

Be sure to fix any dependency issue, if any, before you can proceed to compile and install ModSecurity 3 with Nginx.

If the configure script above completes with no error, proceed to compile and install ModSecurity 3 with Nginx.

Compile and Install ModSecurity 3 on Debian 12

From within each ModSecurity version source directory, execute the commands below to install them;

make

make install

Install Nginx with ModSecurity 3 Support on Debian 12

Next, you need to compile Nginx with support for Modsecurity.

Therefore, download the ModSecurity-nginx connector which provides a communication channel between Nginx and LibModsecurity by cloning its git repository.

cd ~

git clone https://github.com/SpiderLabs/ModSecurity-nginx.git

Next, download the latest staple version of Nginx from Nginx downloads. The latest version as of this writing is version nginx-1.24.0.tar.gz.

Replace the value of E_VER below with the version of Nginx release;

E_VER=1.24.0

wget https://nginx.org/download/nginx-${E_VER}.tar.gz

Extract the archive.

tar xzf nginx-${E_VER}.tar.gz

Remove any installed versions of Nginx;

apt remove --purge --auto-remove nginx -y

Create a non-privileged Nginx system user and group.

useradd -r -M -s /sbin/nologin -d /usr/local/nginx nginx

Navigate to Nginx source directory and configure it.

cd nginx-${E_VER}

./configure --user=nginx \
	--group=nginx \
	--with-pcre-jit \
	--with-debug \
	--with-compat \
	--with-http_ssl_module \
	--with-http_realip_module \
	--add-dynamic-module=/root/ModSecurity-nginx \
	--http-log-path=/var/log/nginx/access.log \
	--error-log-path=/var/log/nginx/error.log

Configuration summary;

...
creating objs/Makefile

Configuration summary
  + using system PCRE2 library
  + using system OpenSSL library
  + using system zlib library

  nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/local/nginx/sbin/nginx"
  nginx modules path: "/usr/local/nginx/modules"
  nginx configuration prefix: "/usr/local/nginx/conf"
  nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx/logs/nginx.pid"
  nginx error log file: "/var/log/nginx/error.log"
  nginx http access log file: "/var/log/nginx/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

Compile and install Nginx on Ubuntu 22.05.

make

Create dynamic modules;

make modules

Install Nginx;

make install

You can check nginx version, compiler version, and configure script parameters with the command;

ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/

nginx -V

Sample output;

nginx version: nginx/1.24.0
built by gcc 12.2.0 (Debian 12.2.0-14) 
built with OpenSSL 3.0.9 30 May 2023
TLS SNI support enabled
configure arguments: --user=nginx --group=nginx --with-pcre-jit --with-debug --with-compat --with-http_ssl_module --with-http_realip_module --add-dynamic-module=/root/ModSecurity-nginx --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log

Confgure Nginx with ModSecurity 3 on Debian 12

Copy the sample ModSecurity configuration file on the source directory to Nginx configuration directory.

cp ~/modsecurity-v${VER}/modsecurity.conf-recommended /usr/local/nginx/conf/modsecurity.conf

Copy the unicode.mapping file from ModSecurity source directory to Nginx configuration directory.

cp ~/modsecurity-v${VER}/unicode.mapping /usr/local/nginx/conf/

Next, edit Nginx configuration file and make the changes as shown below.

Create a backup of the Nginx configuration file.

cp /usr/local/nginx/conf/nginx.conf{,.bak}

Configure Nginx such that your configuration may look like;

cat /usr/local/nginx/conf/nginx.conf

load_module modules/ngx_http_modsecurity_module.so;
user  nginx;
worker_processes  1;
pid        /run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  nginx.kifarunix-demo.com;
        modsecurity  on;
        modsecurity_rules_file  /usr/local/nginx/conf/modsecurity.conf;
        access_log  /var/log/nginx/access_kifarunix-demo.log;
        error_log  /var/log/nginx/error_kifarunix-demo.log;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

The lines below turns on Modsecurity on the root web document and specific the location of the Modsecurity rules.

modsecurity on;
modsecurity_rules_file /usr/local/nginx/conf/modsecurity.conf;

Note that ModSecurity 3 can be turned on per directory basis.

The line below loads the Modsecurity Nginx modules;

load_module modules/ngx_http_modsecurity_module.so;

Create Nginx log directory.

[ -d /var/log/nginx ] || mkdir /var/log/nginx 

Turn on ModSecurity by changing the value of SecRuleEngine to On.

sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /usr/local/nginx/conf/modsecurity.conf

Install OWASP ModSecurity Core Rule Set (CRS)

The OWASP ModSecurity Core Rule Set (CRS) is a set of generic attack detection rules for use with ModSecurity. It aims at protecting the web applications from a wide range of attacks, including the OWASP Top Ten, minimum of false alerts.

Download the CRS from GitHub repository to /usr/local/nginx/conf/ as shown below;

cd ~

wget https://github.com/coreruleset/coreruleset/archive/refs/tags/v3.3.5.tar.gz

mkdir /usr/local/nginx/conf/owasp-crs

tar xzf v3.3.5.tar.gz --strip-components=1 -C /usr/local/nginx/conf/owasp-crs/

Next, rename crs-setup.conf.example to crs-setup.conf.

sudo cp /usr/local/nginx/conf/owasp-crs/crs-setup.conf{.example,}

Once the OWASP rules are in place, configure ModSecurity to use these rules. You therefore need to enter the following lines on the ModSecurity configuration file to tell it where to find the rules.

echo -e "Include owasp-crs/crs-setup.conf
Include owasp-crs/rules/*.conf" >> /usr/local/nginx/conf/modsecurity.conf

This command will append the lines below on /usr/local/nginx/conf/modsecurity.conf.

Include owasp-crs/crs-setup.conf
Include owasp-crs/rules/*.conf

Enable Modsecurity Self-contained mode. In this configuration the default disruptive action becomes ‘deny’. After a rule triggers, it will stop processing the request and return an error 403.

vim /usr/local/nginx/conf/owasp-crs/crs-setup.conf

...
#SecDefaultAction "phase:1,log,auditlog,pass"
#SecDefaultAction "phase:2,log,auditlog,pass"

...
#
SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"
...

Save the changes and exit the file.

Verify Nginx configuration file again.

nginx -t

Output should be;

nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

Next, create Nginx Systemd service unit.

cat > /etc/systemd/system/nginx.service << 'EOL'
[Unit]
Description=A high performance web server and a reverse proxy server
Documentation=man:nginx(8)
After=network.target nss-lookup.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/local/nginx/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/local/nginx/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target
EOL

Reload system;

systemctl daemon-reload

Then Start Nginx;

systemctl enable --now nginx

Check the status;

systemctl status nginx

● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/etc/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Mon 2023-08-07 15:00:56 EDT; 1min 14s ago
       Docs: man:nginx(8)
    Process: 33618 ExecStartPre=/usr/local/nginx/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 33619 ExecStart=/usr/local/nginx/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 33620 (nginx)
      Tasks: 2 (limit: 2307)
     Memory: 19.7M
        CPU: 213ms
     CGroup: /system.slice/nginx.service
             ├─33620 "nginx: master process /usr/local/nginx/sbin/nginx -g daemon on; master_process on;"
             └─33621 "nginx: worker process"

Aug 07 15:00:56 debian systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
Aug 07 15:00:56 debian systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argument
Aug 07 15:00:56 debian systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server.

Testing Modsecurity

Next, test the effectiveness of Modsecurity, for example, command injection. Run the command below;

curl localhost?doc=/bin/ls

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.24.0</center>
</body>
</html>

Check Modsecurity logs;

tail /var/log/modsec_audit.log

---kpenmi6q---H--
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:doc' (Value: `/bin/ls' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "496"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/ls found within ARGS:doc: /bin/ls"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION"] [tag "WASCTC/WASC-31"] [tag "OWASP_TOP_10/A1"] [tag "PCI/6.5.2"] [hostname "127.0.0.1"] [uri "/"] [unique_id "16914349490.619224"] [ref "o1,6v10,7t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase"]

---kpenmi6q---I--

---kpenmi6q---J--

---kpenmi6q---Z--

Access the site from browser and test command injection attack, http://domain.name/?exec=/bin/bash.

Check Nginx Error log;

tail /var/log/nginx/error_kifarunix-demo.log

2023/08/07 15:02:29 [error] 33621#0: *1 [client 127.0.0.1] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:doc' (Value: `/bin/ls' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "496"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/ls found within ARGS:doc: /bin/ls"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION"] [tag "WASCTC/WASC-31"] [tag "OWASP_TOP_10/A1"] [tag "PCI/6.5.2"] [hostname "127.0.0.1"] [uri "/"] [unique_id "16914349490.619224"] [ref "o1,

Similarly, install Nikto on Ubuntu and use it to scan the server to test the modsecurity rules;

Install and Use Nikto Web Scanner on Ubuntu

As a result, this confirms that the Modsecurity is functioning as expected.

That is just about it on our guide on how to install ModSecurity with Nginx on Debian 12.

Reference:

ModSecurity-nginx

ModSecurity

Other Tutorials

Process ModSecurity Logs using Wazuh

Install ModSecurity with Apache on Debian 12