Send Windows logs to Elastic Stack using Winlogbeat and Sysmon

0
2216

In this guide, we are going to learn how to send Windows logs to Elastic Stack using Winlogbeat and Sysmon. Winlogbeat is an Elastic Beat that is used to collect windows system application, security, system or hardware events. Sysmon (System Monitor) on the other hand is a windows application that is used to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time

In order to visualize and analyze the events collected by Winlogbeat/Sysmon, you need to have setup your Elastic Stack. See the links below on how to install and setup Elastic Stack on Fedora/CentOS/Ubuntu servers.

Install Elastic Stack 7 on Ubuntu 18.04/Debian 9.8

Install Elastic Stack 7 on Fedora 30/Fedora 29/CentOS 7

Send Windows Logs to Elastic Stack Using Winlogbeat and Sysmon

Install Winlogbeat and Sysmon on Windows 7

In this guide, we are going to use Windows 7 as our Windows system. Therefore, you need to install both Winlogbeat and Sysmon on your Windows 7 system in order to ship events to Elastic stack.

Install Winlogbeat on Windows 7

Navigate to Winlogbeat downloads page and download Winlogbeat zip file.

Once the download is done, extract the Winlogbeat zipped file, winlogbeat-7.2.0-windows-x86_64.zip.

When you extract, you should get a folder, winlogbeat-7.2.0-windows-x86_64.

Move the winlogbeat-7.2.0-windows-x86_64 directory to C:\Program Files and rename it to Winlogbeat. Your directory should look like as in below;

Install Elastic Winglobeat on Windows System

Next, to install Winlogbeat on Windows 7, you need to execute the install-service-winlogbeat.ps1 installation script. Hence, open the Powershell as the administrator and change to Winlogbeat directory by executing the command below;

cd C:\'Program Files'\Winlogbeat

Next, run the Winlogbeat installer as shown below;

.\install-service-winlogbeat.ps1

If you get the error, cannot be loaded because the execution of scripts is disabled on this system, as shown below, you need to enable the script execution.

PS C:\Program Files\Winlogbeat> .\install-service-winlogbeat.ps1
File C:\Program Files\Winlogbeat\install-service-winlogbeat.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details.
At line:1 char:33
+ .\install-service-winlogbeat.ps1 <<<<
    + CategoryInfo          : NotSpecified: (:) [], PSSecurityException
    + FullyQualifiedErrorId : RuntimeException

As a result, you need to execute the Winlogbeat script with unrestricted execution policy as shown in the command below;

PS C:\Program Files\Winlogbeat> PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-winlogbeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  winlogbeat         winlogbeat

Install Sysmon on Windows 7

Download Sysmon from the downloads page.

Once the download is complete, extract the contents of the zipped file to C:\Program Files directory.

Once the extraction is done, you folder should look like as in below;

Install Sysmon on Windows 7

Install Sysmon with md5 and sha256 hashing of process created, log loading of modules and monitoring network connections, open a CMD as an administrator and navigate to C:\Program Files\Sysmon and execute the command below;

cd C:\Program Files\Sysmon
C:\Program Files\Sysmon> sysmon -i -accepteula -h md5,sha256,imphash -l -n

System Monitor v10.2 - System activity monitor
Copyright (C) 2014-2019 Mark Russinovich and Thomas Garnier
Sysinternals - www.sysinternals.com

Sysmon installed.
SysmonDrv installed.
Starting SysmonDrv.
SysmonDrv started.
Starting Sysmon...
Sysmon started.

Configuring Winlogbeat

The main configuration file for Winlogbeat is C:\Program Files\Winlogbeat\winlogbeat.yml with the reference config file being C:\Program Files\Winlogbeat\winlogbeat.reference.yml.

To edit this file, you can use Notepad++.

By default, Winlogbeat is set to monitor application, security, and system logs, and logs from Sysmon.

...
winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h

  - name: System

  - name: Security
    processors:
      - script:
          lang: javascript
          id: security
          file: ${path.home}/module/security/config/winlogbeat-security.js

  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - script:
          lang: javascript
          id: sysmon
          file: ${path.home}/module/sysmon/config/winlogbeat-sysmon.js
...

If you need to see more event types, you can execute the command Get-EventLog * in PowerShell.

Under the general settings, we are going to setup the optional name of the Beat and the Tags associated with the events.

...
#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
name: winlogbeat

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]
tags: ["windows_systems"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
#  env: staging
...

Next, setup the Winlogbeat output. In this demo, we are sending the logs directly to Elasticsearch nodes.

...
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["localhost:9200"]
  hosts: ["192.168.43.104:9200", "192.168.43.105:9200", "192.168.43.106:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"
...

If Elasticsearch and Kibana are not running on the same host and you want to use Kibana Winlogbeat dashboards, you can specify Kibana URL. Kibana must be reachable on non-loopback address. For example;

...
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  #host: "localhost:5601"
  host: "192.168.43.104:5601"
...

Also, if you are using Logstash, you can comment out the Elasticsearch output and specify Logstash connection addresses.

...
#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["localhost:9200"]
  #hosts: ["192.168.43.104:9200", "192.168.43.105:9200", "192.168.43.106:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]
  hosts: ["192.168.43.104:5044"]
...

Winlogbeat configuration checks

Once done with configuration, save the file and run the configuration checks. To run the configuration checks, open Powershell as an administrator and execute the command below;

cd C:\'Program Files'\Winlogbeat
.\winlogbeat.exe test config -c .\winlogbeat.yml -e

If there is no error with the configuration, you should see the Config Ok.

...
2019-07-15T14:05:08.164+0300    INFO    instance/beat.go:292    Setup Beat: winlogbeat; Version: 7.2.0
2019-07-15T14:05:08.192+0300    INFO    [index-management]      idxmgmt/std.go:178      Set output.elasticsearch.index t
o 'winlogbeat-7.2.0' as ILM is enabled.
2019-07-15T14:05:08.225+0300    INFO    elasticsearch/client.go:166     Elasticsearch url: http://192.168.43.104:9200
2019-07-15T14:05:08.258+0300    INFO    elasticsearch/client.go:166     Elasticsearch url: http://192.168.43.105:9200
2019-07-15T14:05:08.290+0300    INFO    elasticsearch/client.go:166     Elasticsearch url: http://192.168.43.106:9200
2019-07-15T14:05:08.325+0300    INFO    [publisher]     pipeline/module.go:97   Beat name: winlogbeat
2019-07-15T14:05:08.356+0300    INFO    beater/winlogbeat.go:69 State will be read from and persisted to C:\Program Files\Winlogbeat\data\.winlogbeat.yml
Config OK

Load Elasticsearch templates

If Winlogbeat has a direction connection and is using Elasticearch as the output, it will automatically load the template. However, if you are using Logstash as the output, you need to manually load the Elasticsearch template. See example command to load the Elasticsearch template manually below;

cd C:\'Program Files'\Winlogbeat
.\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["192.168.43.104:9200"]'

Setup Kibana Dashboards

To load Winlogbeat default visualization dashboards, you need to have created the index pattern. Hence, navigate to Kibana and create the Winlogbeat index pattern.

create Winlogbeat index pattern

Next, if you are using Elasticsearch as your output, you can load the dashboards by running the setup command or enabling dashboard loading in the winlogbeat.yml (setup.dashboards.enabled: true) configuration.

cd C:\'Program Files'\Winlogbeat>
.\winlogbeat.exe setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

OR

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here or by using the `setup` command.
#setup.dashboards.enabled: false
setup.dashboards.enabled: true

If you are using Logstash as the output, run the command below to load the dashboards.

cd C:\'Program Files'\Winlogbeat>
.\winlogbeat.exe setup -e -E output.logstash.enabled=false -E output.elasticsearch.hosts=['192.168.43.104:9200']

Running Winlogbeat

To start or stop Winlogbeat, navigate to install directory and execute the commands below respectively.

PS C:\Program Files\Winlogbeat> Start-Service winlogbeat
PS C:\Program Files\Winlogbeat> Stop-Service winlogbeat

You can also manage the Winlogbeat from system services.

View Winlogbeat Kibana Dashboard

When Winlogbeat runs, you should now have the events and the dashboards.

send Windows logs to Elastic Stack using Winlogbeat and Sysmon

You can view Winlogbeat dashboards by navigating to Dashboards.

Winlogbeat Kibana dashboards

There you go. You can also check Winlogbeat events on Kibana SIEM.

That is all on how to send Windows logs to Elastic Stack using Winlogbeat and Sysmon. You can continue to explore Kibana SIEM and Winlogbeat dashboards to analyze your windows events. Enjoy.

Related Tutorials:

Setup Multi-node Elasticsearch 7.x Cluster on Fedora 30/Fedora 29/CentOS 7

Install and Configure Elastic Auditbeat on Ubuntu 18.04

Install Filebeat on Fedora 30/Fedora 29/CentOS 7

How to Debug Logstash Grok Filters

LEAVE A REPLY

Please enter your comment!
Please enter your name here