Event logging

Logs are raw events received from the Wazuh agents, external APIs, and network devices. The Wazuh server stores all logs indefinitely. To maximize space optimization, the Wazuh manager automatically compresses log files.

Wazuh manages two types of logs, internal logs from the Wazuh server and external logs from monitored endpoints. These logs are stored indefinitely in the /var/ossec/logs/ directory of the Wazuh server.

The table below describes the log files and their storage location on the Wazuh server.

Log storage file	Log source	Description
`/var/ossec/logs/ossec.log`	Internal	Stores all informational level logs generated by the Wazuh server.
`/var/ossec/logs/api.log`	Internal	Stores logs generated by the Wazuh application when interacting with the Wazuh server APIs.
`/var/ossec/logs/cluster.log`	Internal	Stores logs generated by the activities of the Wazuh cluster.
`/var/ossec/logs/integrations.log`	Internal	Stores logs generated by the Wazuh integration module when interfacing with third-party applications and systems.
`/var/ossec/logs/active-responses.log`	Internal	Stores logs generated by the Wazuh Active Response module.
`/var/ossec/logs/firewall/firewall.log`	Internal	Stores logs generated by the firewall.
`/var/ossec/logs/archives/archives.log`	External	Stores logs received from third-party applications and systems in plaintext.
`/var/ossec/logs/archives/archives.json`	External	Stores logs received from third-party applications and systems in JSON.

Log compression and rotation

Log files can quickly accumulate and consume significant disk space in a system. To prevent this, the Wazuh manager compresses logs during its rotation process, helping to manage disk usage efficiently and maintain system performance. The Wazuh manager compresses log files daily or when they reach a certain threshold (file size, age, time, and more) and archives them. In the log rotation process, Wazuh creates a new log file with the original name to continuously write new events.

Log files are compressed daily and digitally signed using MD5, SHA1, and SHA256 hashing algorithms. The compressed log files are stored in the /var/ossec/logs/ directory within nested directories bearing names with the following format accordingly:

The log file name, indicating the name of the original log file.
The year, indicating the name of the current year.
The month, indicating the name of the current month of the year.

For example, a /var/ossec/logs/archives/archives.log file compressed on the 13th APR, 2024 is stored in the …/archives/2024/Apr/ directory. You can see the contents of directory by executing the following command:

# ls -la /var/ossec/logs/archives/2024/Apr/

total 0
drwxr-x--- 2 wazuh wazuh 62 Apr 17 08:15 .
drwxr-x--- 4 wazuh wazuh 28 Apr 12 07:30 ..
-rw-r----- 1 wazuh wazuh  0 Apr 13 00:00 ossec-archive-13.log.gz
-rw-r----- 1 wazuh wazuh  0 Apr 13 00:00 ossec-archive-13.log.sum

As seen in the output above, the string ossec and suffix day of the current month are prepended and appended respectively to the name of the compressed file and its checksum.

Based on your needs, you might configure the compressed files for removal after a specified period. Additionally, you can move them to log management systems, backup servers, or cloud-based storage devices for longer-term retention.

Archiving event logs

Events are logs generated by applications, endpoints, and network devices. The Wazuh server stores all events it receives, whether or not they trigger a rule. These events are stored in the Wazuh archives located at /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/archives.json. Security teams use archived logs to review historical data of security incidents, analyze trends, and generate reports to hunt threats.

By default, the Wazuh archives are disabled because it stores logs indefinitely on the Wazuh server. When enabled, the Wazuh manager creates archived files to store and retain security data for compliance and forensic purposes.

Note

The Wazuh archives retain logs collected from all monitored endpoints, therefore consuming significant storage resources on the Wazuh server over time. So, it is important to consider the impact on disk space and performance before enabling them.

Enabling archiving

Perform the steps below to enable the archiving on your Wazuh server.

Edit the Wazuh manager configuration file /var/ossec/etc/ossec.conf and set the value of the highlighted fields below to yes:
```
<ossec_config>
  <global>
    <jsonout_output>yes</jsonout_output>
    <alerts_log>yes</alerts_log>
    <logall>yes</logall>
    <logall_json>yes</logall_json>

   ...
</ossec_config>
```
Where:
- <logall> enables or disables archiving of all log messages. When enabled, the Wazuh server stores the logs in a syslog format. The allowed values are yes and no.
- <logall_json> enables or disables logging of events. When enabled, the Wazuh server stores the events in a JSON format. The allowed values are yes and no.
Depending on the format you desire, you can set one or both values of the highlighted fields to yes. However, only the <logall_json>yes</logall_json> option allows you to create an index that can be used to visualize the events on the Wazuh dashboard.
Restart the Wazuh manager to apply the configuration changes:
```
# systemctl restart wazuh-manager
```

Depending on your chosen format, the file archives.log, archives.json, or both will be created in the /var/ossec/logs/archives/ directory on the Wazuh server.

Wazuh uses a default log rotation policy. It ensures that available disk space is conserved by rotating and compressing logs on a daily, monthly, and yearly basis.

Visualizing the events on the dashboard

Edit the Filebeat configuration file /etc/filebeat/filebeat.yml and change the value of archives: enabled from false to true:
```
archives:
 enabled: true
```
Restart Filebeat to apply the configuration changes:
```
# systemctl restart filebeat
```

Wazuh dashboard

Click the upper-left menu icon to open the main menu. Expand Dashboard management and navigate to Dashboards management > Index patterns. Next, click Create index pattern. Use wazuh-archives-* as the index pattern name, and set timestamp in the Time field drop-down list.

The GIF below shows how to create the index pattern.
To view the events on the dashboard, click the upper-left menu icon and navigate to Discover. Change the index pattern to wazuh-archives-*.

Use case: Detecting signed binary proxy execution

Signed binary proxy execution is a technique threat actors use to bypass application whitelisting by using trusted binaries to run malicious code. This technique is identified as T1218.010 based on the MITRE ATT&CK framework.

In this use case, we show how to abuse the Windows utility, regsvr32.exe, to bypass application controls. We then analyze events in the Wazuh archives to detect suspicious activity related to this technique.

Windows 11 configuration

Perform the steps below to install Sysmon and Atomic Red Team (ART) on a Windows 11 endpoint and emulate the signed binary proxy execution technique.

Sysmon integration

Perform the steps below to install and configure Sysmon on the Windows 11 endpoint.

Download Sysmon from the Microsoft Sysinternals page.
Download the Sysmon configuration file: sysmonconfig.xml.
Install Sysmon with the downloaded configuration file using PowerShell as an administrator:
```
> .\sysmon64.exe -accepteula -i .\sysmonconfig.xml
```
Add the following configuration within the <ossec_config> block to the Wazuh agent C:\Program Files (x86)\ossec-agent\ossec.conf file to specify the location to collect Sysmon logs:
```
<localfile>
  <location>Microsoft-Windows-Sysmon/Operational</location>
  <log_format>eventchannel</log_format>
</localfile>
```
Restart the Wazuh agent to apply the changes by running the following PowerShell command as an administrator:
```
> Restart-Service -Name Wazuh
```

Atomic Red Team installation

Perform the following steps to install the Atomic Red Team PowerShell module on a Windows 11 endpoint using PowerShell as an administrator.

By default, PowerShell restricts the execution of running scripts. Run the command below to change the default execution policy to RemoteSigned:
```
> Set-ExecutionPolicy RemoteSigned
```

Install the ART execution framework:

> IEX (IWR 'https://raw.githubusercontent.com/redcanaryco/invoke-atomicredteam/master/install-atomicredteam.ps1' -UseBasicParsing);
> Install-AtomicRedTeam -getAtomics

Import the ART module to use Invoke-AtomicTest function:

> Import-Module "C:\AtomicRedTeam\invoke-atomicredteam\Invoke-AtomicRedTeam.psd1" -Force

Use Invoke-AtomicTest function to show details of the technique T1218.010:

> Invoke-AtomicTest T1218.010 -ShowDetailsBrief

PathToAtomicsFolder = C:\AtomicRedTeam\atomics

T1218.010-1 Regsvr32 local COM scriptlet execution
T1218.010-2 Regsvr32 remote COM scriptlet execution
T1218.010-3 Regsvr32 local DLL execution
T1218.010-4 Regsvr32 Registering Non DLL
T1218.010-5 Regsvr32 Silent DLL Install Call DllRegisterServer

Attack emulation

Emulate the signed binary proxy execution technique on the Windows 11 endpoint.

Run the command below with Powershell as an administrator to perform the T1218.010 test:

> Invoke-AtomicTest T1218.010

PathToAtomicsFolder = C:\AtomicRedTeam\atomics

Executing test: T1218.010-1 Regsvr32 local COM scriptlet execution
Done executing test: T1218.010-1 Regsvr32 local COM scriptlet execution
Executing test: T1218.010-2 Regsvr32 remote COM scriptlet execution
Done executing test: T1218.010-2 Regsvr32 remote COM scriptlet execution
Executing test: T1218.010-3 Regsvr32 local DLL execution
Done executing test: T1218.010-3 Regsvr32 local DLL execution
Executing test: T1218.010-4 Regsvr32 Registering Non DLL
Done executing test: T1218.010-4 Regsvr32 Registering Non DLL
Executing test: T1218.010-5 Regsvr32 Silent DLL Install Call DllRegisterServer
Done executing test: T1218.010-5 Regsvr32 Silent DLL Install Call DllRegisterServer

Several calculator instances will pop up after a successful execution of the exploit.

Wazuh dashboard

Use the Wazuh archives to query and display events related to the technique being hunted. It's important to note that while consulting the archives, some events might already be captured as alerts on the Wazuh dashboard. You can use information from the Wazuh archives, including alerts and events that have no detection to create custom rules based on your specific requirements.

Apply a time range filter to view events that occurred within the last five minutes of when the test was performed. Filter to view logs from the specific Windows endpoint using agent.id, agent.ip or agent.name.

There are multiple hits that you can investigate to determine a correlation with the earlier attack emulation. For instance, you may notice a calculator spawning event similar to the one observed on the Windows endpoint during the test.
Type regsvr32 in the search bar to streamline and investigate events related to the regsvr32 utility.
Expand any of the events to view their associated fields.

Click on the JSON tab to view the JSON format of the archived logs.

You can extract and verify specific details on the activities such as commands, services, paths, and more from the JSON log. Below, you can identify the initial process creation and the attributes related to the executed command:

"data": {
      "win": {
        "eventdata": {
          "originalFileName": "REGSVR32.EXE",
          "image": "C:\\\\Windows\\\\SysWOW64\\\\regsvr32.exe",
          "product": "Microsoft® Windows® Operating System",
          "parentProcessGuid": "{45cd4aff-35fc-6463-6903-000000001300}",
          "description": "Microsoft(C) Register Server",
          "logonGuid": "{45cd4aff-2ce5-6463-2543-290000000000}",

         "parentCommandLine": "C:\\\\Windows\\\\system32\\\\regsvr32.exe  /s /i C:\\\\AtomicRedTeam\\\\atomics\\\\T1218.010\\\\bin\\\\AllTheThingsx86.dll",

         "processGuid": "{45cd4aff-35fc-6463-6a03-000000001300}",
          "logonId": "0x294325",
          "parentProcessId": "7652",
          "processId": "4064",
          "currentDirectory": "C:\\\\Users\\\\THECOT~1\\\\AppData\\\\Local\\\\Temp\\\\",
          "utcTime": "2023-05-16 07:51:24.512",
          "hashes": "SHA1=8E2C6B7F92A560E0E856F8533D62A1B10797828F,MD5=5F7264BD237FAEA46FB240785B78AFAC,SHA256=D9BE711BE2BF88096BB91C25DF775D90B964264AB25EC49CF04711D8C1F089F6,IMPHASH=73F03653209E82368127EB826216A6AD",
          "parentImage": "C:\\\\Windows\\\\System32\\\\regsvr32.exe",
          "ruleName": "technique_id=T1117,technique_name=Regsvr32",
          "company": "Microsoft Corporation",
          "commandLine": "  /s /i C:\\\\AtomicRedTeam\\\\atomics\\\\T1218.010\\\\bin\\\\AllTheThingsx86.dll",
          "integrityLevel": "High",
          "fileVersion": "10.0.22621.1 (WinBuild.160101.0800)",
          "user": "Windows11\\\\Testuser",
          "terminalSessionId": "2",
          "parentUser": "Windows11\\\\Testuser"
        },
        "system": {
          "eventID": "1",
          "keywords": "0x8000000000000000",
          "providerGuid": "{5770385f-c22a-43e0-bf4c-06f5698ffbd9}",
          "level": "4",
          "channel": "Microsoft-Windows-Sysmon/Operational",
          "opcode": "0",

         "message": "\"Process Create:\r\nRuleName: technique_id=T1117,technique_name=Regsvr32\r\nUtcTime: 2023-05-16 07:51:24.512\r\nProcessGuid: {45cd4aff-35fc-6463-6a03-000000001300}\r\nProcessId: 4064\r\nImage: C:\\Windows\\SysWOW64\\regsvr32.exe\r\nFileVersion: 10.0.22621.1 (WinBuild.160101.0800)\r\nDescription: Microsoft(C) Register Server\r\nProduct: Microsoft® Windows® Operating System\r\nCompany: Microsoft Corporation\r\nOriginalFileName: REGSVR32.EXE\r\nCommandLine:   /s /i C:\\AtomicRedTeam\\atomics\\T1218.010\\bin\\AllTheThingsx86.dll\r\nCurrentDirectory: C:\\Users\\THECOT~1\\AppData\\Local\\Temp\\\r\nUser: Windows11\\Testuser\r\nLogonGuid: {45cd4aff-2ce5-6463-2543-290000000000}\r\nLogonId: 0x294325\r\nTerminalSessionId: 2\r\nIntegrityLevel: High\r\nHashes: SHA1=8E2C6B7F92A560E0E856F8533D62A1B10797828F,MD5=5F7264BD237FAEA46FB240785B78AFAC,SHA256=D9BE711BE2BF88096BB91C25DF775D90B964264AB25EC49CF04711D8C1F089F6,IMPHASH=73F03653209E82368127EB826216A6AD\r\nParentProcessGuid: {45cd4aff-35fc-6463-6903-000000001300}\r\nParentProcessId: 7652\r\nParentImage: C:\\Windows\\System32\\regsvr32.exe\r\nParentCommandLine: C:\\Windows\\system32\\regsvr32.exe  /s /i C:\\AtomicRedTeam\\atomics\\T1218.010\\bin\\AllTheThingsx86.dll\r\nParentUser: Windows11\\Testuser\"",

         "version": "5",
          "systemTime": "2023-05-16T07:51:24.5131006Z",
          "eventRecordID": "88509",
          "threadID": "3960",
          "computer": "Windows11",
          "task": "1",
          "processID": "3156",
          "severityValue": "INFORMATION",
          "providerName": "Microsoft-Windows-Sysmon"
        }
      }
    },

Carrying out further investigations on other related events, you can see a process injection event created by the regsvr32 utility and the image loaded:

"data": {
      "win": {
        "eventdata": {
          "originalFileName": "mscoree.dll",
          "image": "C:\\\\Windows\\\\SysWOW64\\\\regsvr32.exe",
          "product": "Microsoft® Windows® Operating System",
          "signature": "Microsoft Windows",

         "imageLoaded": "C:\\\\Windows\\\\SysWOW64\\\\mscoree.dll",

         "description": "Microsoft .NET Runtime Execution Engine",
          "signed": "true",
          "signatureStatus": "Valid",
          "processGuid": "{45cd4aff-35fc-6463-6a03-000000001300}",
          "processId": "4064",
          "utcTime": "2023-05-16 07:51:24.774",
          "hashes": "SHA1=52A6AB3E468C4956C00707DF80C7609EEE74D9AD,MD5=BEE4D173DA78E4D3AC9B54A95C6A464A,SHA256=36B0BA10BBB6575CA4A4CBDE585F6E19B86B3A80014B3C3D8335F861D8AEBFAB,IMPHASH=47F306C12509ADBBC266F7DA43529A4D",
          "ruleName": "technique_id=T1055,technique_name=Process Injection",
          "company": "Microsoft Corporation",
          "fileVersion": "10.0.22621.1 (WinBuild.160101.0800)",
          "user": "Windows11\\\\Testuser"
        },
        "system": {
          "eventID": "7",
          "keywords": "0x8000000000000000",
          "providerGuid": "{5770385f-c22a-43e0-bf4c-06f5698ffbd9}",
          "level": "4",
          "channel": "Microsoft-Windows-Sysmon/Operational",
          "opcode": "0",

         "message": "\"Image loaded:\r\nRuleName: technique_id=T1055,technique_name=Process Injection\r\nUtcTime: 2023-05-16 07:51:24.774\r\nProcessGuid: {45cd4aff-35fc-6463-6a03-000000001300}\r\nProcessId: 4064\r\nImage: C:\\Windows\\SysWOW64\\regsvr32.exe\r\nImageLoaded: C:\\Windows\\SysWOW64\\mscoree.dll\r\nFileVersion: 10.0.22621.1 (WinBuild.160101.0800)\r\nDescription: Microsoft .NET Runtime Execution Engine\r\nProduct: Microsoft® Windows® Operating System\r\nCompany: Microsoft Corporation\r\nOriginalFileName: mscoree.dll\r\nHashes: SHA1=52A6AB3E468C4956C00707DF80C7609EEE74D9AD,MD5=BEE4D173DA78E4D3AC9B54A95C6A464A,SHA256=36B0BA10BBB6575CA4A4CBDE585F6E19B86B3A80014B3C3D8335F861D8AEBFAB,IMPHASH=47F306C12509ADBBC266F7DA43529A4D\r\nSigned: true\r\nSignature: Microsoft Windows\r\nSignatureStatus: Valid\r\nUser: Windows11\\Testuser\"",

         "version": "3",
          "systemTime": "2023-05-16T07:51:24.7768916Z",
          "eventRecordID": "88510",
          "threadID": "3960",
          "computer": "Windows11",
          "task": "7",
          "processID": "3156",
          "severityValue": "INFORMATION",
          "providerName": "Microsoft-Windows-Sysmon"
        }
      }
    },

Apply the data.win.eventdata.ruleName:technique_id=T1218.010,technique_name=Regsvr32 filter to see the technique ID as shown below.
Expand the event to view its associated fields.

Click on the JSON tab to view the JSON format of the archived logs.

From the below log, you can extract more structured details which makes it easier to analyze the event:

"data": {
      "win": {
        "eventdata": {
          "destinationPort": "443",
          "image": "C:\\\\Windows\\\\System32\\\\regsvr32.exe",
          "sourcePort": "63754",
          "initiated": "true",
          "destinationIp": "1.1.123.23",
          "protocol": "tcp",
          "processGuid": "{45cd4aff-36b5-645a-9e07-000000000e00}",
          "sourceIp": "192.168.43.16",
          "processId": "4704",
          "utcTime": "2023-05-09 21:19:25.361",

         "ruleName": "technique_id=T1218.010,technique_name=Regsvr32",

         "destinationIsIpv6": "false",
          "user": "Windows11\\\\Testuser",
          "sourceIsIpv6": "false"
        },
        "system": {
          "eventID": "3",
          "keywords": "0x8000000000000000",
          "providerGuid": "{5770385f-c22a-43e0-bf4c-06f5698ffbd9}",
          "level": "4",
          "channel": "Microsoft-Windows-Sysmon/Operational",
          "opcode": "0",

         "message": "\"Network connection detected:\r\nRuleName: technique_id=T1218.010,technique_name=Regsvr32\r\nUtcTime: 2023-05-09 21:19:25.361\r\nProcessGuid: {45cd4aff-36b5-645a-9e07-000000000e00}\r\nProcessId: 4704\r\nImage: C:\\Windows\\System32\\regsvr32.exe\r\nUser: Windows11\\Testuser\r\nProtocol: tcp\r\nInitiated: true\r\nSourceIsIpv6: false\r\nSourceIp: 192.168.43.16\r\nSourceHostname: -\r\nSourcePort: 63754\r\nSourcePortName: -\r\nDestinationIsIpv6: false\r\nDestinationIp: 185.199.108.133\r\nDestinationHostname: -\r\nDestinationPort: 443\r\nDestinationPortName: -\"",

         "version": "5",
          "systemTime": "2023-05-09T12:04:07.0231156Z",
          "eventRecordID": "63350",
          "threadID": "3096",
          "computer": "Windows11",
          "task": "3",
          "processID": "3156",
          "severityValue": "INFORMATION",
          "providerName": "Microsoft-Windows-Sysmon"
        }
      }
    },

You can use events from the Wazuh archives to develop detection logic and write custom decoders and rules. You can also use the out-of-the-box wazuh-logtest tool to test and verify rules against provided logs. For more information, see the Custom rules, Custom decoders, and wazuh-logtest documentation.