Alert management
Alerts are notifications generated by the Wazuh manager after processing events received from Wazuh agents and agentless devices. By default, the alerts are stored in the /var/ossec/logs/alerts/alerts.log
and /var/ossec/logs/alerts/alerts.json
files.
By default, the Wazuh server uses Filebeat to forward generated alerts to the Wazuh indexer for indexing. Additionally, you can configure the Wazuh manager to forward alerts to other systems involving syslog servers, email systems, and databases.
Alert threshold
An alert threshold is the minimum severity level that must be exceeded for an alert to be triggered. The Wazuh manager assigns a severity level to each event from monitored endpoints based on the matching rule from the ruleset. By default, it only triggers alerts with a severity level of 3
or higher.
Configuration
The alert threshold is configured in the /var/ossec/etc/ossec.conf
configuration file on the Wazuh server within the <alerts>
XML tag.
The code block below shows the default alert threshold configuration for events and forwarding alerts via email:
<ossec_config>
<alerts>
<log_alert_level>3</log_alert_level>
<email_alert_level>12</email_alert_level>
</alerts>
</ossec_config>
Where:
The
<log_alert_level>
tag sets the minimum severity level to trigger alerts stored in the/var/ossec/logs/alerts/alerts.log
and/or the/var/ossec/logs/alerts/alerts.json
file. The default value is3
. The allowed value is any integer from1
to16
as referenced in the rules classification guide.The
<email_alert_level>
tag sets the minimum severity level for an alert to generate an email notification. The default value is12
. The allowed value is any integer from1
to16
. This setting overrides granular email alert configuration. However, thealert_by_email
option within individual rules can override both global and granular alert level thresholds to trigger an email alert.
For details on configuring an alert threshold, refer to the alerts reference guide.
Note
Restart the Wazuh manager when you make any changes to the configuration file. This action ensures that the changes take effect.
Restart the Wazuh manager via the command line interface with the following command:
# systemctl restart wazuh-manager
# service wazuh-manager restart
Forwarding alerts
The Wazuh manager forwards alerts to the Wazuh indexer by default for indexing and analytics capabilities. Additionally, the Wazuh manager provides the capability of configuring and forwarding alerts to other systems for analysis and backup.
Configuring syslog output
You can configure the Wazuh server to send alerts to a syslog server using the syslog_output option. Forwarding alerts to a syslog server can be useful for centralized monitoring and custom reporting.
Configuration
Syslog output is configured in the Wazuh server /var/ossec/etc/ossec.conf
configuration file within the <ossec_config>
block. By default, the Wazuh manager forwards alerts to syslog servers using port 514
via the UDP protocol.
The code block below shows a sample configuration to forward alerts to a syslog server:
<ossec_config>
<syslog_output>
<level>9</level>
<server>192.168.1.241</server>
</syslog_output>
</ossec_config>
The configuration options are defined as follows:
The
<level>
tag sets the minimum severity level of the alerts to be forwarded to the syslog server. The example value9
indicates the Wazuh server only forwards alerts to the syslog server if the alert level is higher than9
. If this option is not defined, the Wazuh server forwards all alerts to the syslog server.The
<server>
tag sets the IP address or hostname of the syslog server to forward the alerts. The IP address192.168.1.241
in the configuration is used as an example.
You can learn more about the available configuration options in the syslog_output reference guide.
Restart the Wazuh manager service to apply the changes after every configuration:
# systemctl restart wazuh-manager
# service wazuh-manager restart
You can also forward alerts to multiple syslog servers by defining multiple <syslog_output>
blocks within the <ossec_config>
block in the /var/ossec/etc/ossec.conf
configuration file.
<ossec_config>
<syslog_output>
<server>192.168.1.240</server>
</syslog_output>
<syslog_output>
<level>9</level>
<server>192.168.1.241</server>
</syslog_output>
</ossec_config>
In the configuration above,
The first
<syslog_output>
block sends all alerts without filtering to the syslog server with IP address192.168.1.240
.The second
<syslog_output>
block sends alerts to the syslog server192.168.1.241
only if the alert level is higher than9
.
Configuring email alerts
Wazuh provides a feature to send alerts to email systems when they are generated on a Wazuh server. You can configure it to send email alerts to one or more email addresses when rules are triggered or based on customized settings. This configuration can help you for daily event reports and more.
A sample email sent by Wazuh when the rule ID 553
is triggered is shown below:
Wazuh Notification.
2024 Apr 29 08:58:30
Received From: wazuh-server->syscheck
Rule: 553 fired (level 7) -> "File deleted."
Portion of the log(s):
File '/var/ossec/test_dir/somefile.
txt' deleted
Mode: realtime
Attributes:
- Size: 0
- Permissions: rw-r--r--
- Date: Mon Apr 29 08:46:12 2024
- Inode: 841858
- User: root (0)
- Group: root (0)
- MD5: d41d8cd98f00b204e9800998ecf8427e
- SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
- SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
--END OF NOTIFICATION
Generic email options
To configure Wazuh to send email alerts, we configure the email options within the <global>
section of the /var/ossec/etc/ossec.conf
file.
A sample email configuration to send alerts to the email address me@test.com
is shown below:
<ossec_config>
<global>
<email_notification>yes</email_notification>
<email_to>me@test.com</email_to>
<smtp_server>mail.test.com</smtp_server>
<email_from>wazuh@test.com</email_from>
</global>
...
</ossec_config>
Once the above has been configured, the email_alert_level
option needs to be set to the minimum alert level to trigger an email. By default, this level is set to 12
.
This example configuration below sets the minimum level to send email alerts to 10
:
<ossec_config>
<alerts>
<email_alert_level>10</email_alert_level>
</alerts>
...
</ossec_config>
For more information about the available email configuration options, see the global and alerts reference guides.
Restart the Wazuh manager service to apply the changes after every configuration:
# systemctl restart wazuh-manager
# service wazuh-manager restart
Warning
Wazuh doesn't handle SMTP authentication. If your email service uses this, you need to configure a server relay.
Granular email options
Wazuh allows granular configuration options for email alerts. This setting extends the generic email options configured in the <global>
section of the /var/ossec/etc/ossec.conf
file. Granular email configurations are defined within the <email_alerts>
tag of the /var/ossec/etc/ossec.conf
file.
Below are some sample granular configurations for sending alerts via email. For more information, see the email_alerts section.
Warning
The minimum severity level configured in the <alerts>
section applies to and overrides these granular email configurations. For example, if you configure the Wazuh manager to send an email when rule 526
is triggered, but the rule has a level lower than the minimum level specified in the <alerts>
section, the alert will not be sent.
Email alert based on level
This option configures the Wazuh manager to send email alerts when the severity level is equal to or greater than the set value. This option is configured as follows:
<email_alerts>
<email_to>you@example.com</email_to>
<level>4</level>
<do_not_delay/>
</email_alerts>
This configuration allows the Wazuh manager to send an email to you@example.com
when any rule with a level greater than or equal to 4
is triggered.
Note
If the severity level here is less than the email_alert_level
configured in the <alerts>
section, the email will not be sent.
Email alert based on the event location
The event_location
option involves sending email alerts based on the location from where the event originated. The generated alert must match the event location to be forwarded via email. The allowed values for this option are the Wazuh agent name, hostname, IP address, or log file.
This option is configured as follows:
<email_alerts>
<email_to>you@example.com</email_to>
<event_location>server1</event_location>
<do_not_delay/>
</email_alerts>
This configuration allows the Wazuh manager to send an email to you@example.com
when the events that generated the alerts originated on the Wazuh agent with name server1
.
Learn more about the event_location option in the reference guide.
Email based on rules ID
The rule_id
option is used to send alert emails based on rule IDs. This option limits the sending of emails only when specific defined rules are tripped.
This option is configured as follows:
<email_alerts>
<email_to>you@example.com</email_to>
<rule_id>515, 516</rule_id>
<do_not_delay/>
</email_alerts>
This configuration allows the Wazuh manager to send an email to you@example.com
when the rules 515
or 516
are triggered.
Learn more about the rule_id option in the reference guide.
Email based on the rule group
The group
option can be configured to send an email based on one or more rule groups that alerts belong to.
This option is configured as follows:
<email_alerts>
<email_to>you@example.com</email_to>
<group>pci_dss_10.6.1,</group>
</email_alerts>
This configuration allows the Wazuh manager to send an email to you@example.com
when any rule that is part of the pci_dss_10.6.1
group is triggered on any Wazuh monitored endpoint.
Learn more about the group option in the reference guide.
Multiple options and multiple emails
Email alerts can be sent to multiple email addresses, each one with unique criteria.
The example configuration below shows how to send email alerts with multiple criteria to multiple email addresses:
<ossec_config>
<email_alerts>
<email_to>alice@test.com</email_to>
<event_location>endpoint1|endpoint2</event_location>
</email_alerts>
<email_alerts>
<email_to>is@test.com</email_to>
<event_location>/log/secure$</event_location>
</email_alerts>
<email_alerts>
<email_to>bob@test.com</email_to>
<event_location>192.168.</event_location>
</email_alerts>
<email_alerts>
<email_to>david@test.com</email_to>
<level>12</level>
</email_alerts>
</ossec_config>
This configuration sends:
An email to
alice@test.com
if any alert onendpoint1
orendpoint2
is triggered.An email to
is@test.com
if the alerts came from the/log/secure
file.An email to
bob@test.com
if the alerts came from any endpoint on the192.168.0.0/24
network.An email to
david@test.com
if the alerts have a level equal to or higher than12
.
Force forwarding an alert by email
The minimum severity level to send an alert via email is 12
by default. You can configure the Wazuh manager to forcefully send an email alert below the configured minimum severity level. To do so, you need to use one of the rule options below:
alert_by_email
to always alert by email.no_email_alert
to never alert by email.no_log
to not log this alert.
For example, the rule definition below sends an email every time rule 502
is triggered, regardless of what the minimum severity level is set to:
<rule id="502" level="3">
<if_sid>500</if_sid>
<options>alert_by_email</options>
<match>Ossec started</match>
<description>Ossec server started.</description>
</rule>
SMTP server with authentication
Wazuh email alerts do not support SMTP servers with authentication, such as Gmail. However, you can send these emails via a server relay like Postfix.
Perform the steps below on your relay server to configure Postfix with Gmail.
Run this command to install the required packages. Select No configuration, if prompted about the mail server configuration type.
# yum update && yum install postfix mailx cyrus-sasl cyrus-sasl-plain
# apt-get update && apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules
Append these lines to the
/etc/postfix/main.cf
file to configure Postfix. Create the file if missing.relayhost = [smtp.gmail.com]:587 smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd smtp_sasl_security_options = noanonymous smtp_tls_CAfile = /etc/ssl/certs/ca-bundle.crt smtp_use_tls = yes
relayhost = [smtp.gmail.com]:587 smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd smtp_sasl_security_options = noanonymous smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt smtp_use_tls = yes smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination
Set the credentials of the sender in the
/etc/postfix/sasl_passwd
file and create a database file for Postfix. Replace the<USERNAME>
and<PASSWORD>
variables with sender’s email address username and password respectively.# echo [smtp.gmail.com]:587 <USERNAME>@gmail.com:<PASSWORD> > /etc/postfix/sasl_passwd # postmap /etc/postfix/sasl_passwd
Note
The password must be an App Password. App Passwords can only be used with accounts that have 2-Step Verification turned on.
Secure your password DB file so that only the
root
user has full read and write access to it. This is because the/etc/postfix/sasl_passwd
and/etc/postfix/sasl_passwd.db
files have plaintext credentials.# chown root:root /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db # chmod 0600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db
Restart Postfix to effect the configuration changes:
# systemctl restart postfix
# service postfix restart
Run the following command to test the configuration:
# echo "Test mail from postfix" | mail -s "Test Postfix" -r "<CONFIGURED_EMAIL>" <RECEIVER_EMAIL>
Replace:
<CONFIGURED_EMAIL>
with your configured email address.<RECEIVER_EMAIL>
with the email address of the recipient.
The command sends an email to the receiver’s email with subject
Test Postfix
and bodyTest mail from postfix
.If you get the error message
fatal: tls_fprint: error computing md5 message digest
in the/var/log/maillog
file, run the following commands to switch Postfix from the defaultMD5
hashing function toSHA-256
:# postconf -e smtp_tls_fingerprint_digest=sha256 # postconf -e smtpd_tls_fingerprint_digest=sha256
Configure email notifications within the
<global>
tag of the Wazuh server’s/var/ossec/etc/ossec.conf
file as follows:<global> <email_notification>yes</email_notification> <smtp_server>localhost</smtp_server> <email_from><USERNAME>@gmail.com</email_from> <email_to><RECEIVER_EMAIL></email_to> </global>
Where:
<email_notification>
toggles the use of email alerting.<smtp_server>
defines the SMTP server to use to deliver alerts.<email_from>
specifies the email address of the configured sender. Replace<USERNAME>
with your configured username of your email address.<email_to>
specifies the email address of the recipient of alerts. Replace<RECEIVER_EMAIL>
with the email address of the recipient.
Restart the Wazuh manager to apply the changes:
# systemctl restart wazuh-manager
# service wazuh-manager restart
Configuring database output
Wazuh supports forwarding alerts to database systems. You can configure the Wazuh manager to output generated alerts into a database. To achieve this configuration, you must compile the Wazuh manager from sources with the database type that you want to use. Wazuh currently supports MySQL and PostgreSQL databases.
Note
This guide assumes you have already installed MySQL or PostgreSQL and know how to create the users and the databases.
Prerequisites
You need to install the development libraries for the database system you want to configure and compile the Wazuh manager to use the required database system.
Install the development libraries for the database system:
For MySQL:
# yum install mysql-devel
# apt-get install libmysqlclient-dev
For PostgreSQL:
# yum install postgresql-devel
# apt-get install libpq-dev
Install the dependencies as outlined in the installing dependencies section.
Download and extract the latest version of Wazuh:
# curl -Ls https://github.com/wazuh/wazuh/archive/v4.9.1.tar.gz | tar zx
Execute the following commands to switch to the Wazuh directory and specify the database type to use, replacing the
<DATABASE_TYPE>
variable with eithermysql
orpgsql
:# cd wazuh-4.9.1/src # make deps && make TARGET=server DATABASE=<DATABASE_TYPE>
Note
Depending on your system specifications, the compilation process might take some time.
Run the
install.sh
script. It displays a wizard to guide you through the installation process using the Wazuh sources:# cd .. # ./install.sh
When the script asks what kind of installation you want, type
manager
to install the Wazuh manager:1- What kind of installation do you want (manager, agent, local, hybrid, or help)? manager
Note
During the installation, you can decide the installation path. Execute the
install.sh
and select the language, set the installation mode tomanager
, then set the installation path (Choose where to install Wazuh [/var/ossec]
). The default path of installation is/var/ossec
. A commonly used custom path might be/opt
.Warning
Be extremely careful not to select a critical installation directory if you choose a different path than the default. If the directory already exists, the installer will ask to delete the directory or proceed by installing Wazuh inside it.
The installer asks if you want to start Wazuh at the end of the installation. If you choose not to, you can start it later with the command below:
# systemctl restart wazuh-manager
# service wazuh-manager restart
Database configuration
According to your database system, create a new database, set up the database user, and add the schema located in the src/os_dbd
directory of the source code with the following commands:
For MySQL:
# mysql -u root -p
mysql> CREATE DATABASE Alerts_DB; Query OK, 1 row affected (2.34 sec) mysql> CREATE USER '<DATABASE_USER>'@'<DATABASE_SERVER_IP>' IDENTIFIED BY '<DATABASE_USER_PASSWORD>'; Query OK, 0 rows affected (0.00 sec) mysql> GRANT INSERT,SELECT,UPDATE,CREATE,DELETE,EXECUTE on Alerts_DB.* to '<DATABASE_USER>'@'<DATABASE_SERVER_IP>'; Query OK, 0 rows affected (0.00 sec) mysql> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec) mysql> quit;
Replace the following variables in the commands above:
<DATABASE_USER>
with the user you want to create for the database server.<DATABASE_SERVER_IP>
with the IP address of the database server.<DATABASE_USER_PASSWORD>
with the user password to access the database server.
# mysql -u root -p Alerts_DB < src/os_dbd/mysql.schema
For PostgreSQL:
# sudo -u postgres createuser -P <DATABASE_USER> # sudo -u postgres createdb -O <DATABASE_USER> Alerts_DB # psql -U <DATABASE_USER> -d Alerts_DB -f src/os_dbd/postgresql.schema
Replace
<DATABASE_USER>
with the user you want to create for the database server.
Note
You will be prompted twice to provide a password when creating the user. Note this password as it is required when configuring the Wazuh manager.
Wazuh manager configuration
Perform the following steps to configure the Wazuh manager to send alerts and other data to the database system.
Add the following block of code within the
<ossec_config>
block of the/var/ossec/etc/ossec.conf
file on the Wazuh server:For MySQL:
<database_output> <hostname><DATABASE_SERVER_IP></hostname> <username><DATABASE_USER></username> <password><DATABASE_USER_PASSWORD></password> <database>Alerts_DB</database> <type>mysql</type> </database_output>
For PostgreSQL:
<database_output> <hostname><DATABASE_SERVER_IP></hostname> <username><DATABASE_USER></username> <password><DATABASE_USER_PASSWORD></password> <database>Alerts_DB</database> <type>postgresql</type> </database_output>
Where:
<hostname>
specifies the IP address of the database server. Replace<DATABASE_SERVER_IP>
the IP address of the database server.<username>
specifies the user to access the database. Replace<DATABASE_USER>
with the database user created above.<password>
specifies the user password to access the database. Replace<DATABASE_USER_PASSWORD>
with the user password created above.<database>
specifies the name of the database in which to store the alerts. For example,Alerts_DB
as specified in the configuration above.<type>
specifies the type of database (MySQL or PostgreSQL). The allowed values aremysql
orpgsql
.
For more information on the database_output option, see the database_output section of the reference guide.
Restart the Wazuh manager service to apply the changes:
# systemctl restart wazuh-manager
# service wazuh-manager restart
Run the command below to verify that the Wazuh manager is connected to the database:
# grep wazuh-dbd /var/ossec/logs/ossec.log
2024/06/24 14:49:11 wazuh-dbd: INFO: Connected to database 'Alerts_DB' at '127.0.0.1'.
The database will now start receiving data from the Wazuh manager.