Elastic Stack

Elastic Stack is a unified suite of popular open source projects for log management, including Elasticsearch, Kibana, Filebeat, and others. The projects that are especially relevant to the Wazuh solution are:

  • Filebeat: A lightweight forwarder used to convey logs across a network, usually to Elasticsearch. It is used on the Wazuh server to ship events and alerts to Elasticsearch. It reads the output of the Wazuh analysis engine and ships events in real time through an encrypted channel. It also provides load balancing when connected to a multi-node Elasticsearch cluster.
  • Elasticsearch: A highly scalable, full-text search and analytics engine. Elasticsearch is distributed, meaning the data indices are divided into shards and each shard can have zero or more replicas. Wazuh uses different indices for alerts data, raw events, and status monitoring information.
  • Kibana: A flexible and intuitive web interface for mining, analyzing, and visualizing data. It runs on top of the indexed content in an Elasticsearch cluster. Wazuh web user interface has been fully embedded in Kibana, in the form of a plugin. It includes out-of-the-box dashboards for security events, regulatory compliance (e.g. PCI DSS, GDPR, CIS, HIPAA, NIST 800-53), detected vulnerable applications, file integrity monitoring data, configuration assessment results, cloud infrastructure monitoring events, and others.

Wazuh integrates with Elastic Stack, providing a feed of already decoded messages to be indexed by Elasticsearch, as well as a real-time web console for alert and log data analysis. In addition, the Wazuh user interface, which runs on top of Kibana, is used for management and monitoring of the Wazuh infrastructure.

An Elasticsearch index is a collection of documents that have somewhat similar characteristics (like certain common fields and shared data retention requirements). Wazuh utilizes as many as three different indices, created daily, to store different event types:

  • wazuh-alerts: Index for alerts generated by the Wazuh server. Those are created each time an event trips a rule with a high enough priority (this threshold is configurable).
  • wazuh-events: Index for all events (archive data) received from the agents, whether or not they trip a rule.
  • wazuh-monitoring: Index for data related to the Wazuh agents status over time. It is used by the web interface to represent when individual agents are or have been Active, Disconnected, or Never connected.

An index is composed of documents. For the indices above, documents are individual alerts, archived events, or the Wazuh agents status-related data.

An Elasticsearch index is divided into one or more shards, and each shard can optionally have one or more replicas. Each primary and replica shard is an individual Lucene index. Thus, an Elasticsearch index is made up of many Lucene indexes. When a search is run on an Elasticsearch index, the search is executed on all the shards in parallel and the results are merged. Dividing Elasticsearch indexes into multiple shards and replicas is used in multiple-node Elasticsearch clusters, with the purpose of scaling out searches and for high availability. Single-node Elasticsearch clusters normally have only one shard per index and no replicas.