Upgrading Elastic Stack from 7.x to 7.y

Prepare the Elastic Stack

  1. Stop the services:

    # systemctl stop filebeat
    # systemctl stop kibana
    
  2. In case of having disabled the repository for Elastic Stack 7.x it can be enabled using:

    • For CentOS/RHEL/Fedora:

      # sed -i "s/^enabled=0/enabled=1/" /etc/yum.repos.d/elastic.repo
      
    • For Debian/Ubuntu:

      # sed -i "s/#deb/deb/" /etc/apt/sources.list.d/elastic-7.x.list
      # apt-get update
      
    • For openSUSE:

      # sed -i "s/^enabled=0/enabled=1/" /etc/zypp/repos.d/elastic.repo
      

Upgrade Elasticsearch

  1. Disable shard allocation

    curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
    {
      "persistent": {
        "cluster.routing.allocation.enable": "primaries"
      }
    }
    '
    
  2. Stop non-essential indexing and perform a synced flush. (Optional)

    curl -X POST "localhost:9200/_flush/synced"
    
  3. Shut down a single node.

    # systemctl stop elasticsearch
    
  4. Upgrade the node you shut down.

    • For CentOS/RHEL/Fedora:

      # yum install elasticsearch-7.9.2
      
    • For Debian/Ubuntu:

      # apt-get install elasticsearch=7.9.2
      
  5. Restart the service.

    # systemctl daemon-reload
    # systemctl restart elasticsearch
    
  6. Start the newly-upgraded node and confirm that it joins the cluster by checking the log file or by submitting a _cat/nodes request:

    curl -X GET "localhost:9200/_cat/nodes"
    
  7. Reenable shard allocation.

    curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
    {
      "persistent": {
        "cluster.routing.allocation.enable": null
      }
    }
    '
    
  8. Before upgrading the next node, wait for the cluster to finish shard allocation.

    curl -X GET "localhost:9200/_cat/health?v"
    
  9. Repeat it for every Elasticsearch node.

Upgrade Filebeat

  1. Upgrade Filebeat.

    • For CentOS/RHEL/Fedora:

      # yum install filebeat-7.9.2
      
    • For Debian/Ubuntu:

      # apt-get install filebeat=7.9.2
      
  2. Update the configuration file.

    # cp /etc/filebeat/filebeat.yml /backup/filebeat.yml.backup
    # curl -so /etc/filebeat/filebeat.yml https://raw.githubusercontent.com/wazuh/wazuh/v3.13.6/extensions/filebeat/7.x/filebeat.yml
    # chmod go+r /etc/filebeat/filebeat.yml
    
  3. Download the alerts template for Elasticsearch:

    # curl -so /etc/filebeat/wazuh-template.json https://raw.githubusercontent.com/wazuh/wazuh/v3.13.6/extensions/elasticsearch/7.x/wazuh-template.json
    # chmod go+r /etc/filebeat/wazuh-template.json
    
  4. Download the Wazuh module for Filebeat:

    # curl -s https://packages.wazuh.com/3.x/filebeat/wazuh-filebeat-0.1.tar.gz | sudo tar -xvz -C /usr/share/filebeat/module
    
  5. Edit the file /etc/filebeat/filebeat.yml and replace YOUR_ELASTIC_SERVER_IP with the IP address or the hostname of the Elasticsearch server. For example:

    output.elasticsearch.hosts: ['http://YOUR_ELASTIC_SERVER_IP:9200']
    
  6. Restart Filebeat.

    # systemctl daemon-reload
    # systemctl restart filebeat
    

Upgrade Kibana

Warning

Since Wazuh 3.12.0 release (regardless of the Elastic Stack version) the location of the wazuh.yml has been moved from /usr/share/kibana/plugins/wazuh/wazuh.yml to /usr/share/kibana/optimize/wazuh/config/wazuh.yml.

  1. Copy the wazuh.yml to its new location. (Only needed for upgrades from 3.11.x to 3.12.y).

    # mkdir -p /usr/share/kibana/optimize/wazuh/config
    # cp /usr/share/kibana/plugins/wazuh/wazuh.yml /usr/share/kibana/optimize/wazuh/config/wazuh.yml
    
  2. Remove the Wazuh app.

    # cd /usr/share/kibana/
    # sudo -u kibana bin/kibana-plugin remove wazuh
    
  3. Upgrade Kibana.

    • For CentOS/RHEL/Fedora:

      # yum install kibana-7.9.2
      
    • For Debian/Ubuntu:

      # apt-get install kibana=7.9.2
      
  4. Remove generated bundles.

    # rm -rf /usr/share/kibana/optimize/bundles
    
  5. Update file permissions. This will prevent errors when generating new bundles or updating the app.

    # chown -R kibana:kibana /usr/share/kibana/optimize
    # chown -R kibana:kibana /usr/share/kibana/plugins
    
  6. Install the Wazuh app.

    • From URL:

      # cd /usr/share/kibana/
      # sudo -u kibana bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.13.6_7.9.2.zip
      
    • From the package:

      # cd /usr/share/kibana/
      # sudo -u kibana bin/kibana-plugin install file:///path/wazuhapp-3.13.6_7.9.2.zip
      
  7. Update configuration file permissions.

    # sudo chown kibana:kibana /usr/share/kibana/optimize/wazuh/config/wazuh.yml
    # sudo chmod 600 /usr/share/kibana/optimize/wazuh/config/wazuh.yml
    
  8. For installations on Kibana 7.6.X versions it is recommended to increase the heap size of Kibana to ensure the Kibana's plugins installation:

    # cat >> /etc/default/kibana << EOF
    NODE_OPTIONS="--max_old_space_size=2048"
    EOF
    
  9. Restart Kibana.

    # systemctl daemon-reload
    # systemctl restart kibana
    

Disabling repositories

  • For CentOS/RHEL/Fedora:

    # sed -i "s/^enabled=1/enabled=0/" /etc/yum.repos.d/elastic.repo
    
  • For Debian/Ubuntu:

    # sed -i "s/^deb/#deb/" /etc/apt/sources.list.d/elastic-7.x.list
    # apt-get update
    

    Alternatively, you can set the package state to hold, which will stop updates (although you can still upgrade it manually using apt-get install).

    # echo "elasticsearch hold" | sudo dpkg --set-selections
    # echo "kibana hold" | sudo dpkg --set-selections
    
  • For openSUSE:

    # sed -i "s/^enabled=1/enabled=0/" /etc/zypp/repos.d/elastic.repo