Restore Elasticsearch Snapshot to another Cluster

In this tutorial, we will try to show how to restore Elasticsearch snapshot to another Cluster. Elasticsearch data can be backed up by taking a snapshot of the running Elasticsearch cluster. In our previous tutorial, we learnt how to backup and restore a single node Elasticsearch cluster. Link is provided below;

Backup and Restore Elasticsearch Index Data

Similarly, in this tutorial, we will still be dealing with a single node Elasticsearch cluster data backup and restore.

The snapshot was taken on Elasticsearch 7.10.1 and we are restoring to Elasticsearch 7.12.1. Read more on version compatibility.

Restoring Elasticsearch Snapshot to another Cluster

As already stated, we are dealing with a single node Elasticsearch cluster.

For the purposes of this demo, we have separate single Elasticsearch cluster nodes. We will call them nodeA and nodeB

Take Snapshot of Elasticsearch on NodeA

Before you can restore Elasticsearch data, you need to have taken snapshot of the Elasticsearch cluster, specific indices or data streams on the first node (nodeA).

To take a backup/snapshot of the Elasticsearch cluster;

1. Register a snapshot repository
2. Create Elasticsearch Snapshot/Backup

Take Snapshot of Elasticsearch on NodeB

Once you have registered and taken a snapshot of the Elasticsearch data on the nodeA, do the same on second Elasticsearch server, nodeB in this case, using the same settings as in the nodeA.

Ensure there is enough space on the other cluster to accommodate all the data backed up.

We have attached a storage disk on nodeB mounted on /mnt/es_backup just like as it was on nodeA;

df -hT -P /mnt/es_backup/

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sdb1      ext4  3.9G   16M  3.7G   1% /mnt/es_backup

Define the location of the path to the backup location on Elasticsearch configuration file, use the option, path.repo;

echo 'path.repo: ["/mnt/es_backup"]' >> /etc/elasticsearch/elasticsearch.yml

Set the ownership of the repository path to elasticsearch user.

chown -R elasticsearch: /mnt/es_backup/

Restart elasticsearch.

systemctl restart elasticsearch

Register Backup repository;

curl -X PUT "192.168.59.12:9200/_snapshot/es_backup?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "fs",
  "settings": {
    "location": "/mnt/es_backup"
  }
}
'

Output;

{
  "acknowledged" : true
}

Create Snapshot Elasticsearch Cluster on NodeB

Create a snapshot with the same name as the other snapshot on NodeA;

curl -X PUT "192.168.59.12:9200/_snapshot/es_backup/es_backup_202104192200?pretty"

Sample output;

{
  "accepted" : true
}

Listing the contents of the repository directory;

ls -1 /mnt/es_backup/

index-0
index.latest
meta-uEpnUzM4QOKOqT0g05Jg5g.dat
snap-uEpnUzM4QOKOqT0g05Jg5g.dat

Delete Snapshot Data on NodeB repository

Since we are going to restore snapshot data from another cluster, nodeA in this setup, delete the contents the snapshot data on NodeB.

rm -rf /mnt/es_backup/*

Copy Snapshot Data from NodeA to NodeB repository

Next, copy the snapshot data from NodeA to NodeB repository path;

rsync -avP /mnt/es_backup/ [email protected]:/mnt/es_backup/

Listing contents of the repository path on NodeB after copying;

s -1 /mnt/es_backup/

index-0
index.latest
indices
meta-33qzhT82QTmvH4GkWn-vhw.dat
snap-33qzhT82QTmvH4GkWn-vhw.dat

Restart Elasticsearch on NodeB

Once you have copied the snapshot data to the other node backup/snapshot repository, proceed to restart Elasticsearch service.

systemctl restart elasticsearch

View Snapshot Information

Check the details about the snapshot;

curl -X GET "192.168.59.12:9200/_snapshot/es_backup/es_backup_202104192200?pretty"

{
  "snapshots" : [
    {
      "snapshot" : "es_backup_202104192200",
      "uuid" : "33qzhT82QTmvH4GkWn-vhw",
      "version_id" : 7100099,
      "version" : "7.10.0",
      "indices" : [
        ".kibana_task_manager_1",
        "filebeat-7.12.0-2021.04.19-000001",
        "filebeat-7.10.1-2021.04.16-000001",
        ".kibana-event-log-7.10.0-000001",
        ".async-search",
        ".apm-agent-configuration",
        "ilm-history-3-000001",
        ".kibana_1",
        ".apm-custom-link"
      ],
      "data_streams" : [ ],
      "include_global_state" : true,
      "state" : "SUCCESS",
      "start_time" : "2021-04-19T19:57:08.912Z",
      "start_time_in_millis" : 1618862228912,
      "end_time" : "2021-04-19T19:57:56.691Z",
      "end_time_in_millis" : 1618862276691,
      "duration_in_millis" : 47779,
      "failures" : [ ],
      "shards" : {
        "total" : 9,
        "failed" : 0,
        "successful" : 9
      }
    }
  ]
}

The details should match those of the snapshot in the original node.

Restoring Elasticsearch Snapshot to another Cluster

You can now restore Elasticsearch data to another cluster.

curl -X POST "192.168.59.12:9200/_snapshot/es_backup/es_backup_202104192200/_restore?pretty"

{
  "accepted" : true
}

Verify Indices

You can list indices using the command below;

curl -XGET "192.168.59.12:9200/_cat/indices?pretty"

They should be same as on the previous node;

yellow open filebeat-7.10.1-2021.04.16-000001 SUcNGbsPRN6bvkPrAfEiPw 1 1    24  0  146kb  146kb
green  open .apm-custom-link                  6O37J9vLS1eqplnEovhdaQ 1 0     0  0   208b   208b
yellow open filebeat-7.12.0-2021.04.19-000001 4ElgYLt9Qceo73onTw-UqA 1 1 66423  0 15.5mb 15.5mb
green  open .kibana_task_manager_1            ueVULNo-R92kcXMrPQaeXg 1 0     5  1 98.8kb 98.8kb
green  open .apm-agent-configuration          39Qhl6AgTBmIvo4MOyK7_w 1 0     0  0   208b   208b
green  open .kibana-event-log-7.10.0-000001   sp0-b6FZTKK3gGHzVkJy8w 1 0     2  0   11kb   11kb
green  open .async-search                     -8UhlSbyS2Oyfs_BUA6OEg 1 0     0  0   231b   231b
green  open .kibana_1                         C08RBXLhSG2NZ1scoxSb3w 1 0  1555 12 10.7mb 10.7mb

You should similarly be having same data on your Kibana.

Reference

Restore a snapshot

StackOverflow

Other tutorials

Setup Kibana Elasticsearch and Fluentd on CentOS 8

Setup Multi-node Elasticsearch 7.x Cluster on Fedora 30/Fedora 29/CentOS 7

Update/Change Kibana Visualization Index Pattern