Backing up and Restoring a Cluster¶
What’s in this document?
Back up a cluster¶
The GraphDB cluster backup is similar to the repository backup with the complexity of synchronizing all concurrent cluster operations.
The master node performs the cluster backup by selecting a worker node with the latest timestamp and an OK status.
In addition to the standard repository backup, the backup also includes the latest entry from the master’s transaction log, so it can recover the master’s state as well.
GraphDB supports:
Full backup of the cluster
Incremental backup of the cluster
The backup image will go into a directory under the master node’s repository data directory, which by default is $GDB_HOME/data/repositories/<repositoryId>/backup.
Each master node maintains its own set of backup images. The set of backup images available from one master is entirely unrelated to the collections of images available from other masters. Backing up a second time with the same image name will overwrite the previous image with that name.
Full backup of the cluster¶
The full backup operation switch in read-only mode one of the workers and copy all its data to the master.
The operation accepts a single optional parameter that identifies the backup name. If the parameter is left null, then the name will become default.
The backup name will appear here: $GDB_HOME/data/repositories/<repositoryId>/backup/<backup-name>.
Start a full backup named default with:
curl -H 'content-type: application/json' -d '{"type":"exec", "mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId", "operation":"backup", "arguments":[null]}' http://masterNode:7200/jolokia/
Start a named backup 20190501 with:
curl -H 'content-type: application/json' -d '{"type":"exec", "mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId", "operation":"backup", "arguments":["20190501"]}' http://masterNode:7200/jolokia/
Note
At least two notification messages will be sent under a normal operation - one to indicate that the backup operation has been started, and another to suggest that it has been completed successfully.
Warning
The image name “default” is reserved for internal use only. Backing up to an image named “default” would interfere with the cluster’s failure recovery capabilities.
The master can use only images named “default” to recover worker nodes if there are no other workers available to serve the replication.
If you want to create a backup of a cluster on another, identical cluster, and then restore it from the latter, you need to do the following:
Make a backup of your cluster following the steps described in this section. A new backup folder is created:
$GDB_HOME/data/repositories/<repositoryId>/backup/<backup-name>.Copy this folder to the new cluster’s data folder:
<new-cluster-data-folder>/repositories/<new-repository-id>/backup/<backup-name>.Restore the backup on the new cluster by following the procedure below.
Incremental backup of the cluster¶
The incremental backups are optimized for large repositories requiring frequent recovery points. During an incremental backup, a worker running with the latest timestamp will check the modified pages since the last full backup, and store only these changes. The backup will also include the full backup of all plugin data.
The incremental backup operation expects three parameters:
The backup name of the last full backup, against which the cluster will calculate the delta
The incremental backup name
Whether to rebuild a full image from the delta ready for a direct restore
For example, if the standard backup pattern is to perform a full backup on Sunday and then every day an incremental backup:
# Performs a full backup on sunday
curl -H 'content-type: application/json' -d '{"type":"exec", "mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId", "operation":"backup", "arguments":["20190504-sunday"]}' http://masterNode:7200/jolokia/
# Performs the first incremental backup storing only the delta compared to Sunday
curl -H 'content-type: application/json' -d '{"type":"exec", "mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId", "operation":"incrementalBackup", "arguments":["20190504-sunday","20190505-monday", true]}' http://masterNode:7200/jolokia/
# Performs the second incremental backup storing only the delta compared to Sunday including the data from Monday
curl -H 'content-type: application/json' -d '{"type":"exec", "mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId", "operation":"incrementalBackup", "arguments":["20190504-sunday","20190506-tuesday", true]}' http://masterNode:7200/jolokia/
Restore a cluster¶
The cluster restore is initiated from a backup image stored on the master node. The backup image is replicated through all worker nodes in the cluster and propagated by other peered master nodes.
Start the cluster restore from the default backup with:
curl -H 'content-type: application/json' -d '{"type":"exec","mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId","operation":"restoreFromImage","arguments":[null]}' http://masterNode:7200/jolokia/
Start the cluster restore from a named backup with:
curl -H 'content-type: application/json' -d '{"type":"exec","mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId","operation":"restoreFromImage","arguments":["20190504-sunday"]}' http://masterNode:7200/jolokia/
To recover an incremental backup, you need to apply the delta to the latest full backup with the incremental backup recovery tool $GDB_HOME/bin/ibrtool.
The tool will list all incremental backups for the current master and provide you with instructions on how to do this.
After rebuilding the full image from the delta, the user can recover the incremental backup with:
curl -H 'content-type: application/json' -d '{"type":"exec","mbean":"ReplicationCluster:name=ClusterInfo\/repositoryId","operation":"restoreFromImage","arguments":["20190505-monday"]}' http://masterNode:7200/jolokia/
Note
At least two notification messages will be sent under normal the operation, one to indicate that the restore operation has been started, and another to suggest that it has been completed successfully.
Warning
After a successful restore operation, any updates executed from the moment of the backup image creation to the moment the cluster was restored will be irreversibly lost.
Upon failure of a restore operation, the cluster may be in an inconsistent state.
External stores like Solr and Elasticsearch synchronized by the GraphDB Connector will not be restored. You need to drop and recreate all connectors.
Warning
Trying to initiate a backup or restore while another backup or restore operation is already in progress on the same master node will be treated as an error, and will result in immediate failure. The failure of the second operation will not interfere with the operation that is already in progress.
If you want to restore the backup of a cluster on another, identical cluster, please follow the steps described above.