How GraphDB EE works

WRITE requests (loading data)

Every WRITE request is sent to all available workers. The master keeps a log of the recent transactions, while all workers keep a full copy of the repository data.

../_images/GraphDB_write.png

READ requests (query evaluations)

Every READ request (SPARQL query) is passed to one of the available worker nodes. The node is chosen based on the number of queries currently running on it. This ensures that the read load is spread throughout the cluster.

The available nodes are organised in a priority queue, which is rearranged after every read operation. If an error occurs (time-out, lost connection, etc.), the request is resent to one of the other available nodes and so on until it is successfully processed.

../_images/GraphDB_read.png

Updates (transaction logs)

Each update is first logged. Then a probe node is selected and the request is sent to it. If the update is successful, control queries are evaluated against it in order to check its consistency with respect to the data it holds. If this control suite passes, the update is forwarded to the rest of the nodes in the cluster.

../_images/update_success.png

During the handling of an update, the nodes that do not successfully process it are disabled and scheduled for recovery replication.

../_images/update_failure.png

Limitations

Warning

An update executed on a GraphDB EE cluster must be completely deterministic in terms of changing the workers state.

For example, the following update causes a synchronisation error, because the result of the update is different for each worker node:

PREFIX ex:<http://example.com#/>
INSERT { ex:a ex:p ?timestamp }
WHERE { BIND( now() as ?timestamp ) }

i.e. the value of ?timestamp will be set according to the time of the update execution at each worker. After promulgating this update to all worker nodes, the master will detect the difference in each worker nodes signature (which will not match any previous state in its logs) and will trigger a full replication between the worker that first received the update and all other worker nodes.

Adding a worker node

Worker node synchronisation

When a master with the full transaction history detects a new worker, it just synchronises the updates with the new worker.

../_images/GraphDB_synchronise.png

Worker node replication

When a master with the recent transaction history detects a new worker, it selects another worker and instructs it to copy its data on the new one.

Note

Masters keep only limited transactions. You can configure how many transaction you want the master to keep and for how long with the parameters LogMaxSize & LogMaxDepth in the JMX console.

  • LogMaxSize: the maximum number of stored transactions in the log;
  • LogMaxDepth: the maximum minutes to store transactions in the log.
../_images/GraphDB_replication1.png

Managing out-of-sync worker

Master with two worker nodes

When a master, which is in a read-write mode detects an out-of-syncworker, it goes in a read-only mode. If it has enough history, it replays the updates to the out-of-syncworker (synchronisation). Otherwise, it selects another (in-synch) worker at random and instructs it to replicate directly to the problem worker. When the worker is back in-synch, the cluster (master) goes back to read/write mode.

Master with three worker nodes

Tip

Having a cluster with a master and at least three workers ensures that the cluster will remain fully functional in the event of any node failure.

When a master with enough history detects an out-of-syncworker, it replays the updates to the out-of-syncworker (synchronisation). Otherwise, it selects another (in-synch) worker at random and instructs it to replicate directly to the problem worker.

../_images/GraphDB_out-of-sync.png

During the replication, the master DOES NOT go in a read-only mode. It stays in a read-write mode and is fully functional.

../_images/GraphDB_replication2.png

Master peering

The WRITE requests are divided by the number of the available masters and sent to all of them. Each master keeps a log of the its own transactions and then they synchronise between each other. Now the masters are ready to apply the requests to the workers.

../_images/GraphDB_peers.png

Tip

Having a cluster with two peered masters ensures high-availability without any single point of failure.

Cluster backup

The master selects a worker with a full transaction log, switches it off and instructs it to create a backup. After finishing the backup, the switched off worker synchronises with the other workers.

../_images/GraphDB_backup.png

Tip

During the backup operation, the cluster remains in read-write mode and is able to process updates.