Managing a Cluster¶
What’s in this document?
GraphDB cluster management is handled via Java Management Extensions Technology (JMX). Creating a cluster consists of several steps, which, depending on the topology, are setting or updating a set of JMX properties, and are executing JMX operations, e.g., adding a worker node to the master. GraphDB provides a JMX-HTTP bridge for easier management of the JMX configuration using a Jolokia request. Of course, you can use any other software for working with JMX, e.g., JConsole, Java VisualVM, as well as third party clients. e.g., Jmxterm.
Note
To use the JMX interface, you need to configure the JMX endpoint.
Managing workers¶
Create a worker¶
To create workers, follow the procedure described in Create a worker.
Clone a worker¶
To clone a worker:
Click a worker node to view its information.
Click the Clone to another location button.
In the dialog that opens, enter the ID and the title, and choose the target location.
Add a worker to the cluster¶
Using the Workbench¶
Go to
.The cluster manager visualizes the nodes (masters and workers) and the connections between them. The workers are represented by a W bucket, and the masters - by an M bucket. The connections between the nodes are shown in different colors and patterns depending on the status of each worker. All possible statuses are listed in the legend on the bottom left.
Each master has a different color, and every worker connected to it takes its color. If a worker is connected to more than one master, it is constantly switching between their respective colors. When a worker has not been connected to any master(s) yet, its color is grey.
Drag and drop nodes to connect them. Note that you can connect workers to masters, and a master to another master, but not a worker to another worker. Masters always connect bi-directionally.
Click on a worker node to check its status or to clone it to another location as described above.
Click on a master node to check its status, or to control its read-only state and normal/mute mode. These configurations apply for both local and remote masters.
When connecting to workers and masters from a remote location, this is indicated by the location label below each node, as is the case here with r_master1 and r_worker1:
If you disconnect these remote nodes, in this case they will remain visible due to the fact that they were both connected to the local master, displayed the following way:
Tip
The cluster manager continually updates the visualization so you can use it as a tool for monitoring the status of the workers. In case you experience errors or issues, check if your attached remote locations are accessible, and if the JMX settings for these locations are correct.
Using Jolokia¶
Link a worker to a master by executing the following request in a browser:
http://masterhost:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/masterID/addClusterNode/http:!/!/workerhost:<port>!/repositories!/workerID/0/true
Using CURL¶
curl -H 'content-type: application/json' \
-d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/masterID\",\"operation\":\"addClusterNode\",\"arguments\":[\"http://workerhost:<port>/repositories/workerID\",0,true]}" \
http://masterhost:<port>/jolokia/
Using JMX¶
After the instantiation of a master node, worker nodes can be added
using a JMX client application, e.g., JConsole. From the MBeans tab,
select the bean associated with the master node to be modified. Each
bean will be named: ReplicationCluster/ClusterInfo/<repository_id>
.
Add worker nodes using the addClusterNode operation.
Check the attribute
NodeStatus
to ensure that the worker node has been added correctly. This is an array and needs to be expanded by double-clicking it. The newly added worker node is shown here with its full URL. Its status should beON
.
Notifications and statistics can be monitored through the JMX client.
Note
All masters are always in read-write mode. There is no additional step to put the master in RW mode, as in previous GraphDB versions.
Remove a worker from the cluster¶
Using the Workbench¶
To remove a worker from a cluster, click the links connecting the worker to the other nodes.
Using Jolokia¶
Remove a worker from a cluster by executing the following request in a browser:
http://masterhost:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/masterID/removeClusterNode/http:!/!/workerhost:<port>!/repositories!/workerID
Using CURL¶
curl -H 'content-type: application/json' \
-d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/masterID\",\"operation\":\"removeClusterNode\",\"arguments\":[\"http://workerhost:<port>/repositories/workerID\"]}" \
http://masterhost:<port>/jolokia/
Using JMX¶
Worker nodes are removed using the removeClusterNode
operation, which
requires only the repository URL as a parameter.
Note
If a master node is assigned two worker nodes of unknown but different status, then it will not be able to decide which of the two is the correct one. In this situation, both workers should be removed, the master’s transaction log cleared, and the workers added back with the correct worker first.
Reconfigure a worker¶
To reconfigure a worker, follow the procedure described in Configuring a repository.
Managing masters¶
Create a master¶
To create a master, follow described procedure.
Initialize a master¶
To initialize a master, follow the described procedure.
Reconfigure a master¶
To reconfigure a master, follow the described procedure.
Add a new master to the cluster¶
Using the Workbench¶
To add a master as a sync peer to a second master, follow the same procedure as for adding a worker.
Warning
Master-to-master connections are always created in both directions.
Using Jolokia¶
Add a master as a sync peer to a second master by executing the following request:
http://master2host:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/master2ID/addSyncPeer/master1ID/http:!/!/master1host:<port>!/repositories!/master1ID
Using CURL¶
curl -H 'content-type: application/json' \
-d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/master1ID\",\"operation\":\"addSyncPeer\",\"arguments\":[\"master1ID\",\"http://master1host:<port>/repositories/master1ID\"]}" \
http://master2host:<port>/jolokia/
Using JMX¶
Another option is to use the JMX Console.
After the instantiation of a master node, other masters can be added via the MBeans tab.
In order to do this, select the bean associated with the master node to be modified. Each
bean will be named: ReplicationCluster/ClusterInfo/<repository_id>
.
Master nodes can be added using the addSyncPeer
operation with the
following parameters:
id
: String - ID of the peer master, e.g., “master2”url
: String - A string holding the full URI of the RDF4J endpoint of the new node, e.g., http://192.0.2.1:7200/repositories/master
Using cluster.properties
¶
If you want to set up several master nodes for the cluster, properly configure the config file
cluster.properties
of each master before bringing it online:
location
<master-repository>/cluster.properties
format
version=2.0 node-id=master3 #own workers node.1.url=http://node1:7200/repositories/worker node.2.url=http://node2:7200/repositories/worker #peers peer.1.id=master1 peer.1.url=http://server1:7200/repositories/master peer.2.id=master2 peer.2.url=http://server2:7200/repositories/master #replication min-time-to-consider-full-replication-s=600 net-speed-bits-per-sec=104857600 #misc remote-master=false slow-op-threshold-ms=0 system-transaction-password= timeout=-1 update-timeout=-1
Remove a master from the cluster¶
Using the Workbench¶
To remove a master from a cluster, click the links connecting it to the other nodes.
Using Jolokia¶
Remove a master as a sync peer to a second master by executing the request:
http://master2host:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/master2ID/removeSyncPeer/master1ID
Using CURL¶
curl -H 'content-type: application/json' \
-d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/master1ID\",\"operation\":\"removeSyncPeer\",\"arguments\":[\"master1ID\"]}" \
http://master2host:<port>/jolokia/
Using JMX¶
Master nodes are removed using the removeSyncPeer
operation, which
requires only the ID of the master.
Warning
Keep in mind that master-to-master connections are never unidirectional, so you have to remove the sync peer in both directions.
Checking the cluster status¶
Start all cluster nodes.
Go to http://localhost:7200 - local if you have a Workbench on the same server where the master node is (http://localhost:7200/repositories/master), or a remote location if the Workbench is on another server (http://masterhost:7200/repositories/master).
Copy the master repository ID.
Start a JMX client, e.g., JConsole, from the Java Software Development Kit.
After connecting, select the MBeans tab and open the
ReplicationCluster/ClusterInfo/<repository_id>
bean (if it is not visible, copy and paste in a browserhttp://masterhost:<port>/repositories/<master-repository-id>/size
, or execute it using curl).Check the
ClusterStatusSummary
attribute to ensure that the cluster is available. Its status should beON
.
Secured connection¶
The communication between the cluster nodes can be performed over HTTPS. The configuration is straightforward:
Enable HTTPS on the hosts.
Make sure the Locations configured in
are configured to use HTTPS.
In case the nodes’ certificates are self-signed, you need to add them to the other nodes’ JVM SSL TrustStores. Here, you should keep in mind that:
Both masters and workers need to trust the other nodes’ certificates. For example, if master1 and worker1 are in a cluster, you should have worker1’s certificate in master1’s TrustStore, and vice versa.
The master should have its own certificate in its TrustStore, otherwise you would not be able to use the Cluster Management from the Workbench.