Managing a cluster

GraphDB cluster management is handled via Java Management Extensions Technology (JMX). Creating a cluster consists of several steps which depending on the topology are either setting/updating a set of JMX properties and are executing JMX operations e.g. adding a worker node to the Master. GraphDB provides a JMX-HTTP bridge for easier management of the JMX configuration using Jolokia request. Of course, you can use any other software for working with JMX e.g. jconsole, jvisualvm or third party clients e.g. jmxterm.

Note

To use the JMX interface, you need to configure the JMX endpoint.

Managing workers

Create a worker

To create workers, follow the procedure described in Create a worker.

Clone a worker

To clone a worker:

  1. Click a worker node to view its information.
  2. Click the Clone to another location button.
  3. In the dialog that opens, enter the ID, the title and choose the target location.

Add a worker to the cluster

Using the Workbench

  1. Go to Setup -> Cluster management.

    The cluster manager shows the nodes (masters and workers) and the connections between them visually. The workers are represented by a bee icon and the masters - by a person with a hat. The connections between the nodes are shown in different colours depending on the status of each worker.

    ../_images/cluster.png

  2. Drag and drop nodes to connect them.

  3. Click a node to get additional information or to clone it to another location.

  4. Check the status of the worker.

Tip

The cluster manager continually updates the visualisation so you can use it as a tool to monitor the status of the workers. In case you experience errors or issues, check if your attached remote locations are accessible and if the JMX settings for these locations are correct.

Using Jolokia

Link a worker to a master by executing the following request in a browser:

http://masterhost:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/masterID/addClusterNode/http:!/!/workerhost:<port>!/repositories!/workerID/0/true

Using CURL

curl -H 'content-type: application/json' \
  -d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/masterID\",\"operation\":\"addClusterNode\",\"arguments\":[\"http://workerhost:<port>/repositories/workerID\",0,true]}" \
  http://masterhost:<port>/jolokia/

Using JMX

After the instantiation of a master node, worker nodes can be added using a JMX client application, e.g., jconsole. From the MBeans tab, select the bean associated with the master node to be modified. Each bean will be named: ReplicationCluster/ClusterInfo/<repository_id>.

  1. Configure the JMX client application.
  2. Add worker nodes using the addClusterNode operation.
  3. Check the attribute NodeStatus to ensure that the worker node has been added correctly. This is an array and needs to be expanded by double-clicking it. The newly added worker node is shown here with its full URL. Its status should be [ON].

Notifications and statistics can be monitored through the JMX client.

Note

From GraphDB version 6 onwards all masters are always in a read-write mode. There is no additional step to put the master in RW mode, as in previous versions.

Remove a worker from the cluster

Using the Workbench

To remove a worker from a cluster, click the links connecting the worker to the other nodes.

Using Jolokia

Remove a worker from a cluster by executing the following request in a browser:

http://masterhost:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/masterID/removeClusterNode/http:!/!/workerhost:<port>!/repositories!/workerID

Using CURL

curl -H 'content-type: application/json' \
  -d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/masterID\",\"operation\":\"removeClusterNode\",\"arguments\":[\"http://workerhost:<port>/repositories/workerID\"]}" \
  http://masterhost:<port>/jolokia/

Using JMX

Worker nodes are removed using the removeClusterNode operation, which requires only the repository URL as a parameter.

Note

If a master node is assigned two worker nodes of unknown, but different, status, then it will not be able to decide which of the nodes is correct. In this situation, both workers should be removed, the master’s transaction log cleared, and the workers added back with the ‘correct’ worker first.

Reconfigure a worker

To reconfigure a worker, follow the procedure described in Configuring a repository.

Managing masters

Create a master

To create a master, follow the procedure described in Create a master.

Initialise a master

To initialize a master, follow the procedure described in Initialise a master.

Reconfigure a master

To reconfigure a master, follow the procedure described in Configuring a repository.

Add a new master to the cluster

Using the Workbench

To add a master as a sync peer to a second master , follow the same procedure as for adding a worker.

Warning

Master-to-Master connections are always created in both directions.

Using Jolokia

Add a master as a sync peer to a second master by executing the following request:

http://master2host:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/master2ID/addSyncPeer/master1ID/http:!/!/master1host:<port>!/repositories!/master1ID

Using CURL

curl -H 'content-type: application/json' \
  -d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/master1ID\",\"operation\":\"addSyncPeer\",\"arguments\":[\"master1ID\",\"http://master1host:<port>/repositories/master1ID\"]}" \
  http://master2host:<port>/jolokia/

Using JMX

Another option is to use the JMX Console. After the instantiation of a master node, other masters can be added via the MBeans tab. In order to do this, select the bean associated with the master node to be modified. Each bean will be named: ReplicationCluster/ClusterInfo/<repository_id>.

Master nodes can be added using the addSyncPeer operation, with the following parameters:

Using cluster.properties

If you want to setup several master nodes for the cluster, before bringing each master online, properly configure its config file cluster.properties:

  • location <master-repository>/cluster.properties

  • format

    version=2.0
node-id=master3

#own workers
node.1.url=http://node1:7200/repositories/worker
node.2.url=http://node2:7200/repositories/worker

#peers
peer.1.id=master1
peer.1.url=http://server1:7200/repositories/master
peer.2.id=master2
peer.2.url=http://server2:7200/repositories/master


#replication
min-time-to-consider-full-replication-s=600
net-speed-bits-per-sec=104857600

#misc
remote-master=false
slow-op-threshold-ms=0
system-transaction-password=
timeout=-1
update-timeout=-1

Remove a master from the cluster

Using the Workbench

To remove a master from a cluster click the links connecting it to the other nodes.

Using Jolokia

Remove a master as a sync peer to a second master by executing the request:

http://master2host:<port>/jolokia/exec/ReplicationCluster:name=ClusterInfo!/master2ID/removeSyncPeer/master1ID

Using CURL

curl -H 'content-type: application/json' \
  -d "{\"type\":\"exec\",\"mbean\":\"ReplicationCluster:name=ClusterInfo\/master1ID\",\"operation\":\"removeSyncPeer\",\"arguments\":[\"master1ID\"]}" \
  http://master2host:<port>/jolokia/

Using JMX

Master nodes are removed using the removeSyncPeer operation, which requires only the ID of the master.

Warning

Never forget that master-to-master connections are never unidirectional, so you have to remove the sync peer in both directions.

Checking the cluster status

  1. Start all cluster nodes.
  2. Go to http://localhost:7200. A local one if you have a workbench on the same server where the master node is (http://localhost:7200/repositories/master), or a remote location if the workbench is on another server (http://masterhost:7200/repositories/master).
  3. Copy the master repository ID.
  4. Start a JMX client, e.g., jconsole from the Java Software Development Kit.
  5. After connecting, select the MBeans tab and open the ReplicationCluster/ClusterInfo/<repository_id> bean (if it is not visible, copy and paste in a browser http://masterhost:<port>/repositories/<master-repository-id>/size or execute it using CURL).
  6. Check the attribute ClusterStatusSummary to ensure that the cluster is available. Its status should be [ON].

Secured connection

The communication between the cluster nodes can be performed over HTTPS. The configuration is straight-forward: #. Enable HTTPS on the hosts #. Make sure the Locations configured in Setup -> Repositories are configured to use HTTPS

In case the nodes’ certificates are self-signed you should add them to the other nodes’ JVM SSL TrustStores. Here you should keep in mind that: #. Both Masters and Workers need to trust the other nodes’ certificates. For example if Master1 and Worker1 are in a cluster you should have Worker1’s certificate in Master1’s TrustStore and vice versa. #. The Master should have its own certificate in its TrustStore. Otherwise you wouldn’t be able to use the Cluster Management from Workbench.