Client Failover Utility

Introduction

The Client Failover Utility (a.k.a. ClientAPI) is designed to provide support for a failover scenario, in which there is no access to the primary master node of the GraphDB replication cluster, such as connectivity issues. In this case, Java exceptions are thrown, the failed master is flagged, and the Client Failover Utility retries accessing the next available master node.

At regular intervals (currently 15 seconds), the repository instance checks each of the failed master nodes. When it finds a reachable and active master, it returns it for subsequent use.

Implementation

The Client Utility implements the RDF4J org.eclipse.rdf4j.repository.Repository interface and is instantiated by a configuration file, which describes the available replication cluster master nodes.

Procedure

Creating a configuration object

There are two configuration modes for the Client API. One is through a configuration file, which can be scanned for changes, and the other is to configure the masters through Java calls:

  • To configure the Client through a configuration file, create the configuration object with the following:

final ClientConfig config = ClientConfig
    .builderFromConfig(new File("Path to file"), intervalToCheckForChanges)
    // set other options here as needed
    .build();
  • To configure the Client in Java, create a configuration object with the following:

final ClientConfig config = ClientConfig
    .builder()
    .addMaster("http://localhost:7200/repositories/master")
    .addMaster("http://localhost:7200/repositories/secondMaster")
    // set other options here as needed
    .build();

Obtaining an instance

To get an instance of the ClientHTTPRepository, construct it with the configuration object described above:

ClientHTTPRepository repository = tnew ClientHTTPRepository(config);
// initialize once before getting connections
repository.initialize();

Support for custom HTTP headers

Each instance of ClientHTTPRepository implements an additional interface, which provides the possibility for adding per-request additional HTTP headers:

package com.ontotext.repository.http;

import java.util.Map;

public interface AccessClientAPI {
    public void addRequestThreadLocalHttpParam(String param, String value);

    public Map<String, String> getRequestThreadLocalHttpParamMap();
}

Note

They are stored in a map wrapped as ThreadLocal. When they are no longer needed, the map has to be removed.

Control of query handling and retry policy behavior

The configuration object supports many parameters that can be set through a Java Runtime property with -D, or by using the methods of the configuration builder:

  • To control the retry policy behavior when processing HTTP 503 responses and server side HTTP 4xx, set the parameters config.dontRetryOnHTTP503() and .dontRetryOnHTTP4xx():

    config.dontRetryOnHTTP503()
           .dontRetryOnHTTP4xx()
    

    By default, both of them are set to true (retrying on the 4xx and 503 response codes from the server) and can be changed through the Runtime Java properties retry-on-503 and retry-on-4xx.

  • To control the HTTP Client Builder that is used internally by the Client, use the code below:

    config.setHttpClientBuilder(/* create a builder here*/)
    

    For example, to control the socket timeout for requests that are made by the Client failover, provide the proper HTTP Client Builder.

    config.setHttpClientBuilder(
        HttpClientBuilder.create().setDefaultRequestConfig(
            RequestConfig.custom().setSocketTimeout(10_000).build()
        )
    )
    
  • To acquire a connection from the repository instance that implements RDF4J’s org.eclipse.rdf4j.repository.RepositoryConnection, invoke the Repository getConnection() method. All operations are available through this connection.

  • The connection instance also implements another interface, which can be used to alter the behavior of the commit(), so it waits for any delayed updates (indicated by the HTTP202 result code) to finish before returning:

    package com.ontotext.repository.http;
    
    public interface ClientConnectionExtentions {
        public boolean isCommitSynchronizedOnHTTP202();
    
        public void setCommitSynchronizedOnHTTP202(boolean mode);
    }
    

Configuration

The configuration file is a text file that describes the available master nodes by their URL location and, optionally, user and password credentials, if some security policy is applied to the SPARQL endpoints.

The URL has to be preceded by the case insensitive keyword uri or url followed by the = sign, and the rest of the line is trimmed of white space and used as url. A basic url scheme check is used to make sure the url content is a valid URL.

Note

The user and password should follow the uri description line. They are denoted by ‘user’ for the username and ‘pass’ for the password. The symbol # in the beginning of a line and the double slash // are interpreted as comments, and are not processed.

The following is an example of a Client configuration:

# Example client.config file
# it is used to check the initialization of the ClientAPI component, which is
# an instance of ClientHTTPRepository so that it works with a list of master nodes described here

# first master
uri= server1:7200/repositories/master
user=
pass=

#second master
uri=server2:7200/repositories/master
user=
pass=

Examples

The following are examples of how to instantiate a Client Utility and execute various operations such as evaluating query, SPARQL updates, add or remove data, etc.

Procedure

  1. Instantiate the Client Utility.

    The Client Utility Java class is com.ontotext.repository.http.ClientHTTPRepository.

  2. Initialize the Client Utility before acquiring any connection.

  3. Invoke the shutdown() method to clean up all local resources when the Client Utility is no longer necessary.

Query evaluation

The following code snippet shows how to evaluate a SPARQL query. The Constructor takes a single argument, which is the name of the configuration file.

In addition, you may also set the location folder where the configuration file uses the repository’s setDataDir() method.

The same location is used to temporary store any submitted updates that are still not confirmed as completed.

import java.io.File;

import org.eclipse.rdf4j.query.BooleanQuery;
import org.eclipse.rdf4j.query.MalformedQueryException;
import org.eclipse.rdf4j.query.QueryEvaluationException;
import org.eclipse.rdf4j.query.QueryLanguage;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.RepositoryException;

import com.ontotext.repository.http.ClientConfig;
import com.ontotext.repository.http.ClientHTTPRepository;

public class TestQueryEvaluation {
    public static void main(String[] args) {
        ClientHTTPRepository rep = new ClientHTTPRepository(new ClientConfig.builderFromConfig(new File("client.config"), 0).build());
        try {
            rep.initialize();
            RepositoryConnection conn = rep.getConnection();
            try {
                BooleanQuery q = conn.prepareBooleanQuery(QueryLanguage.SPARQL,
                                        "ask {?s a rdfs:Class .}");

                boolean result = q.evaluate();
                System.out.println("reult:"+result);
            } catch (MalformedQueryException e) {
                e.printStackTrace();
            } catch (QueryEvaluationException e) {
                e.printStackTrace();
            } finally {
                conn.close();
            }
        } catch (RepositoryException e) {
            e.printStackTrace();
        } finally {
            try {
                rep.shutDown();
            } catch (RepositoryException e) {
            }
        }
    }
}

Executing a SPARQL update

The initialization code is the same, and the way to execute the update follows the RDF4J API.

RepositoryConnection conn = rep.getConnection();
try {
    Update q = conn.prepareUpdate(QueryLanguage.SPARQL, "delete data {<urn:1> a <urn:class>.}");

    q.execute();
} catch (MalformedQueryException e) {
    e.printStackTrace();
} catch (UpdateExecutionException e) {
    e.printStackTrace();
} finally {
    conn.close();
}

Grouping updates into a single transaction

Here again, the initialization is the same, and the key points are to invoke begin() and commit() around the sequence of update operations.

RepositoryConnection conn = rep.getConnection();
// create resource for the operation
URI resource = rep.getValueFactory().createURI("urn:1");
try {
    // initiate transaction
    conn.begin();
    try {
        // first operation is to remove some statements with subject 'urn:1' from a context 'urn:1'
        conn.remove(resource, null, null, resource);
        // second operation is a sparql update
        Update q = conn.prepareUpdate(QueryLanguage.SPARQL, "delete data {<urn:1> a <urn:class>.}");
        q.execute();
    } finally {
        // commit both as a single transaction
        conn.commit();
    }
} catch (MalformedQueryException e) {
    e.printStackTrace();
} catch (UpdateExecutionException e) {
    e.printStackTrace();
} finally {
    conn.close();
}

Additional notes and dependencies

Logging is based on slf4j and loggers.