Graph replacement optimization¶

GraphDB offers an optimized graph replacement algorithm which makes graph updates faster in situations where the new graph partially overlaps with data in the old one.

Clearing an old graph and then importing the new information can be inefficient. If the two operations are handled separately, it is impossible to determine whether a statement will also be present in the new graph and thus preserve it instead of deleting it. The same applies for preserving connectors or inferring statements.

As long as the replacement is done in a single transaction, the graph replacement optimization will be applied to avoid such inefficiencies. The property graphdb.replace.graph.batch.size controls the batch size of the buffer, and has a default value of 10000 statements.

The graph replacement optimization algorithm implements the following logic:

Check transaction contents. If the transaction includes a graph replacement, proceed.
Check if any of the graphs to be replaced are valid and if any of them have data. If so, store their identifiers in a list.
While processing transaction statements for insertion, if their context (graph) matches an identifier from the list, store them inside a tracker.
While clearing the graph to be replaced, if it is not mentioned in the tracker, directly delete all its contents.
If a graph is mentioned in the tracker, iterate over its triples.
Triples in the replacement graph that are also in the tracker are preserved. Otherwise, they are deleted.

Deletions may trigger re-inference and are a more costly process than the check described in the algorithm. Therefore, in some test cases due to the optimization users can observe a speedup of up to 200%.

Here is an example of an update that will use the replacement optimization algorithm:

curl -X PUT -H "Content-Type: application/x-trig" --data-binary '@test_modified.trig'\
    'http://localhost:7200/repositories/test/rdf-graphs/service?graph=http://example.org/optimizations/replacement'

By contrast, the following approach will not use the optimization since it performs the replacement in two separate steps:

curl -X POST -H 'Content-Type: application/sparql-update'\
    --data-binary 'CLEAR GRAPH <http://example.org/optimizations/replacement>'\
    'http://localhost:7200/repositories/test/statements'

curl -X POST -H "Content-Type: application/x-trig" --data-binary '@test_modified.trig'\
    'http://localhost:7200/repositories/test/statements'

Note

The replacement optimization described here applies to all forms of transactions. It will be triggered by standard PUT requests, such as the ones in the example, but also by SPARQL INSERT queries containing the http://www.ontotext.com/replaceGraph predicate, such as <http://any/subject> <http://wwww.ontotext.com/replaceGraph> <http://example.org/graph>