Migrating GraphDB Configurations

To migrate from one GraphDB version to another, follow the instructions in the last column of the table below, and then the steps described further down in this page.

Compatibility between the versions of GraphDB, Connectors, and third party connectors

GraphDB and RDF4J

Connectors: Elasticsearch, Lucene, Solr

Steps for migrating from an older version

GDB

RDF4J

Con

ES

Luc

Solr

9.6.x

3.5.1

13.0.1

7.9.2

8.6.3

8.6.3

No special attention needed.

9.5.x

3.4.4

13.0.0

7.9.2

8.6.3

8.6.3

Introduced Entity change plugin (used by the Ontotext Platform) as part of the fingerprint. In a cluster, this will lead to out of sync for the workers. Stop the master and rename the txLog for backup purposes. Start the master again.

The following authentication parameters have been deprecated, but still work: graphdb.auth.module, graphdb.auth.kerberos.enabled

9.4.x

3.3.1

12.1.x

7.7.0

8.5.1

8.5.1

No special attention needed.

9.3.x

3.2.0

12.1.x

7.7.0

8.5.1

8.5.1

Recreate all connectors with the repair option.

9.2.x

3.2.0 -M1

12.0.x

7.3.1

8.2.0

8.2.0

Language tags in literals are now returned according to the recommended case normalization specified by BCP47, Section 2.1.1. For comparing language literals we recommend the SPARQL 1.1 langMatches function instead of comparing language tags directly (e.g. FILTER(lang(?var) = "en-gb") would fail because lang(?var) will return the case-normalized tag as en-GB).

Some aggregate queries may return different results due to a fix in RDF4J.

9.1.x

3.0.1

12.0.1

7.3.1

8.2.0

8.2.0

Introduced History plugin as part of the fingerprint. In a cluster, this will lead to out of sync for the workers. Stop the master and rename the txLog for backup purposes. Start the master again.

9.0.x

2.5.2

12.0.0

7.3.1

8.2.0

8.2.0

Recreate all connectors with the repair option.

8.11.x

2.5.2

11.0.0

6.6.x

7.7.x

7.7.x

No special attention needed.

8.10.x

2.5.2

11.0.0

6.6.x

7.7.x

7.7.x

No special attention needed.

8.9.x

2.4.6

10.1.0

6.6.x

7.7.x

7.7.x

Removal of SPARQL-MM from the fingerprint. In a cluster, this will lead to out of sync for the workers. Stop the master and rename the txLog for backup purposes. Start the master again.

8.8.x

2.4.2

10.0.0

6.3.x

7.4.x

7.4.x

No special attention needed.

8.7.x

2.3.2

9.0.0

6.3.x

7.4.x

7.4.x

Recreate all the connectors with the repair option. Need to rebuild Semantic similarity search indexes.

8.6.x

2.3.2

8.0.0

6.2.x

7.2.x

7.2.x

Need to rebuild GeoSPARQL index.

8.5.x

2.2.4

7.2.0

5.3.x

6.5.x

6.5.x

No more system repository in future installations. During the upgrade procedure, system repository will be removed while the data insight will be backed up. There will be config.ttl file in each repository directory with config data. Important points: 1. Cannot query System repo. 2. More strict parser - IRI validation according to RFC3987

8.4.x

2.2.2

7.2.0

5.3.x

6.5.x

6.5.x

Autocomplete is no longer part of the fingerprint. In case of cluster, this will lead to out of sync for the workers. Stop the master and rename the txlog for backup purposes. Start the master again.

8.3.x

2.2.2

7.1.0

5.3.x

6.5.x

6.5.x

No special attention needed.

8.2.x

2.2.1

7.0.0

5.3.x

6.5.x

6.5.x

No special attention needed.

8.1.x

2.1.6

6.0.2

2.4.0

6.2.1

6.2.1

Recreate all the connectors with the repair option. Please note the breaking changes for each connector:

8.0.x

2.0.3

6.0.2

2.4.0

6.2.1

6.2.1

No special attention needed.

Important points:

  1. The engine replaces PCSO and PCOS indexes with CPSO.

  2. Remove the owlim-license parameter from the repository TTL configuration.

For older versions, please ask for support.

Migrating a repository

If you want to migrate your GraphDB configurations and replicate the setup, there are three steps that you need to follow:

  1. Back up your repository by copying the binary image (explained in more detail here), or restore it from an RDF export or from a binary image or zip backup (explained in more detail here).

For steps 2 and 3, it is useful to know that the GraphDB distribution package consists of several folders that are described in detail here.

  1. Copy the conf directory to the new instance. It contains the logback and the GraphDB configuration.

  2. Copy the work directory to the new instance. It contains all Workbench-related details, e.g., saved queries, users, user roles, etc.

Migrating a cluster

Hint

When upgrading to a newer GraphDB version, it might contain plugins that are not present in the older version. In this case, the PluginManager disables the newly detected plugins, so you need to enable them by executing the following SPARQL query:

insert data {
    [] <http://www.ontotext.com/owlim/system#startplugin> "plugin-name"
}

Then create your plugin following the steps described in the corresponding documentation, and also make sure to not delete the database in the plugin you are using.

You can also stop a plugin before the migration in case you deem it necessary:

insert data {
    [] <http://www.ontotext.com/owlim/system#stopplugin> "plugin_name"
}

These steps apply for both cluster and single repository configurations.

If you want to migrate a cluster configuration from GraphDB versions 8.5 and older to 9.x, please follow the steps outlined below. The described procedures refer to the three recommended cluster topologies:

  • Single master with three or more workers (more details about the configuration here)

  • Two masters sharing workers, one of the masters is read-only (more details about the configuration here)

  • Multiple masters with dedicated workers (more details about the configuration here)

Note

A change in the fingerprint has now been implemented, as the SPARQL-MM plugin has been removed from it. This will not affect your migration, as long as the transaction log is removed.

Note

The Jolokia secret has been replaced by a general security mechanism. When security is enabled, roles and permissions are implemented.

Single master with three or more workers

  1. Stop all running nodes.

  2. Copy the data directory of the master to the newer version, clearing its txLog.

  3. Copy the data directory of the worker nodes to the newer version.

  4. Start the workers.

  5. Start the master.

Two masters sharing workers, one of the masters is read-only

  1. One master is RW (primary master), and the other one is RO (secondary master).

  2. Make the secondary master RW.

  3. Redirect the load balancer to the secondary master. (Note: You can also use the GraphDB Client Failover Utility. This option requires no action on your part, as it is designed to provide support in a failover scenario where there is no access to the primary master node of the GraphDB replication cluster.)

  4. Disconnect worker 1 from the primary master.

  5. Disconnect worker 2 and worker 3 from secondary master.

  6. Stop the primary master, worker 2, and worker 3.

  7. Copy the data directory of the primary master to the newer version, clearing its txLog.

  8. Copy the data directory of worker 2 and worker 3 to the newer version.

  9. Start worker 2 and worker 3.

  10. Verify that these workers are operational, and start the primary master of the newer version.

  11. Verify that you have an operational cluster consisting of RW master with worker 2 and 3.

  12. Redirect the load balancer to the RW master. Now you have a fully operational newer cluster consisting of the RW master with two nodes and the same dataset.

  13. Disconnect worker 1 from the RO master and stop both.

  14. Copy the data directory of the RO master, clearing its txLog as well, and the data directory of the version 8.4 worker 1 to the newer version.

  15. Start worker 1.

  16. Verify that worker 1 is operational, and start the RO master.

  17. Connect the RW master to worker 1, and RO master to all workers into a cluster.

Multiple masters with dedicated workers

  1. One master is RW (primary master), and the other one is muted (secondary master).

  2. Make the secondary master RW.

  3. Redirect the load balancer to the secondary master. (Note: You can also use the GraphDB Client Failover Utility. This option requires no action on your part, as it is designed to provide support in a failover scenario where there is no access to the primary master node of the GraphDB replication cluster.)

  4. Stop the primary master and its workers.

  5. Copy the data directory of the primary master to the newer version, clearing its txLog.

  6. Copy the data directory of the primary master’s workers to the newer version.

  7. Start these workers.

  8. Start the primary master and wait until the cluster is operational.

  9. Redirect the load balancer to the primary master.

  10. Mute the secondary master.

  11. Stop the secondary master and its workers.

  12. Copy the data directory of the secondary master to the newer version, clearing its txLog.

  13. Copy the data directory of the secondary master’s workers to the newer version.

  14. Start these workers.

  15. Start the secondary master.

Migrating connectors and plugins

Lucene: All you need to do is recreate the connector. If new properties are introduced, they will have default values.

Elasticsearch (recommended steps): Drop the connector from the Workbench, make changes if needed, and recreate it. This ensures better control of what has been created, as otherwise GraphDB would take the config and update the properties with default values for those that are unknown.

  • for version 8.6:

    • Option elasticserachNode changes from hostname:port (where port is the transport protocol port, default is 9300) to http://hostname:port (where port is the REST port, default is 9200). Https is allowed as well.

    • Option elasticsearchCluster is removed as the REST client does not need the cluster.

    • Option elasticsearchClientOptions is removed as the REST client does not have options.

    • New elasticsearchBasicAuthUser option is introduced to supply the authentication user.

    • New elasticsearchBasicAuthPassword option is introduced to supply the authentication password.

  • for version 8.7:

    • Introduction of type hints when requesting aggregation. The type hint is part of the aggregation name, and consists of a dollar sign + one of the words “iri” (or “uri”), “int”, “long”, “double”, “float”, “date”. For example, a simple aggregation name could be “my_agg” and a type-hinted one “my_agg$long”. The hints are used by the connector to return IRIs or literals with the proper data type. Apart from fetching the value of the key with or without a type hint (via predicate :key), you need to introduce an alternative :keyAsString predicate that always returns an untyped literal and corresponds to the Elasticsearch API’s getKeyAsString().

    • Introduction of new option bulkUpdateRequestSize (default value 5 megabytes). It sends updates to Elasticsearch using the bulk API, combining multiple small requests into a single large one.

  • for version 8.8:

    • Introduction of automatic detection of fields. This mode reintroduces / as a valid character in field names. In versions prior to 8.8, GraphDB used / to separate variant chains of the same property, e.g., fields “field/1” and “field/2” were merged into a single field “field”. This functionality now uses $ as a separator (e.g., “field$1”, “field$2”).

Solr (recommended steps): Drop the connector from the Workbench, make changes if needed, and recreate it. This ensures better control of what has been created, as otherwise GraphDB would take the config and update the properties with default values for those that are unknown.

Autocomplete: The index will be disabled to keep the cluster healthy, so you will need to enable it in the Workbench.

RDF Rank: You need to compute the rank as it will be outdated. This can be done through the Workbench interface.