Migrating GraphDB Configurations — GraphDB EE 9.11.0 documentation

Compatibility between the versions of GraphDB, Connectors, and third party connectors ¶

GraphDB and RDF4J		Connectors: Elasticsearch, Lucene, Solr, Kafka					Steps for migrating from an older version
GDB	RDF4J	Conn.	ES	Luc.	Solr	Kafka	Steps for migrating from an older version
9.11.x	3.7.6	15.2.x	7.16.3	8.11.1	8.11.1	2.8.0	Introduced Sequences plugin as part of the fingerprint. In a cluster, the plugin will be automatically disabled to protect cluster integrity. You need to enable the Sequences plugin manually after upgrade. Recreate all connectors with the repair option.
9.10.x	3.7.3	15.1.1	7.13.2	8.9.0	8.9.0	2.8.0	Introduced SPARQL Template as part of the fingerprint. In a cluster, the plugin will be automatically disabled to protect cluster integrity. You need to enable the SPARQL Template manually after upgrade.
9.9.x	3.7.1	15.1.0	7.13.2	8.9.0	8.9.0	2.8.0	Removed the Lucene FTS plugin from distribution and fingerprint. Disable the Lucene FTS plugin manually before upgrade in order to protect cluster integrity.
9.8.x	3.6.3	15.0.0	7.9.2	8.6.3	8.6.3	2.7.0	Introduced Kafka connector as part of the fingerprint. In a cluster, the plugin will be automatically disabled to protect cluster integrity. You need to enable the Kafka connector plugin manually after upgrade.
9.7.x	3.6.1	14.0.0	7.9.2	8.6.3	8.6.3		No special attention needed.
9.6.x	3.5.1	13.0.1	7.9.2	8.6.3	8.6.3		No special attention needed.
9.5.x	3.4.4	13.0.0	7.9.2	8.6.3	8.6.3		Introduced Entity change plugin (used by the Ontotext Platform) as part of the fingerprint. In a cluster, the plugin will be automatically disabled to protect cluster integrity. You need to enable the Entity change plugin manually after upgrade. The following authentication parameters have been deprecated, but still work: `graphdb.auth.module`, `graphdb.auth.kerberos.enabled`
9.4.x	3.3.1	12.1.x	7.7.0	8.5.1	8.5.1		No special attention needed.
9.3.x	3.2.0	12.1.x	7.7.0	8.5.1	8.5.1		Recreate all connectors with the repair option.
9.2.x	3.2.0 -M1	12.0.x	7.3.1	8.2.0	8.2.0		Language tags in literals are now returned according to the recommended case normalization specified by BCP47, Section 2.1.1. For comparing language literals we recommend the SPARQL 1.1 langMatches function instead of comparing language tags directly (e.g. `FILTER(lang(?var) = "en-gb")` would fail because `lang(?var)` will return the case-normalized tag as `en-GB`). Some aggregate queries may return different results due to a fix in RDF4J.
9.1.x	3.0.1	12.0.1	7.3.1	8.2.0	8.2.0		Introduced History plugin as part of the fingerprint. In a cluster, this will lead to out-of-sync for the workers. Stop the master and rename the `txLog` for backup purposes. Start the master again.
9.0.x	2.5.2	12.0.0	7.3.1	8.2.0	8.2.0		Recreate all connectors with the repair option.
8.11.x	2.5.2	11.0.0	6.6.x	7.7.x	7.7.x		No special attention needed.
8.10.x	2.5.2	11.0.0	6.6.x	7.7.x	7.7.x		No special attention needed.
8.9.x	2.4.6	10.1.0	6.6.x	7.7.x	7.7.x		Removal of SPARQL-MM from the fingerprint. In a cluster, this will lead to out-of-sync for the workers. Stop the master and rename the `txLog` for backup purposes. Start the master again.
8.8.x	2.4.2	10.0.0	6.3.x	7.4.x	7.4.x		No special attention needed.
8.7.x	2.3.2	9.0.0	6.3.x	7.4.x	7.4.x		Recreate all the connectors with the repair option. Need to rebuild Semantic similarity search indexes.
8.6.x	2.3.2	8.0.0	6.2.x	7.2.x	7.2.x		Need to rebuild GeoSPARQL index.
8.5.x	2.2.4	7.2.0	5.3.x	6.5.x	6.5.x		No more system repository in future installations. During the upgrade procedure, system repository will be removed while the data insight will be backed up. There will be a `config.ttl` file in each repository directory with config data. Important points: Cannot query System repo. More strict parser - IRI validation according to RFC3987.
8.4.x	2.2.2	7.2.0	5.3.x	6.5.x	6.5.x		Autocomplete is no longer part of the fingerprint. In case of cluster, this will lead to out-of-sync for the workers. Stop the master and rename the `txlog` for backup purposes. Start the master again.
8.3.x	2.2.2	7.1.0	5.3.x	6.5.x	6.5.x		No special attention needed.
8.2.x	2.2.1	7.0.0	5.3.x	6.5.x	6.5.x		No special attention needed.
8.1.x	2.1.6	6.0.2	2.4.0	6.2.1	6.2.1		Recreate all the connectors with the repair option. Please note the breaking changes for each connector: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking-changes-5.0.html https://lucene.apache.org/solr/guide/6_6/upgrading-solr.html#upgrading-solr
8.0.x	2.0.3	6.0.2	2.4.0	6.2.1	6.2.1		No special attention needed. Important points: The engine replaces PCSO and PCOS indexes with CPSO. Remove the `owlim-license` parameter from the repository TTL configuration.
For older versions, please ask for support.

Migrating a repository ¶

If you want to migrate your GraphDB configurations and replicate the setup, there are three steps that you need to follow:

Back up your repository by copying the binary image (explained in more detail here), or restore it from an RDF export or from a binary image or zip backup (explained in more detail here).

For steps 2 and 3, it is useful to know that the GraphDB distribution package consists of several folders that are described in detail here.

Copy the conf directory to the new instance. It contains the logback and the GraphDB configuration.
Copy the work directory to the new instance. It contains all Workbench-related details, e.g., saved queries, users, user roles, etc.

When upgrading to a newer GraphDB version, it might contain plugins that are not present in the older version. In this case, the PluginManager disables the newly detected plugins, so you need to enable them by executing the following SPARQL query:

insert data {
    [] <http://www.ontotext.com/owlim/system#startplugin> "plugin-name"
}

Then create your plugin following the steps described in the corresponding documentation, and also make sure to not delete the database in the plugin you are using.

You can also stop a plugin before the migration in case you deem it necessary:

insert data {
    [] <http://www.ontotext.com/owlim/system#stopplugin> "plugin_name"
}

These steps apply for both cluster and single repository configurations.

If you want to migrate a cluster configuration from GraphDB versions 8.5 and older to 9.x, please follow the steps outlined below. The described procedures refer to the three recommended cluster topologies:

Single master with three or more workers (more details about the configuration here)

Two masters sharing workers, one of the masters is read-only (more details about the configuration here)

Multiple masters with dedicated workers (more details about the configuration here)

Note

A change in the fingerprint has now been implemented, as the SPARQL-MM plugin has been removed from it. This will not affect your migration, as long as the transaction log is removed.

Note

The Jolokia secret has been replaced by a general security mechanism. When security is enabled, roles and permissions are implemented.

Single master with three or more workers¶

Stop all running nodes.

Copy the data directory of the master to the newer version, clearing its txLog.

Copy the data directory of the worker nodes to the newer version.

Start the workers.

Start the master.

Multiple masters with dedicated workers¶

One master is RW (primary master), and the other one is muted (secondary master).

Make the secondary master RW.

Redirect the load balancer to the secondary master. (Note: You can also use the GraphDB Client Failover Utility. This option requires no action on your part, as it is designed to provide support in a failover scenario where there is no access to the primary master node of the GraphDB replication cluster.)

Stop the primary master and its workers.

Copy the data directory of the primary master to the newer version, clearing its txLog.

Copy the data directory of the primary master’s workers to the newer version.

Start these workers.

Start the primary master and wait until the cluster is operational.

Redirect the load balancer to the primary master.

Mute the secondary master.

Stop the secondary master and its workers.

Copy the data directory of the secondary master to the newer version, clearing its txLog.

Copy the data directory of the secondary master’s workers to the newer version.

Start these workers.

Start the secondary master.

Migrating connectors and plugins ¶

Lucene: All you need to do is recreate the connector. If new properties are introduced, they will have default values.

Elasticsearch (recommended steps): Drop the connector from the Workbench, make changes if needed, and recreate it. This ensures better control of what has been created, as otherwise GraphDB would take the config and update the properties with default values for those that are unknown.

for version 8.6:
- Option elasticserachNode changes from hostname:port (where port is the transport protocol port, default is 9300) to http://hostname:port (where port is the REST port, default is 9200). Https is allowed as well.
- Option elasticsearchCluster is removed as the REST client does not need the cluster.
- Option elasticsearchClientOptions is removed as the REST client does not have options.
- New elasticsearchBasicAuthUser option is introduced to supply the authentication user.
- New elasticsearchBasicAuthPassword option is introduced to supply the authentication password.
for version 8.7:
- Introduction of type hints when requesting aggregation. The type hint is part of the aggregation name, and consists of a dollar sign + one of the words “iri” (or “uri”), “int”, “long”, “double”, “float”, “date”. For example, a simple aggregation name could be “my_agg” and a type-hinted one “my_agg$long”. The hints are used by the connector to return IRIs or literals with the proper data type. Apart from fetching the value of the key with or without a type hint (via predicate :key), you need to introduce an alternative :keyAsString predicate that always returns an untyped literal and corresponds to the Elasticsearch API’s getKeyAsString().
- Introduction of new option bulkUpdateRequestSize (default value 5 megabytes). It sends updates to Elasticsearch using the bulk API, combining multiple small requests into a single large one.
for version 8.8:
- Introduction of automatic detection of fields. This mode reintroduces / as a valid character in field names. In versions prior to 8.8, GraphDB used / to separate variant chains of the same property, e.g., fields “field/1” and “field/2” were merged into a single field “field”. This functionality now uses $ as a separator (e.g., “field$1”, “field$2”).

Solr (recommended steps): Drop the connector from the Workbench, make changes if needed, and recreate it. This ensures better control of what has been created, as otherwise GraphDB would take the config and update the properties with default values for those that are unknown.

Autocomplete: The index will be disabled to keep the cluster healthy, so you will need to enable it in the Workbench.

RDF Rank: You need to compute the rank as it will be outdated. This can be done through the Workbench interface.

Migrating GraphDB Configurations¶

Compatibility between the versions of GraphDB, Connectors, and third party connectors ¶

Migrating a repository ¶

Migrating a cluster ¶

Single master with three or more workers¶

Multiple masters with dedicated workers¶

Migrating connectors and plugins ¶

Migrating GraphDB Configurations¶

Compatibility between the versions of GraphDB, Connectors, and third party connectors¶

Migrating a repository¶

Migrating a cluster¶

Single master with three or more workers¶

Two masters sharing workers, one of the masters is read-only¶

Multiple masters with dedicated workers¶

Migrating connectors and plugins¶

Compatibility between the versions of GraphDB, Connectors, and third party connectors ¶

Migrating a repository ¶

Migrating a cluster ¶

Migrating connectors and plugins ¶