Vector search in nested fields with Elasticsearch¶
What’s in this document?
Semantic vector search over RDF data enables retrieval based on conceptual similarity rather than exact lexical matching. By integrating GraphDB with Elasticsearch (as well as OpenSearch) through the connector framework, RDF literals can be automatically transformed into vector embeddings while maintaining their associations to the graph, which allows for high-dimensional search within the context of their structural relationships. This allows SPARQL queries to delegate semantic similarity search to Elasticsearch while still returning RDF resources as results.
A critical modeling decision in such configurations is how multivalued and multilingual properties are indexed. In the Star Wars dataset, both film titles (rdfs:label) and descriptions (https://swapi.co/vocabulary/desc) are provided in multiple languages. A single film may therefore have several literal values for the same property, each tagged with a different language (for example, English, German, or Finnish).
If these values were indexed without using datatype: native:nested, they would be treated as simple multivalued fields. In that scenario, GraphDB would concatenate the literal values into a single logical field during indexing, and embeddings would be generated over the aggregated content rather than per individual literal, eliminating the one-to-one association between a specific language literal and its corresponding embedding vector. Similarity search would then operate over merged multilingual text, reducing precision and making it impossible to distinguish which language-specific value contributed to the similarity score.
By defining the fields as native:nested, each literal is stored as a separate nested object which Elasticsearch maintains as independent subdocuments. This guarantees that every language-specific literal is indexed independently and remains structurally bound to its own embedding.
The GraphDB Elasticsearch documentation describes the Prerequisites for vector search using the connector, as well as how to perform Vector search queries. However, if you want to combine vector search with Nested objects, creating a connector that supports both can prove to be more involved than usual. This tutorial covers this more complicated scenario by demonstrating how the GraphDB Elasticsearch connector can be used to enable semantic vector search with nested objects over the Star Wars RDF dataset, specifically films defined in the SWAPI vocabulary.
Tip
Although this example focuses on Elasticsearch, OpenSearch also supports vector search, and the example provided here can prove useful when creating an OpenSearch connector with both vector search and nested fields.
Prerequisites for vector search in nested fields¶
This example uses the Star Wars RDF dataset, which contains data about the Star Wars movies. In this dataset, some of the properties — such as the descriptions of the movies — are present in several different languages, a perfect use case for the application of nested objects in vector search.
Create a repository. For the purposes of this tutorial, you can name it
starwars.Import the
starwars.trigdataset in the repository.Configure the embedding model you want to use for vector search.
Creating the Elasticsearch connector¶
Create the Elasticsearch connector using the following query:
PREFIX conn:<http://www.ontotext.com/connectors/elasticsearch#>
PREFIX inst:<http://www.ontotext.com/connectors/elasticsearch/instance#>
INSERT DATA {
inst:starwars-index conn:createConnector '''
{
"fields": [
{
"fieldName": "film_label",
"propertyChain": [
"http://www.w3.org/2000/01/rdf-schema#label"
],
"datatype": "native:nested",
"objectFields": [
{
"fieldName": "film_label_text",
"propertyChain": [
"$self"
]
},
{
"fieldName": "film_label_vector",
"propertyChain": [
"$self"
],
"datatype": "vector"
}
]
},
{
"fieldName": "film_description",
"propertyChain": [
"https://swapi.co/vocabulary/desc"
],
"datatype": "native:nested",
"objectFields": [
{
"fieldName": "film_description_text",
"propertyChain": [
"$self"
]
},
{
"fieldName": "film_description_vector",
"propertyChain": [
"$self"
],
"datatype": "vector"
}
]
}
],
"types": [
"https://swapi.co/vocabulary/Film"
],
"embeddingModel" : "com.ontotext.embeddings.GraphwiseTransformerClient",
"elasticsearchNode": "localhost:9200",
"languages": [],
"readonly": false,
"detectFields": false,
"importGraph": false,
"skipInitialIndexing": false,
"elasticsearchClusterSniff": false,
"manageIndex": true,
"manageMapping": true,
"bulkUpdateBatchSize": 5000
}
''' .
}
Note
The embeddingModel configuration property in this query assumes the use of the Graphwise Transformer Client, described in the Embedding model configuration for vector search documentation.
This will create an Elasticsearch connector instance named starwars-index using the conn:createConnector predicate, which accepts a JSON configuration describing how RDF data should be indexed and how embeddings should be generated. The connector is configured to index only resources of type https://swapi.co/vocabulary/Film — in other words, only entities representing films from the Star Wars dataset will be exported to Elasticsearch.
Two RDF properties are indexed — rdfs:label and voc:desc. The first corresponds to the films title, and the second to its description. Both fields are defined as native:nested, which means they are stored in Elasticsearch as nested objects rather than flat fields.
Inside each nested field definition, two object fields are declared. For the label, these are film_label_text and film_label_vector. The $self property chain indicates that the literal value itself should be used. The _text field stores the raw string value of the RDF literal, while the _vector field stores its embedding. The embedding is generated automatically because the connector is configured with the embedding model com.ontotext.embeddings.GraphwiseTransformerClient. This model converts textual literals into numerical vectors suitable for semantic similarity search. For more information: https://graphdb.ontotext.com/documentation/11.3/embedding-model-configuration.html#using-a-local-embedding-model-deployed-with-graphwise-transformer.
The same structure is used for film_description, where both the textual description and its vector embedding are stored. As a result, each film document in Elasticsearch contains nested objects holding both original text and its corresponding embedding representation.
The remaining configuration parameters define operational behavior. The connector runs against an Elasticsearch node at localhost:9200. Since manageIndex and manageMapping are set to true, the connector automatically creates and manages the Elasticsearch index and mapping. skipInitialIndexing is false, so existing RDF data is indexed immediately after connector creation. The connector is not read-only, meaning it synchronizes changes from GraphDB to Elasticsearch.
Search query for similar label¶
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX inst: <http://www.ontotext.com/connectors/elasticsearch/instance#>
PREFIX conn: <http://www.ontotext.com/connectors/elasticsearch#>
SELECT ?entity ?actualLabel {
?search a inst:starwars-index ;
conn:query '''
{
"size": 3,
"query": {
"nested": {
"path": "film_label",
"query": {
"knn": {
"field": "film_label.film_label_vector",
"query_vector": "Tähtien varjossa",
"k": 5
}
}
}
}
}
''' ;
conn:entities ?entity .
?entity rdfs:label ?actualLabel .
FILTER(LANG(?actualLabel) = "en")
}
Because of the vector fingerprint, you don’t have to search for exact words anymore. This query performs semantic similarity search over labels (film titles). Since the indexed labels are stored as nested objects, a nested query must be used inside the Elasticsearch JSON. Within that nested context, a KNN query performs vector similarity search on the field film_label.film_label_vector.
The query_vector is provided as a text string because the connector uses the configured embedding model. This text is automatically transformed into a vector before executing the k-nearest neighbors search. The parameter k: 5 specifies that the five closest vectors should be considered, and size: 3 limits the returned hits to three. The connector then binds the matching RDF entities to ?entity, and their English labels are retrieved via rdfs:label.
The query should return the following results:
entity |
actualLabel |
|
|---|---|---|
1 |
“Star Wars: Episode I — The Phantom Menace”@en |
|
2 |
“Star Wars: Episode II — Attack of the Clones”@en |
|
3 |
“Star Wars” |
Note
When the same query is executed using an English translation instead of the Finnish title, the results are almost identical but may differ slightly in ranking. This difference occurs because each language-specific label is embedded and indexed independently — the Finnish query aligns most strongly with Finnish label vectors, while the English query aligns with English label vectors. Since values are not concatenated but stored as separate nested objects, similarity scoring is performed per literal value.
Search query for similar description¶
The query below applies the same principle to film descriptions. A nested kNN search is performed over film_description.film_description_vector. The German query text is embedded and compared against all indexed description vectors.
PREFIX voc: <https://swapi.co/vocabulary/>
PREFIX inst: <http://www.ontotext.com/connectors/elasticsearch/instance#>
PREFIX conn: <http://www.ontotext.com/connectors/elasticsearch#>
SELECT ?entity ?descNoLang {
?search a inst:starwars-index ;
conn:query '''
{
"size": 3,
"query": {
"nested": {
"path": "film_description",
"query": {
"knn": {
"field": "film_description.film_description_vector",
"query_vector": "Diese epische Weltraumoper folgt dem eskalierenden Kampf zwischen einem tyrannischen galaktischen Regime und einer kleinen Rebellengruppe, während ein junger Held in einer alten mystischen Disziplin trainiert und seine engsten Gefährten von einem mächtigen Vollstrecker des Regimes gnadenlos durch die Galaxie verfolgt werden; während Freundschaften auf die Probe gestellt werden und schwierige Entscheidungen getroffen werden müssen, vertieft die Geschichte den Konflikt und gestaltet das Schicksal ihrer zentralen Figuren neu.",
"k": 5
}
}
}
}
}
''' ;
conn:entities ?entity .
?entity voc:desc ?descNoLang .
FILTER(LANG(?descNoLang) = "")
}
The query should provide results such as:
entity |
actualLabel |
|
|---|---|---|
1 |
“The Empire Strikes Back (also known as Star Wars: Episode V — The Empire… |
|
2 |
“Star Wars (later retitled Star Wars: Episode IV — A New Hope) is a 1977 Ame… |
|
3 |
“Star Wars: Episode II — Attack of the Clones is a 2002 American epic space… |
Note
When the same semantic content is queried in English, the most relevant film remains the same, but lower-ranked results may differ in order. This confirms that English and German descriptions are indexed independently and that similarity is computed against each language-specific embedding separately.
See also
Elasticsearch GraphDB Connector documentation.
Embedding model configuration for vector search documentation.
OpenSearch GraphDB Connector documentation.