Embedding model configuration for vector search

Both Elasticsearch GraphDB Connector and OpenSearch GraphDB Connector support integrated vector search, which uses an embedding model to generate vectors and provide better recall than full-text search. You can use a variety of models to generate the embeddings, including OpenAI embedding models, Elasticsearch (when using Elasticsearch), or locally deployed models. Before you can create a connector instance with vector search capabilities, you need to first choose and configure the embedding model.

Note

This does not apply if you are using Elasticsearch to create the embeddings. Refer to the Elasticsearch documentation for more information on how to configure the creation of the embeddings in that scenario.

The Embedding Model Clients can also read a set of system properties configured through GraphDB. These properties can configured through GraphDB, but are described in the configuration section of the Embedding Model Clients project.

Using a local embedding model deployed with Graphwise Transformer

You can use a locally deployed model to generate the embeddings, allowing for advanced customization and increased security. In fact, GraphDB is distributed with a client for such a locally deployed Hugging Face transformer model, which can be deployed using the Graphwise Transformer tool.

The github page of the Graphwise Transformer provides thorough instructions on how to deploy the model using Graphwise Transformer.

Once you have deployed the model, when creating the connector instance you need to configure the embeddingModel parameter to com.ontotext.embeddings.GraphwiseTransformerClient. By default both the transformer and the client are configured with the sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 model. To change the default model used by Graphwise Transformer:

1. Set the default_model property in the config.properties file of the Graphwise Transformer to your <model_name>, as described in the configuration section of the Graphwise Transformer documentation.

2. Set the GraphDB configuration property graphwise.transformer.embedding.model.name to <model_name> before starting GraphDB.

Note

The default model sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 has a limit of 128 tokens. Text above that limit will be automatically truncated by the model. The Graphwise Transformer client will also truncate texts for embeddings larger than 4 megabytes because bigger documents will not fit GRPC limitations. If you have texts for embeddings bigger than these limitations, make sure you apply document chunking and create vector fields over the chunks of the documents.

Tip

If you are using a locally deployed embedding model, consider CUDA cards for faster generation. The model will automatically be placed on the most performant available device.

Using an OpenAI embedding model

You can also use an OpenAI embedding model to create the embeddings for vector search.

When using OpenAI to create the embeddings, the following GraphDB configuration properties must be configured prior to starting GraphDB:

• openai.embedding.model.name — the name of the OpenAI model you want to use for the embeddings

• openai.embedding.model.api.key — your OpenAI API key

• openai.embedding.model.dimensions — the embedding model dimensions

Then, when creating the GraphDB Elasticsearch GraphDB Connector and OpenSearch GraphDB Connector, configure the embeddingModel property (or the Embedding model if using the GraphDB Workbench) to com.ontotext.embeddings.OpenAiEmbeddingModel.

Using an alternative model provider through the EmbeddingModel interface of langchain4j

You can use an alternative model provider by implementing the EmbeddingModel interface of langchain4j. To do so, package the implementation and all its dependencies and put it in the connector classpath (.../lib/plugins/elasticsearch-connector/).

When you create the connector in GraphDB, specify the fully classified class name of the implementation class in Java using the embedding model field of the connector.

Tip

You can base your solution on the examples in our Embedding Model clients project.