Using GraphDB’s LLM tools with external clients¶

What’s in this document?

Using GraphDB’s LLM tools with MCP clients
Using GraphDB’s LLM tools with Dify and other LLM agent frameworks

The Large Language Model (LLM) tools that enable features such as Talk to Your Graph can be used outside of GraphDB with external clients. The tools are exposed through the API and can be used with either Model Context Protocol (MCP)-compatible clients or with frameworks such as Dify.

Using GraphDB’s LLM tools with MCP clients ¶

The Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to large language models (LLMs). It provides a standardized way to connect AI models to different data sources and tools, including the LLM tools of GraphDB, and build agents and complex workflows. The MCP protocol is designed to support streaming communication where the client sends messages and the server responds with events in a structured format, one per message. Each message is sent as a data: block in a separate event, just like a chunked stdio or websocket stream. The MCP responses are wrapped as valid SSE events, preserving the MCP structure.

GraphDB runs an MCP server that enables MCP-compatible clients to perform semantic and structured queries against RDF repositories using GraphDB. It is designed to serve requests over the streamable HTTP transport, providing streaming compatibility across various clients. Communication through the transport is done through HTTP POST and GET requests.

The GraphDB MCP server runs on the same HTTP port as GraphDB. The GraphDB MCP server exposes an /mcp endpoint for POST and GET requests.

MCP security configurations ¶

The GraphDB MCP server uses Spring Security to define basic access permissions. By default, both endpoints are open to anonymous users with access READ. This means any client can connect to these endpoints, as long as the free access is enabled with at least READ given repository rights. This is acceptable only if the underlying GraphDB instance allows read access to the target repository, either because the user is authenticated or GraphDB is configured for free access.

GraphDB’s built-in GraphDB access control rules are still enforced at the repository level. This ensures that only users or API clients with the appropriate permissions (such as the ability to read a given resource) can query specific data or graphs in the repositories, and that unauthorized access is blocked by GraphDB itself, and not just at the HTTP level.

Exposed LLM tools for MCP clients ¶

GraphDB exposes all query methods available when creating Talk to Your Graph agents as tools, as well as a few additional others:

Repository listing: Provides system information about available repositories in GraphDB MCP Server.
Full-text search: Queries GraphDB using Full-text search and returns a subgraph of RDF triples.
IRI discovery: Discovers IRIs based on full-text search label matches.
Retrieval search: Queries GraphDB by full-text search and returns textual data suitable for summarization or memory operations. Optimized for ChatGPT Retrieval.
Autocomplete IRI discovery: Uses GraphDB’s autocomplete index to fetch relevant IRIs by searching their names.
Similarity search: Finds conceptually related entities using a text similarity index.
SPARQL query interface: Executes a SPARQL SELECT, CONSTRUCT, or DESCRIBE query against GraphDB and return the results.
Ontology schema extraction: Extracts ontology schemas used by clients to build structured queries.
Similarity options discovery: Provides information about available similarity search capabilities in a GraphDB repository, including supported vector fields, connector types, and similarity indexes. Clients use this information to correctly configure parameters when performing similarity searches.

SSE transport and the MCP gateway ¶

The GraphDB MCP server also supports communication over the Server-Sent Events (SSE) for backward compatibility with older clients. The use of SSE can be configured through the graphdb.mcp.transport.mode configuration property, which is described in details in the MCP server properties documentation.

To handle the full bidirectional behavior required by the MCP protocol, the GraphDB MCP server exposes two endpoints for SSE communication:

POST /mcp/message for the client to server communication
GET /mcp/sse for the streaming of responses from the server to the client

Note

Prior to GraphDB 11.3, the MCP server supported only the SSE protocol, and did so by default.

Communication through the SSE is facilitated by an MCP gateway, which allows MCP clients to connect to a secured GraphDB MCP server. Some older MCP clients support only stdio-based MCP servers, and thus a midleware mcp-gateway service is required to bridge the gap between stdio employed by those MCP clients, and http/sse used by the GraphDB MCP server. The MCP gateway converts stdio into http/sse and routes it to the GraphDB MCP server.

Tip

GraphDB maintains a fork of the MCP gateway service, with its own installation and configuration instructions.

Using GraphDB’s LLM tools with Dify and other LLM agent frameworks ¶

GraphDB also exposes LLM tools using the OpenAPI specification for integration with LLM agent frameworks such as Dify. , custom UIs, and Python-based agents. This API facilitates the use of Talk to Your Graph by bridging the gap between rapid prototyping in the Workbench and production-grade orchestration with external agents.

Supported configuration types ¶

Unlike with MCP clients, tools exposed to these frameworks are exposed on a per-repository basis, using the base path /rest/llm. The exposed tools can be configured in one of two ways:

Static — Static tool definitions configured in yaml
Dynamic — Tools defined in the preconfigured agents in Talk to Your Graph

Tip

If you have already configured a Talk to Your Graph agent, you can obtain the configuration ID of that agent by navigating to Talk to Your Graph ‣ Edit agent settings of the agent you want to use and then selecting External integration configuration on the Edit Agent dialog box.

Exposed tools for the Dify framework ¶

GraphDB exposes all query methods available when creating Talk to Your Graph agents as tools. Since tools are configured on a per repository level, the Repository listing and Ontology schema extraction tools exposed to MCP clients are not needed and thus not available.

The full list of tools can be obtained by the client with the following call, where {configType} is either yaml or ttyg, and configId is the unique identifier of the LLM tool configuration:

GET /rest/llm/tool/{configType}/{configId}

Tip

See the Example yaml configuration below.

You can access tools with the following call, where {toolType} is the name of the tool:

POST /rest/llm/tool/{configType}/{configId}/{toolType}

The name of the tools are:

sparql_query
fts_search
similarity_search
retrieval_search
autocomplete_iri_discovery_search
iri_discovery
now

Example yaml configuration ¶

The following yaml configuration enables the SPARQL query tool and full-text search for a repository named acme, and the SPARQL query tool for a repository named starwars:

acmeConfig:
  repositoryId: "acme"
  tools:
    sparql_query:
      enabled: true
      ontologyGraph: "http://example.com/acme"
      ontologyQuery: null
      addMissingNamespaces: false
    retrieval_search:
      enabled: false
      connectorInstance: null
      queryTemplate: "{\n  \"query\": {\n    \"type\": \"string\",\n    \"title\"\
        : \"Query\"\n  },\n  \"filter\": {\n    \"type\": \"object\",\n    \"properties\"\
        : {\n      \"document_id\": {\n        \"type\": \"string\",\n        \"title\"\
        : \"Document Id\"\n      }\n    }\n  },\n  \"top_k\": {\n    \"type\": \"\
        integer\",\n    \"title\": \"Top K\",\n    \"default\": 3\n  }\n}"
      limit: 0
    similarity_search:
      enabled: false
      similarityIndex: null
      resultThreshold: 0.6
      limit: 0
    fts_search:
      enabled: true
      limit: 0
    iri_discovery:
      enabled: false
starwarsConfig:
  repositoryId: "starwars"
  tools:
    sparql_query:
      enabled: true
      ontologyGraph: "https://swapi.co/ontology/"
      ontologyQuery: null
      addMissingNamespaces: false

Using Dify API extension ¶

You can also enable compatibility with Dify’s tool-calling protocol, which supports seamless plug-in of GraphDB tools into Dify workflows:

POST /rest/llm/{configType}/{configId}/dify

When creating an agent in Dify, this extension allows you to obtain the SPARQL instructions or ontology directly from a static yaml config file or from the configurations of a Talk to Your Graph agent.

To obtain static instructions, simply point to the yaml file you want to use, for example:
```
POST /rest/llm/yaml/acmeConfig/dify
```
To obtain dynamic instructions from a Talk to Your Graph agent, use the Dify extension endpoint you can find by navigating to Talk to Your Graph ‣ Edit configurations of the agent whose instructions or ontology you want to use, and then navigating to External integration configuration ‣ Dify extension.

You can then use that endpoint by creating a new variable in the agent (which can take either sparql_instructions or ontology), and adding the endpoint to that variable.