Extending GraphDB Workbench¶

What’s in this document?

Clone, download, and run GraphDB Workbench
Add your own page and controller
Add repository checks
Repository setup
Select departure and destination airport
Find the paths between the selected airports
Visualize results
Add status message

GraphDB Workbench is now a separate open-source project, enabling the fast development of knowledge graph prototypes or rich UI applications. This provides you with the ability to add your custom colors to the graph views, as well as to easily start a FactForge-like interface.

This tutorial will show you how to extend and customize GraphDB Workbench by adding your own page and Angular controller. We will create a simple paths application that allows you to import RDF data, find paths between to nodes in the graph, and visualize them using D3.

Clone, download, and run GraphDB Workbench ¶

Download and run GraphDB 9.x on the default port 7200.
Clone the GraphDB Workbench project from GitHub.
Enter the project directory and execute npm install in order to install all necessary dependencies locally.

Run npm run start to start a webpack development server that proxies REST requests to localhost:7200:

git clone https://github.com/Ontotext-AD/graphdb-workbench.git graphdb-workbench-paths
cd graphdb-workbench-paths
git checkout <branch>
npm install
npm run start

Now GraphDB Workbench is opened on http://localhost:9000/.

Add your own page and controller ¶

All pages are located under src/pages/, so you need to add your new page paths.html there with a {title} placeholder. The page content will be served by an Angular controller, which is placed under src/js/angular/graphexplore/controllers/paths.controller.js. Path exploration is a functionality related to graph exploration, so you need to register your new page and controller there.

In src/js/angular/graphexplore/app.js:

Import the controller:

'angular/graphexplore/controllers/paths.controller',

Add it to the route provider:

.when('/paths', {
        templateUrl: 'pages/paths.html',
        controller: 'GraphPathsCtrl',
        title: 'Graph Paths',
        helpInfo: 'Find all paths in a graph.',
});

And register it in the menu:

$menuItemsProvider.addItem({
    label: 'Paths',
    href: 'paths',
    order: 5,
    parent: 'Explore',
});

Now you can see your page in GraphDB Workbench.

Next, let’s create the paths controller itself.

In the paths.controller.js that you created, add:

define([
        'angular/core/services',
        'lib/common/d3-utils'
    ],
    function (require, D3) {
        angular
            .module('graphdb.framework.graphexplore.controllers.paths', [
                'toastr',
                'ui.bootstrap',
            ])
            .controller('GraphPathsCtrl', GraphPathsCtrl);

        GraphPathsCtrl.$inject = ["$scope", "$rootScope", "$repositories", "toastr", "$timeout", "$http", "ClassInstanceDetailsService", "AutocompleteService", "$q", "$location"];

        function GraphPathsCtrl($scope, $rootScope, $repositories, toastr, $timeout, $http, ClassInstanceDetailsService, AutocompleteService, $q, $location) {

        }
    }
);

And register the module in src/js/angular/graphexplore/modules.js
'graphdb.framework.graphexplore.controllers.paths',

Now your controller and page are ready to be filled with content.

Add repository checks ¶

In your page, you need a repository with data in it. Like most views in GraphDB, you need to have a repository set. The template that most of the pages use is similar to this, where the repository-is-set div is where you put your html. Error handling related to repository errors is added for you.

<div class="container-fluid">
    <h1>
        {{title}}
        <span class="btn btn-link"
              popover-template="'js/angular/templates/titlePopoverTemplate.html'"
              popover-trigger="mouseenter"
              popover-placement="bottom-right"
              popover-append-to-body="true"><span class="icon-info"></span></span>
    </h1>
    <div core-errors></div>
    <div system-repo-warning></div>
    <div class="alert alert-danger" ng-show="repositoryError">
        <p>The currently selected repository cannot be used for queries due to an error:</p>
        <p>{{repositoryError}}</p>
    </div>

    <div id="repository-is-set" ng-show="getActiveRepository() && !isLoadingLocation() && hasActiveLocation() && 'SYSTEM' !== getActiveRepository()">
        {{getActiveRepository()}}
    </div>
</div>

You need to define the functions on which this snippet depends in your paths.controller.js. They use the repository service that you imported in the controller definition.

$scope.getActiveRepository = function () {
    return $repositories.getActiveRepository();
};

$scope.isLoadingLocation = function () {
    return $repositories.isLoadingLocation();
};

$scope.hasActiveLocation = function () {
    return $repositories.hasActiveLocation();
};

Repository setup ¶

Create a repository.
Import the airports.ttl dataset.
Enable the Autocomplete index for your repository.
Execute the following SPARQL insert to add direct links for flights:

PREFIX onto: <http://www.ontotext.com/>
INSERT {
  ?node onto:hasFlightTo ?destination .
}  WHERE {
    ?flight <http://openflights.org/resource/route/sourceId> ?node .
    ?flight <http://openflights.org/resource/route/destinationId> ?destination .
}

Now we will search for paths between airports based on the hasFlightTo predicate.

Select departure and destination airport ¶

Now let’s add inputs using Autocomplete to select the departure and destination airports. Inside the repository-is-set diff, add the two fields. Note the visual-callback="findPath(startNode, uri)" snippet that defines the callback to be executed once a value is selected through the Autocomplete. uri is the value from the Autocomplete. The following code sets the starNode variable in Angular and calls the findPath function when the destination is given. You can find out how to define this function in the scope a little further down in this tutorial.

<div class="card mb-2">
    <div class="card-block">
        <h3>From</h3>
        <p>Search for a start node</p>
        <search-resource-input class="search-rdf-resources"
                               namespacespromise="getNamespacesPromise"
                               autocompletepromisestatus="getAutocompletePromise"
                               text-button=""
                               visual-button="Show"
                               visual-callback="startNode = uri"
                               empty="empty"
                               preserve-input="true">
        </search-resource-input>
    </div>
</div>
<div class="card mb-2">
    <div class="card-block">
        <h3>To</h3>
        <p>Search for an end node</p>
        <search-resource-input class="search-rdf-resources"
                               namespacespromise="getNamespacesPromise"
                               autocompletepromisestatus="getAutocompletePromise"
                               text-button=""
                               visual-button="Show"
                               visual-callback="findPath(startNode, uri)"
                               empty="empty"
                               preserve-input="true">
        </search-resource-input>
    </div>
</div>

They need the getNamespacesPromise and getAutocompletePromise to fetch the Autocomplete data. They should be initialized once the repository has been set in the controller.

function initForRepository() {
    if (!$repositories.getActiveRepository()) {
        return;
    }
    $scope.getNamespacesPromise = ClassInstanceDetailsService.getNamespaces($scope.getActiveRepository());
    $scope.getAutocompletePromise = AutocompleteService.checkAutocompleteStatus();
}

$scope.$on('repositoryIsSet', function(event, args) {
    initForRepository();
});
initForRepository();

Note that both of these functions need to be called when the repository is changed, because you need to make sure that Autocomplete is enabled for this repository, and fetch the namespaces for it. Now you can autocomplete in your page.

Find the paths between the selected airports ¶

Now let’s implement the findPath function in the scope. It finds all paths between nodes by using a simple depth-first search algorithm (recursive algorithm based on the idea of backtracking).

For each node, you can obtain its siblings with a call to the rest/explore-graph/links endpoint. This is the same endpoint the Visual graph is using to expand node links. Note that it is not part of the GraphDB API, but we will reuse it for simplicity.

As an alternative, you can also obtain the direct links of a node by sending a SPARQL query to GraphDB.

Note

This is a demo implementation. For each repository containing a lot of links, the proposed approach is not appropriate, as it will send a request to the server for each node. This will quickly result in a huge amount of requests, which will very soon flood the browser.

var maxPathLength = 3;

var findPath = function (startNode, endNode, visited, path) {
    // A path is found, return a promise that resolves to it
    if (startNode === endNode) {
        return $q.when(path)
    }
    // Find only paths with maxLength, we want to cut only short paths between airports
    if (path.length === maxPathLength) {
        return $q.when([])
    }
    return $http({
        url: 'rest/explore-graph/links',
        method: 'GET',
        params: {
            iri: startNode,
            config: 'default',
            linksLimit: 50
        }
    }).then(function (response) {
        // Use only links with the hasFlightTo predicate
        var flights = _.filter(response.data, function(r) {return r.predicates[0] == "hasFlightTo"});
        // For each links, continue to search path recursively
        var promises = _.map(flights, function (link) {
            var o = link.target;
            if (!visited.includes(o)) {
                return findPath(o, endNode, visited.concat(o), path.concat(link));
            }
            return $q.when([]);
        });
        // Group together all promises that resolve to paths
        return $q.all(promises);
    }, function (response) {
        var msg = getError(response.data);
        toastr.error(msg, 'Error looking for path node');
    });
}

$scope.findPath = function (startNode, endNode) {
    findPath(startNode, endNode, [startNode], []).then(function (linksFound) {
        renderGraph(_.flattenDeep(linksFound));
    });
}

The findPath recursive function returns all the promises that will or will not resolve to paths. Each path is a collection of links.

When all promises are resolved, you can flatten the array to obtain all links from all paths and draw one single graph with these links. Graph drawing is done with D3 in the renderGraph function. It needs a graph-visualization element to draw the graph inside. Add it inside the repository-is-set element below the autocomplete divs.

Additionally, import graphs-visualizations.css to reuse some styles.

<div class="card mb-2">
    ..
</div>
<div class="card mb-2">
    ...
</div>
<div class="graph-visualization"></div>

<link href="css/graphs-vizualizations.css" rel="stylesheet"/>

Now add the renderGraph render function mentioned above:

var width = 1000,
    height = 1000;

var nodeLabelRectScaleX = 1.75;

var force = d3.layout.force()
    .gravity(0.07)
    .size([width, height]);

var svg = d3.select(".main-container .graph-visualization").append("svg")
    .attr("viewBox", "0 0 " + width + " " + height)
    .attr("preserveAspectRatio", "xMidYMid meet");

function renderGraph(linksFound) {
    var graph = new Graph();

    var nodesFromLinks = _.union(_.flatten(_.map(linksFound, function (d) {
        return [d.source, d.target];
    })));

    var promises = [];
    var nodesData = [];

    // For each node in the graph find its label with a rest call
    _.forEach(nodesFromLinks, function (newNode, index) {
        promises.push($http({
            url: 'rest/explore-graph/node',
            method: 'GET',
            params: {
                iri: newNode,
                config: 'default',
                includeInferred: true,
                sameAsState: true
            }
        }).then(function (response) {
            // Save the data for later
            nodesData[index] = response.data;
        }));
    });

    // Waits for all of the collected promises and then:
    // - adds each new node
    // - redraws the graph
    $q.all(promises).then(function () {
        _.forEach(nodesData, function (nodeData, index) {
            // Calculate initial positions for the new nodes based on spreading them evenly
            // on a circle.
            var theta = 2 * Math.PI * index / nodesData.length;
            var x = Math.cos(theta) * height / 3;
            var y = Math.sin(theta) * height / 3;
            graph.addNode(nodeData, x, y);
        });

        graph.addLinks(linksFound);
        draw(graph);
    });
}

function Graph() {
    this.nodes = [];
    this.links = [];

    this.addNode = function (node, x, y) {
        node.x = x;
        node.y = y;
        this.nodes.push(node);
        return node;
    };

    this.addLinks = function (newLinks) {
        var nodes = this.nodes;
        var linksWithNodes = _.map(newLinks, function (link) {
            return {
                "source": _.find(nodes, function (o) {
                    return o.iri === link.source;
                }),
                "target": _.find(nodes, function (o) {
                    return o.iri === link.target;
                }),
                "predicates": link.predicates
            };
        });
        Array.prototype.push.apply(this.links, linksWithNodes);
    };
}

// Draw the graph using d3 force layout
function draw(graph) {
    d3.selectAll("svg g").remove();

    var container = svg.append("g").attr("class", "nodes-container");

    var link = svg.selectAll(".link"),
        node = svg.selectAll(".node");

    force.nodes(graph.nodes).charge(-3000);
    force.links(graph.links).linkDistance(function (link) {
        // link distance depends on length of text with an added bonus for strongly connected nodes,
        // i.e. they will be pushed further from each other so that their common nodes can cluster up
        return getPredicateTextLength(link) + 30;
    });

    function getPredicateTextLength(link) {
        var textLength = link.source.size * 2 + link.target.size * 2 + 50;
        return textLength * 0.75;
    }


    // arrow markers
    container.append("defs").selectAll("marker")
        .data(force.links())
        .enter().append("marker")
        .attr("class", "arrow-marker")
        .attr("id", function (d) {
            return d.target.size;
        })
        .attr("viewBox", "0 -5 10 10")
        .attr("refX", function (d) {
            return d.target.size + 11;
        })
        .attr("refY", 0)
        .attr("markerWidth", 10)
        .attr("markerHeight", 10)
        .attr("orient", "auto")
        .append("path")
        .attr("d", "M0,-5L10,0L0,5 L10,0 L0, -5");

    // add the links, nodes, predicates and node labels
    var link = container.selectAll(".link")
        .data(graph.links)
        .enter().append("g")
        .attr("class", "link-wrapper")
        .attr("id", function (d) {
            return d.source.iri + '>' + d.target.iri;
        })
        .append("line")
        .attr("class", "link")
        .style("stroke-width", 1)
        .style("fill", "transparent")
        .style("marker-end", function (d) {
            return "url(" + $location.absUrl() + "#" + d.target.size + ")";
        });

    var predicate = container.selectAll(".link-wrapper")
        .append("text")
        .text(function (d, index) {
            return d.predicates[0];
        })
        .attr("class", function (d) {
            if (d.predicates.length > 1) {
                return "predicates";
            }
            return "predicate";
        })
        .attr("dy", "-0.5em")
        .style("text-anchor", "middle")
        .style("display", "")
        .on("mouseover", function (d) {
            d3.event.stopPropagation();
        });

    var node = container.selectAll(".node")
        .data(graph.nodes)
        .enter().append("g")
        .attr("class", "node-wrapper")
        .attr("id", function (d) {
            return d.iri;
        })
        .append("circle")
        .attr("class", "node")
        .attr("r", function (d) {
            return d.size;
        })
        .style("fill", function (d) {
            return "rgb(255, 128, 128)";
        })

    var nodeLabels = container.selectAll(".node-wrapper").append("foreignObject")
        .style("pointer-events", "none")
        .attr("width", function (d) {
            return d.size * 2 * nodeLabelRectScaleX;
        });
    // height will be computed by updateNodeLabels

    updateNodeLabels(nodeLabels);

    function updateNodeLabels(nodeLabels) {
        nodeLabels.each(function (d) {
            d.fontSize = D3.Text.calcFontSizeRaw(d.labels[0].label, d.size, 16, true);
            // TODO: get language and set it on the label html tag
        })
            .attr("height", function (d) {
                return d.fontSize * 3;
            })
            // if this was kosher we would use xhtml:body here but if we do that angular (or the browser)
            // goes crazy and resizes/messes up other unrelated elements. div seems to work too.
            .append("xhtml:div")
            .attr("class", "node-label-body")
            .style("font-size", function (d) {
                return d.fontSize + 'px';
            })
            .append('xhtml:div')
            .text(function (d) {
                return d.labels[0].label;
            });
    }


    // Update positions on tick
    force.on("tick", function () {

        // recalculate links attributes
        link.attr("x1", function (d) {
            return d.source.x;
        }).attr("y1", function (d) {
            return d.source.y;
        }).attr("x2", function (d) {
            return d.target.x;
        }).attr("y2", function (d) {
            return d.target.y;
        });

        // recalculate predicates attributes
        predicate.attr("x", function (d) {
            return d.x = (d.source.x + d.target.x) * 0.5;
        }).attr("y", function (d) {
            return d.y = (d.source.y + d.target.y) * 0.5;
        });

        // recalculate nodes attributes
        node.attr("cx", function (d) {
            return d.x;
        }).attr("cy", function (d) {
            return d.y;
        });


        nodeLabels.attr("x", function (d) {
            return d.x - (d.size * nodeLabelRectScaleX);
        }).attr("y", function (d) {
            // the height of the nodeLabel box is 3 times the fontSize computed by updateNodeLabels
            // and we want to offset it so that its middle matches the centre of the circle, hence divided by 2
            return d.y - 3 * d.fontSize / 2;
        });

    });
    force.start();
}

It obtains the URIs for the nodes from all links, and finds their labels through calls to the rest/explore-graph/node endpoint. A graph object is defined to represent the visual abstraction, which is simply a collection of nodes and links. The draw(graph) function does the D3 drawing itself using the D3 force layout.

Visualize results ¶

Now let’s find all paths between Sofia and La Palma with maximum 2 nodes in between (maximum path length 3):

Note

The airports graph is highly connected. Increasing the maximum path length will send too many requests to the server. The purpose of this tutorial is to introduce you to the Workbench extension with a naive paths prototype.

Add status message ¶

Noticing that path finding can take some time, we may want to add a message for the user.

$scope.findPath = function (startNode, endNode) {
    $scope.pathFinding = true;
    findPath(startNode, endNode, [startNode], []).then(function (linksFound) {
        $scope.pathFinding = false;
        renderGraph(_.flattenDeep(linksFound));
    });
}

<div ng-show="pathFinding">Looking for all paths between nodes...</div>
<div class="graph-visualization"></div>

The source code for this example can be found in the workbench-paths-example GitHub project.