A pretty sweet RML engine
(Disclaimer: The current state of CARML is early beta. The team expects to deliver a release in November 2017. The next release will offer improved code quality, more test coverage, more documentation and several features currently on the product backlog.)
CARML is being developed by Taxonic in cooperation with Kadaster.
- Releases
- Introduction
- Getting started
- Validating your RML mapping
- Input stream extension
- Function extension
- Supported data source types
- Roadmap
20 Sep 2017 - CARML 0.0.1 20 Okt 2017 - CARML 0.1.0
CARML is a java library that transforms structured sources to RDF based as declared in and RML mapping, in accordance with the RML spec. It is considered by many as the optimal choice for mapping structured sources to RDF.
The best place to start learning about RML is at the source, but basically RML is defined as a superset of R2RML which is a W3C recommendation that describes a language for expressing mappings from relational databases to RDF datasets. RML allows not only the expression of mappings for relational databases, but generalizes this to any structured source. All you need is a way to iterate over and query the source.
[ CARML will soon be available at the Central Maven Repository. ]
CARML is based on RDF4J, and currently the Mapper directly outputs an RDF4J Model.
Set<TriplesMap> mapping =
RmlMappingLoader
.build()
.load(Paths.get("path-to-mapping-file"), RDFFormat.TURTLE);
RmlMapper mapper =
RmlMapper
.newBuilder()
.fileResolver(Paths.get("folder/containing/data/sources"))
.addFunctions(new RmlFunctions())
.build();
Model result = mapper.map(mapping);We're not set up for full mapping validation yet. But, to help you get those first nasty mapping errors out, we've created a SHACL shapes graph (here) that validates RML mappings. You can use the SHACL playground to easily test your mapping.
When it comes to non-database sources, the current RML spec only supports the specification of file based sources in an rml:LogicalSource. However, it is often very useful to be able to transform stream sources.
To this end CARML introduces the notion of 'Named Streams'. Which follows the ontology defined here.
So now, you can define streams in your mapping like so:
:SomeLogicalSource
rml:source [
a carml:InputStream ;
# NOTE: name is not mandatory and can be left unspecified, when working with a single stream
carml:streamName "stream-A" ;
];
rml:referenceFormulation ql:JSONPath;
rml:iterator "$"
.
In order to provide access to the input stream, it needs to be registered on the mapper.
RmlMapper mapper = RmlMapper.newBuilder().build();
mapper.bindInputStream("stream-A", inputStream);Note that it is possible to register several streams, allowing you to combine several streams to create your desired RDF output.
A recent development related to RML is the possibility of adding functions to the mix. This is a powerful extension that increases the expressivity of mappings immensely. Do note that the function ontology is still under development at UGhent.
Because we believe that this extension can already be of great value, we've already adopted it in CARML.
The way it works is, you describe the execution of a function in terms of the Function Ontology (FnO).
Take for example the SumFunction example of the FnO spec. This defines an instance ex:sumFunction of class fno:Function that is able to compute the sum of two values provided as parameters of the function at execution time.
It also describes an instance ex:sumExecution of fno:execution, which fno:executes ex:sumFunction which descibes an instance of an execution of the defined sum function. In this case with parameters 2 and 4.
To be able to use this in RML mappings we use executions of instances of fno:Function to determine the value of a term map. The execution of a function can be seen as a post-processing step in the evaluation of a term map.
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix rml: <http://semweb.mmlab.be/ns/rml#> .
@prefix fnml: <http://semweb.mmlab.be/ns/fnml#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix fno: <http://semweb.datasciencelab.be/ns/function#> .
@prefix ex: <http://example.org#> .
ex:sumValuePredicateObjectMap
rr:predicate ex:total ;
rr:objectMap [
a fnml:FunctionTermMap ;
fnml:functionValue [
rml:logicalSource ex:LogicalSource ;
rr:subjectMap [
rr:template "functionExec";
rr:termType rr:BlankNode ;
rr:class fno:Execution
] ;
rr:predicateObjectMap
[
rr:predicate fno:executes ;
rr:objectMap [
rr:constant ex:sumFunction ;
]
] ,
[
rr:predicate ex:intParameterA ;
rr:objectMap [ rml:reference "foo" ]
] ,
[
rr:predicate ex:intParameterB ;
rr:objectMap [ rml:reference "bar" ]
]
] ;
rr:datatype xsd:integer ;
]
.
A function can be created in any .java class. The function should be annotated with @FnoFunction, providing the string value of the function IRI, and the function parameters with @FnoParam, providing the string value of the function parameter IRIs.
public class RmlFunctions {
@FnoFunction("http://example.org/sumFunction")
public int sumFunction(
@FnoParam("http://example.org/intParameterA") int intA,
@FnoParam("http://example.org/intParameterB") int intB
) {
return intA + intB;
}
}The class or classes containing the annotated functions can then be registered on the mapper via the RmlMapper#addFunctions method.
RmlMapper mapper =
RmlMapper
.newBuilder()
.classPathResolver(contextPath)
.addFunctions(new RmlFunctions())
.build();
Model result = mapper.map(mapping);It is recommended to describe and publish new functions in terms of FnO for interpretability of mappings, and, possibly, reuse of functions, but it's not mandatory for use in CARML.
Note that it is currently possible to specify and use function executions as parameters of other function executions in CARML, although this is not (yet?) expressible in FnO.
| Data source type | Reference query language |
|---|---|
| JSON | Jayway JsonPath 2.4.0 |
Coming soon: XML, CSV
- Add support for XML sources
- Add support for CSV sources
- CARML Command line interface
- Better support for large sources
- Improved join / parent triples map performance
- Split off provisional RDF Mapper as separate library