Odinson can be used to rapidly query a natural language knowledge base and extract structured relations. Query patterns can be designed over (a) surface (e.g. #1), syntax (e.g., #2), or a combination of both (e.g., #3-5). These examples were executed over a collection of 8,479 scientific papers, corresponding to 1,105,737 sentences. Please note that the rapidity of the execution allows a user to dynamically develop these queries in real-time, immediately receiving feedback on the coverage and precision of the patterns at scale.
Odinson supports several features:
- Patterns over tokens, including boolean patterns over token features like lemma, POS tags, NER tags, chunk tags, etc
- Patterns over syntax by matching paths in a dependency graph. Note that this actually agnostic to the tags in the graph edges and it could be repurposed for matching over semantic roles or something else.
- Named captures for extracting the different entities involved in a relation
And there are many more on the way:
- Testing framework and extensive test suite
- Better error messages
- Better support for greedy and lazy quantifiers
- Lookaround assertions
- Support for an internal state to hold intermediate mentions, allowing for the application of patterns in a cascade
- Support for grammars (similar to odin)
- Filtering results by document metadata (e.g., authors, publication date)
We would also love to hear any questions, requests, or suggestions you may have.
It consists of several subprojects:
- core: the core odinson library
- extra: these are a few apps that we need but don't really belong in core, for example, licensing issues
- backend: this is a REST API for odinson
- ui: this is a webapp that we are building to interact with the system and visualize results to enable rapid development
The three apps in extra are:
- AnnotateText: parses text documents using processors
- IndexDocuments: reads the parsed documents and builds an odinson index
- Shell: this is a shell where you can execute queries (we will replace this with the webapp soon)
We have made a few example queries to show how the system works. For this we used a collection of 8,479 scientific papers (or 1,105,737 sentences). Please note that the rapidity of the execution allows a user to dynamically develop these queries in real-time, immediately receiving feedback on the coverage and precision of the patterns at scale.
This example shows odinson applying a pattern over surface features (i.e., words) to extract mentions of causal relations. Note that Odinson was able to find 3,774 sentences that match the pattern in 0.18 seconds.
This example shows how Odinson can also use patterns over syntax. In this case it tries to find hypernym relations. It finds 10,562 matches in 0.37 seconds.
Example of how a surface pattern can be extended (using syntax patterns) to extract additional contextual information.
This example shows how surface and syntax can be combined in a single pattern. This pattern finds 12 sentences that match in our corpus of 1,105,737 sentences. It does this in 0.01 seconds.
This example shows how we can match over different aspects of tokens, lemmas in this example. Note that the ability to utilize syntax helps with the precision of the extractions (as compared with the overly simple surface rule above). Odinson finds 5,489 matches in 0.18 seconds.
Example of how more complex patterns can be developed, for example, to extract the polarity of a causal influence and a context in which it applies.
This is an example of a slightly more complex pattern. Odinson is able to apply it over our corpus and finds 228 matches in 0.04 seconds.
We are also working on a web interface that will simplify debugging by displaying more information than the shell. This interface will allow us to display syntactic information when needed. We would also like to be able to interact with it to correct extractions or bootstrap patterns.
