How to extend the reasoner

You probably want to do more than reason about rdf triples, and now you are asking yourself: "How do I get my data in there? And custom conditions? And what else do I need?"

Despite the title of this chapter, you do not really need to extend the reasoner. Its algorithm is pretty generic and does not need any customization to handle your data -- it is the rete network and the rule parser you want to add to.

In the following sections I will describe step by step what is needed to support your own custom datatypes in this rule based reasoning system. To illustrate this we will take a look at the test/MutableWMEs.cpp: This test case was created to test the just recently introduces UPDATE flag which allows us to change data that was already added to the reasoner. Please note that I would still advise you to make your data immutable whenever possible. I will probably add some excerpts from rete-rdf when the mutable wme case does something unusual.

[[TOC]]

Working memory element

First of all you will need to implement your own working memory element that holds the data you want to add. One example for such a WME is the Triple class which represents a single rdf triple, or the aforementioned, completely artificial MutableWME.

Your own WME must inherit rete::WME and implement three methods:

1. std::string toString() const returns a string representation of the WME for visualization purposes, e.g. in the images of rete networks shown before.
2. const std::string& type() const returns a string that unambiguously identifies the type of the WME. Instances of the Triple class will e.g. return the string Triple. Please note that the return type is a reference to const, and it is advised to use a static variable for this.
3. bool operator < (const WME& other) const is needed to keep unique sets of WMEs. Make sure that your implementation ensures a proper ordering even when comparing with WMEs of different classes. For exactly this problem the previous method was introduced: You can first compare the others type identifier with yours, and if they differ use them for the ordering. Also, the ordering of your WMEs must never change, or else the internal datastructures (mainly std::set) of our system get corrupted. This is directly fulfilled if your WME is immutable, but for mutable WMEs you must base the implementation on a never changing value. In case of the MutableWME example I simply chose to compare the memory addresses. This also implies that every instance of it is unique and has to be processed by the network -- it is what makes immutable WMEs so charming: Whenever we try to add multiple instances of Triple with the same values for subject, predicate and object, the system treats them all as the same and will only keep one instance of it, but link it with all the evidences. It also means that when multiple rules on multiple paths infer the same triple, it really is just that one triple, we get only one notification for its ASSERT, etc. This does not work for mutable WMEs. Just keep that in mind.

So, the first part is done, our WME class (as said before, please take this with a grain of salt):

/**
    A very simple mutable WME.
*/
class MutableWME : public WME {
    static const std::string type_;
public:
    using Ptr = std::shared_ptr<MutableWME>;
    std::string value_;

    std::string toString() const override { return "Mutable: " + value_; }

    const std::string& type() const override
    {
        return type_;
    }

    bool operator < (const WME& other) const override
    {
        if (type() != other.type()) return type() < other.type();

        // we could omit this cast, as we already checked for the type()
        auto o = dynamic_cast<const MutableWME*>(&other);
        if (o)
        {
            return this < o;
        }
        // reaching this means that the cast failed despite type() being equal
        throw std::exception();
    }
};
const std::string MutableWME::type_("MutableWME");

Accessors to the WMEs internal values

In order to use your WMEs internal values in your rules, you will need to implement accessor classes. In our example, the only value we are interested in is the std::string value_, so we only need one accessor class that inherits rete::AccessorBase. But rete::AccessorBase only exists as a common base class and is itself not of much use, though it defines some methods we need to implement one way or another:

1. bool equals(const Accessor& other) const returns true if the accessors access the same data, i.e. access the same WME in token (this is already checked in rete::Accessor::operator ==) and the same values in the WME. Since our MutableWMEAccessor can only access that one value it only needs to check if the other is a MutableWMEAccessor at all.
2. Accessor* clone() const accessors must be clonable since we need to do some indexing magic inside the rule parser. Make sure to also copy the value of the current token-index, index_.
3. (optional) std::string toString() const is used for visualization purposes.

These are necessary requirements for your accessor to live within the rete/reasoner/rule parser environment. But in order to be actually useful, your accessor needs to provide an interface to the value in our WME. These interfaces are called Interpretation<T>, and your accessor may provide multiple interpretations of the same value -- e.g. an integer value could be interpreted as an int or a string, if you like. You can get a specific interpretation via AccessorBase::getInterpretation<T>, and use the interpretation to populate your value type:

auto interpretation = accessor->getInterpretation<std::string>();
// nullptr if the value cannot be interpreted as a string /
// the accessor has no such interpretation for it
if (interpretation)
{
    std::string value;
    interpretation->getValue(token, value);
    
    // do sth with value
    // ...
}

More details on the implementation of accessor, interpretations, and some utility classes for dealing with them can be found in rete/Implementation.

Thanks to these interfaces we can write builtins that work with strings or numbers and never need to know the WMEs they are accessing to get the values -- if ?a is inside a triple and ?b in some strange FooBarWME does not matter at all, the sum builtin only cares that it gets accessors that provide a Interpretation<float> and can then compute the sum of these values for you.

Note: Please have a look at rete-core/Accessors.hpp for more information.

The rete::Accessor<class WMEType, class... Interpretations> already provides a lot of glue code to integrate interpretations with you accessor. We derive our MutableWMEAccessor example from Accessor<MutableWME, std::string>, so that we only need to implement:

1. bool equals(const AccessorBase& other) const
2. void getValue(MutableWME::Ptr wme, std::string& value) const
3. MutableWMEAccessor* clone() const

The equals method is easy, since there is no variation, no configuration for the accessor, all of them are the same. In getValue we simply copy the value from MutableWME::value_ to value, and for the clone we create a new instance and just set the index:

/**
    An accessor for the MutableWME
*/
class MutableWMEAccessor : public StringAccessor {
    bool equals(const Accessor& other) const override
    {
        return nullptr != dynamic_cast<const MutableWMEAccessor*>(&other);
    }

    void getValue(MutableWME::Ptr wme, std::string& value) const override
    {
        value = wme->value_;
    }

    MutableWMEAccessor* clone() const override
    {
        auto acc = new MutableWMEAccessor();
        acc->index() = index_;
        return acc;
    }
};

For a little more complex example, please take a look at the implementation for triples.

Your own AlphaNode

Next we need a way to reasonably process your WME in the rete network. At bare minimum we need to sort incoming WMEs by type, so that we then can safely use the accessor to expose the internal values. What we need is an AlphaNode, and implement its interface:

1. bool operator == (const AlphaNode& other) const the rule parser wants to check nodes for equality, so that it can reuse existing nodes for new rules. The parser does check the external connections of the node itself, so you only need to check the internal configuration of the node (and of course the type).
2. void activate(WME::Ptr wme, PropagationFlag flag) this method get called whenever a new WME arrives, is retracted or updated. If it is retracted you simply need to pass on the message by calling propagate(wme, PropagationFlag::RETRACT). If it is asserted, you do your checks and only propagate PropagationFlag::ASSERT, only if your test passes. If it is updated you again do your check, but propagate PropagationFlag::UPDATE on success, and PropagationFlag::RETRACT on fail. The reason is that your node does not know if the WME has been asserted before, and must let the next AlphaMemory decide.
3. (optional) std::string getDOTAttr() const return a string to be used in as the attribute for the node representing this node in a dot-file.

Again, since our node will not implement any serious, configurable checks, all instances of it are basically the same. The logic in activate is simple, too, as we only do a dynamic cast to check the WMEs type (you could as well just check the type identifier string we introduced before).

So here is your first alpha node:

/**
    An alpha-node to get access to MutableWMEs -- actually:
    Just check *if* a WME is a MutableAlphaNode.
*/
class MutableAlphaNode : public AlphaNode {
    std::string getDOTAttr() const override
    {
        return "[label=\"MutableAlphaNode\"]";
    }

    void activate(WME::Ptr wme, PropagationFlag flag) override
    {
        if (flag == rete::PropagationFlag::RETRACT)
        {
            // shortcut: just propagate the retract
            propagate(wme, PropagationFlag::RETRACT);
        }
        else if (flag == rete::PropagationFlag::ASSERT)
        {
            // check the type, only propagate on match
            if (std::dynamic_pointer_cast<MutableWME>(wme))
            {
                propagate(wme, PropagationFlag::ASSERT);
            }
        }
        else if (flag == PropagationFlag::UPDATE)
        {
            // same check, but explicitely propagate RETRACT too,
            // and UPDATE instead of ASSERT
            if (std::dynamic_pointer_cast<MutableWME>(wme))
            {
                propagate(wme, PropagationFlag::UPDATE);
            }
            else
            {
                propagate(wme, PropagationFlag::RETRACT);
            }
        }
    }

    bool operator == (const AlphaNode& other) const override
    {
        return nullptr != dynamic_cast<const MutableAlphaNode*>(&other);
    }
};

Teach the RuleParser: The NodeBuilder

All that is left to do is to bring all the pieces together and tell the rule parser how to use them. This is the job of the NodeBuilder class: They build the bridge between a condition, builtin or effect in the string representation of a rule and the actual construction of the nodes. The NodeBuilder class

• takes two arguments in its constructor: A string that identifies the condition -- this is what you will need to write in your rules -- and the type of builder that it is:
  • BuilderType::ALPHA is what we need here, for alpha conditions.
  • BuilderType::BUILTIN constructs builtin nodes and
  • BuilderType::EFFECT is used to create effects of rules.
• has three methods from which you will need to override just one, depending on the type of the builder:
  • void buildAlpha(ArgumentList& args, std::vector<AlphaNode::Ptr>& nodes) const constructs a list of necessary nodes to handle a single precondition. E.g., the node builder for triple conditions may create multiple nodes to check subject, predicate and object for constant values, or for equality inside the triple. You do not need to connect these nodes, this will be done by the parser.
  • Builtin::Ptr buildBuiltin(ArgumentList& args) const constructs a single builtin node
  • Production::Ptr buildEffect(ArgumentList& args) const constructs a single effect, a production (this is actually not a node) which can be used in agenda nodes.

The ArgumentList is simply a std::vector<Argument>, where an Argument can be one of three things:

1. A constant value that is given in the rule definition. Can be checked with bool isConst() const. If it is, you can use ast::Argument& getAST() const to get the parsed expression, which you can check if it is a number or a string and get the value with
   • bool isNumber() const
   • bool isString() const
   • bool isURI() const
   • float toFloat() const
   • The ast::Argument itself is derived from string and holds exactly what was written in the rule.
2. A variable, which can be checked with bool isVariable(), in which case it can be either
   • unbound, in which case Accessor::Ptr getAccessor() const returns a nullptr, or
   • bound, in which case Accessor::Ptr getAccessor() const returns an accessor to the value that the variable was bound to.

Unbound variables can be bound by using void bind(Accessor::Ptr).

The node builder should check if the condition was used correctly, or a mistake happened in the rule, e.g.: Is the number of arguments correct? Is the correct argument unbound to store the result in? Are the bound variables accessors of the correct type, i.e., provide the necessary interpretations? If not, throw a NodeBuilderException.

If everything is correct, the node builder should do exactly that: Build [a] node[s] that implement[s] the desired functionality. Also, if the condition or builtin provides new information, e.g. a result of a mathematical operation, create an accessor object that matches the returned WME type and accesses the correct value. In our case we want to provide the value of the MutableWME, and we know that everything that passes our alpha node is indeed of that type. So we expect exactly one argument which must be an unbound variable and bind it to a MutableWMEAccessor.

Note: You could of course extend this to check internal values of the WME for certain criteria, just like the triple condition allows you to check for specific values in the fields, or for equality of internal values. You could e.g. expect an additional string argument construct an additional node that filters the WMEs based on that argument, and does some fancy, very specific things there.

A lot of description for quite a small class:

/**
    Last but not least: A node builder that enables us to use the node in a rule.
*/
class MutableNodeBuilder : public NodeBuilder {
public:
    MutableNodeBuilder() : NodeBuilder("MutableWME", BuilderType::ALPHA)
    {
    }
    
    void buildAlpha(ArgumentList& args, std::vector<AlphaNode::Ptr>& nodes) const override
    {
        if (args.size() == 1 && args[0].isVariable() && args[0].getAccessor() == nullptr)
        {
            // create a new MutableAlphaNode
            nodes.push_back(MutableAlphaNode::Ptr(new MutableAlphaNode()));

            // bind the variable to a matching accessor
            args[0].bind(MutableWMEAccessor::Ptr(new MutableWMEAccessor()));
        }
        else if (args.size() == 1 && args[0].isConst())
        {
            nodes.push_back(
                MutableAlphaNode::Ptr(
                    new MutableAlphaNode(
                        std::string(args[0].getAST())
                    )
                )
            );
        }
        else
        {
            throw NodeBuilderException("invalid use of MutableWME-condition");
        }
    }
};

Put it together

Now you can already use your own condition and data in the reasoner, or rather, the rule parser. All you need to do is register your node builder at the parser and you are ready to parse rules containing conditions regarding your data:

RuleParser p;
p.registerNodeBuilder<MutableNodeBuilder>();

Reasoner reasoner;

// remember to store the rules, else the network will be automatically deconstructed!
auto rules = p.parseRules(
    "[rule1: MutableWME(?value) -> (<foo> <mutable> ?value)]"
    // ...
    ,
    reasoner.net()
);

// add data as usual
auto wme = std::make_shared<MutableWME>();
wme->value_ = "Hello, World!";

auto ev = std::make_shared<AssertedEvidence>("fact-group-1");
reasoner.addEvidence(wme, ev);

// ...

reasoner.performInference();

// ...