Remiges Products » Crux: BRE/WFE

*(For a conceptual overview and design of Crux, see [[cruxdesign|this page]] if you haven't already.)*

# Implementation of Remiges Crux

{{>toc}}

A rules engine implementation must include the following:

* **RULE SCHEMA**. A notation to specify the list of valid terms in a rule. This list will be separate for each class of entities. For instance, for items in inventory, the list of attributes may be:

  * Price

  * Full name

  * Age in stock

  * Quantity in inventory

    For vendors, the list could include:

  * Amount outstanding

  * Total value of business done  in the last financial year

* **RULE NOTATION**. A notation to specify the pattern and actions of a rule.

* **THE MATCHING ENGINE**. Something which will take an entity with all its attributes, apply each rule to it, and follow the trail of rules to come up with a list of actions which will emerge.

So, if these three can be designed and then implemented, the core of a rules engine or a flow engine can be built.

## Representing the schema of patterns

If using JSON, the schema of all valid patterns may be represented in structures of this form:

``` json

"patternschema": {

    "class": "inventoryitems",

    "attr": [{

        "name": "cat",

        "type": "enum",

        "vals": [ "textbook", "notebook", "stationery", "refbooks" ]

    },{

        "name": "mrp",

        "type": "float"

    },{

        "name": "fullname",

        "type": "str",

    },{

        "name": "ageinstock",

        "type": "int"

    },{

        "name": "inventoryqty",

        "type": "int"

    }]

}

```

In this example, the object `patternschema` is the schema for one category of entities. This schema says that for rules which work on entities of type `inventoryitems`, there are five attributes available which may be used to make patterns. Each attribute has a type. Enum types, integers, floating point numbers, timestamps (`ts`) and strings are supported. The example above does not have any attribute of type `ts`.

So, the full schema of the rules engine will be an array of `patternschema` blocks. Initial examples have discussed inventory items and vendors. The `patternschema` block above is for inventory items. If the schema of patterns for vendors needed to be specified, there would be a second `patternschema` with `“class”: “vendors”`

## Representing the schema of actions

The schema of the action section of rules is simpler than patterns. Each rule's action section will contain a set of zero or more words, each denoting an action, and zero or more attribute assignments. There is no need for any type specification, etc.

* An example of an action word: `invitefordiwali`

* An example of an attribute assignment: `discount=7`

So, the schema of the actions will just specify the valid action names and the attribute names for assignments.

``` json

"actionschema": {

    "class": "inventoryitems",

    "actions": [ "invitefordiwali", "allowretailsale", "assigntotrash" ],

    "attribs": [ "discount", "shipby" ],

    "tags": [ "specialvendor", "tryoverseas" ]

}

```

The schema of actions above indicates that there are three actions, any or all of which may be present in any rule for this class of entities. There are two attributes which may be assigned values by any rule. And there are two tags for this class of entities – if a rule wishes to tag an entity with one or both of these tags, it may do so.

Putting the `patternschema` and `actionschema` blocks together, a better representation for the full schema for a class of entities will be:

``` json

"ruleschema": {

    "class": "inventoryitems",

    "patternschema": {

        "attr": [{

            "name": "cat",

            "type": "enum",

            "vals": [ "textbook", "notebook", "stationery", "refbooks" ]

        },{

            "name": "mrp",

            "type": "float"

        },{

            "name": "fullname",

            "type": "str",

        },{

            "name": "ageinstock",

            "type": "int"

        },{

            "name": "inventoryqty",

            "type": "int"

        }]

    }

    "actionschema": {

        "actions": [ "invitefordiwali", "allowretailsale", "assigntotrash" ],

        "attribs": [ "discount", "shipby" ],

    }

}

```

There will need to be one such `ruleschema` block for each class.

## Representing a pattern

``` json

"rulepattern": {

    "pattern": [{

        "attr": "cat",

        "op": "eq",

        "val": "textbook"

    },{

        "attr": "mrp",

        "op": "ge",

        "val": 2000

    },{

        "attr": "ageinstock",

        "op": "ge",

        "val": 90

    }]

}

```

If a rule has this pattern, it will match any entity which falls in the class `inventoryitems` which

* is of type textbook

* has MRP (max retail price) greater than INR 2000

* has been in stock longer than 90 days 

For attributes which are of type `int`, `float`, `str` and `ts`, the following comparison operators are available:

* Greater than or equal to: `ge`

* Greater than: `gt`

* Less than or equal to: `le`

* Less than: `lt`

* Equal to: `eq`

* Not equal to: `ne`

Collation sequences for strings are system dependent, and will need to be standardised so that they work reliably across programming languages and Unicode strings in any language. That's an implementation issue.

For enum types, only `eq` and `ne` are available.

## Representing an action

A rule has a set of one or more actions. The following are all examples of the action section of rules:

* `invitefordiwali`

* `discount=7`

* `shipwithoutpo`

* `CALL=intlbiz`

The terms which identify actions, *e.g.* `invitefordiwali`, will automatically be converted to lower-case and stored in the system. Reserved attribute names like `CALL`, `RETURN`, `EXIT`, will always be in uppercase. For an attribute assignment, the value of the attribute will be everything after the first `=` character till the end of the string, thus supporting multi-word values, *e.g.*

* `reprimand=This cannot go on any longer`

The action portion of a rule can have zero or one occurrence of a `CALL` term, a `RETURN` term, and an `EXIT` term. If it contains both a `RETURN` and an `EXIT`, then the `RETURN` will be ignored.

The action portion of a rule will have the following structure, shown here as an example:

``` json

"ruleactions": {

    "actions": [ "christmassale", "vipsupport" ],

    "attribs": [ "shipby=fedex" ],

    "call": "internationalrules",

    "return": true,

    "exit": false

}

```

This example shows all five attributes of `ruleactions`, but in reality, some of the attributes will typically be missing from most of the rules.

## An entire rule

This is what an entire rule looks like:

``` json

"rule": {

    "class": "inventoryitems",

    "ver": 4,

    "rulepattern": [{

        "attr": "cat",

        "op": "eq",

        "val": "textbook"

    },{

        "attr": "mrp",

        "op": "ge",

        "val": 5000

    }],

    "ruleactions": {

        "actions": [ "christmassale" ],

        "attribs": [ "shipby=fedex" ]

    }

}

```

This structure represents one rule. The rule applies to entities of class `inventoryitems`. It has a pattern section which tries to match two attributes and an action section which throws up one action and one assignment.

A rule has a version number, which is incremented whenever the rule is updated. This number is for internal logging and rule engine debugging.

An array of such structures is a set of rules, and will be traversed in the order in which the rules appear in the array. Named rulesets will be represented thus:

``` json

"ruleset": {

    "class": "inventoryitems",

    "setname": "overseaspo",

    "rules": [{

        "ver": 4,

        "rulepattern": {

            :

            :

        },

        "ruleactions": {

            :

            :

        }

    }, {

        "ver": 3,

        "rulepattern": {

            :

            :

        },

        "ruleactions": {

            :

            :

        }

    }]

}

```

The example above shows a ruleset named `overseaspo` for class `inventoryitems` which has two rules. This ruleset may be invoked from any other rule with the action `CALL=overseaspo`.

## The schema manager

The schema for each class of entities may be written by hand using a text editor. JSON or YAML files are easy to write. It is unlikely that the schema of one class will have more than a dozen attributes, which makes the schema short enough to edit or audit by hand. However, a tool to manage and maintain the schema eliminates typos and enforces various types of consistency, and a second-level implementation of a schema manager may also enforce authorisation policies.

A schema manager will have the following features:

* It will allow the user to create new instances of `ruleschema`

* It will sharply restrict editing of, and prevent deletion of any `patternschema` block or `actionschema` block if there are rules defined in the rules engine for this class of entities. In other words, schema are editable only as long as there are no rules for the class. The only kind of editing it will permit for “live” schema are

  * the addition of additional attributes in a `patternschema` or

  * additional attributes, action names or tags in an `actionschema`.

* It will ensure that there is no scope for typos when defining the schema.

## The rule manager

The rule manager will allow a user to manage rules. Core functionality:

* It will provide a user interface to let the user edit rules.

* It will check each rule against the schema for the class, and will not give the user the opportunity to define any rule inconsistent with the schema.

* It will allow the user to move a rule up or down in the sequence, since ordering is important.

* If a rule is being defined with a `CALL` action, then the rule manager will ensure that a ruleset with that target name exists.

* Most important: it will provide a testing facility by which sample entities may be submitted to the rule engine for testing, and the rule manager will display a full trace showing which rules were attempted to match, which rules actually matched, and how the result set of actions, attributes, *etc* grew with each step. This feature will be provided without having to save the rule changes.

* Finally, when the editing session is complete and all rulesets need to be saved, it will perform a detailed cross-validation of all rules across each other to ensure consistency. If there is any inconsistency, it will give readable explanations of the problems and not permit saving of the updates.

## The matching engine

The matching engine has a one-line job. It will take a full set of attributes of one entity, apply all the rules which apply to its class, and return with the list of actions, attributes, *etc* from all the matching rules.

The operation of the engine is best understood if it is broken down into the units of its work.

### Matching one rule's pattern

The algorithm for the matching of one rule's pattern will be as shown below. Here, it is assumed that the object being matched is in `entity` and pattern of the rule being matched is in `rulepattern`.

```

func matchOnePattern()

    input parameters: entity, rulepattern

    returns patternmatch: boolean

for patternterm in rulepattern do

    for entityoneterm in entity.attrs do

        if entityoneterm.attr == patternterm.attr then

            entitytermval = entityoneterm.val

        endif

    endfor

    case patternterm.op in

    "eq":

        if entitytermval != patternterm.val then

            return false

        endif

    "ne":

        if entitytermval == patternterm.val then

            return false

        endif

    endcase

    if patternterm.type in [ "int", "float", "ts", "str" ] then

        case patternterm.op in

        "le":

            if entitytermval > patternterm.val then

                return false

            endif

        "lt":

            if entitytermval >= patternterm.val then

                return false

            endif

        "ge":

            if entitytermval < patternterm.val then

                return false

            endif

        "gt":

            if entitytermval <= patternterm.val then

                return false

            endif

        default:

            log error with priority = CRITICAL: "system inconsistency with BRE rule terms"

        endcase

    endif

endfor

return true

```

### Collecting the actions from one rule

If the pattern for one rule matches the entity being processed, then the actions of that rule will need to be added to the result set for that entity. Here we assume that the result of the action-collection function will return an object of the following structure. This object will be passed as input to the action-collecting function, and a (possibly extended) object will be returned, after merging the input object with the action terms from the rule just matched. The object structure will be:

``` json

"actionset": {

    "actions": [ "dodiscount", "yearendsale" ],

    "attribs": [ "shipby=fedex" ],

    "call": "overseaspo",

    "return": true,

    "exit": false

}

```

These five attributes will always be present in the object. The `actions` and `attribs` attributes will carry an array of strings, which will be a union set of all the action terms and attribute assignments collected from rules matched so far. The `call` attribute will either be a zero-length string or will carry the name of one ruleset to call after the current rule. The `return` and `exit` attributes will carry boolean values.

Performing a set union of action names is straightforward. Performing a set union of attribute assignments requires choosing one value of an attribute, if there was already the same attribute in the `actionset` and the current rule's actions also assigns a value to that attribute. In that case, the old value of the attribute will be overwritten by the new value.

```

function collectActions()

input parameters: actionset, ruleactions

    returns actionset

actionset.actions = actionset.actions UNION ruleactions.actions

actionset.attribs = actionset.attribs UNION ruleactions.attribs

actionset.call = ""

actionset.return = false

actionset.exit = false

if ruleactions.call is defined, then

    actionset.call = ruleactions.call

endif

if ruleactions.return is defined, then

    actionset.return = true

endif

if ruleactions.exit is defined,  then

    actionset.exit = true

endif

```

The matching engine needs to look at what has emerged from `collectActions()` and then take action. The flow of the matching engine will change based on the values of the `call`, `return` and `exit` attributes.

### The overall matching engine

This engine will go through rules one after another, and for each rule, it will call `matchOnePattern()` followed by `collectActions()` if the pattern matches. And then it will inspect the result obtained from `collectActions()` and decide what to do next.

This engine will be implemented by the `getRules()` function, which will be a recursive function. It will be called with two parameters:

* an entity, for which the rules need to be pulled out

* a `depth` parameter, which is an integer, always called with the value zero when called by the framework. Internally, the `getRules()` function will use this parameter to track how deep its recursive calls are at any particular point, to distinguish between `RETURN` and `EXIT`.

### API for the matching engine

The matching engine must support the following set of operations:

* `doMatch()`: take an entity, pass it through all relevant rules and rulesets, and respond with the set of final results.

* `getAttrSet()`: take a class name, pull out from the `patternschema` all the attributes listed against that class, with full details. This is useful to let the caller know what attributes are to be specified when calling `doMatch()`.