This is one of the possible Use Cases.
1. Abstract
The engineering of business rules, rule based business policies or rule based contracts is typically complicated and error-prone. In this use case, we show how certain principles that are part of agile software engineering, in particular test-driven software development, can be adapted so that they can be employed to model business rules. A business rule set is considered valid if it correctly solves all test cases, and there is evidence that it will correctly solve any further added test case. The main advantage of such a test-driven approach based on test cases is that the rules can then be validated automatically against an intended model, i.e. against user requirements, which help to safeguard the authoring process of rules. Test cases are not only useful to validate rules but also to verify rule sets, e.g. verifying the correctness of logic programs. In an iterative process, rules and test cases can be co-adapted to reach ideal rule specifications.
Furthermore, the test cases can be used in the further rule life cycle, i.e. during run-time of logic programs. For example, when a logic program is updated, test cases can be used to test whether the program is still valid after the update. Another very important application scenario consists of rules that are exchanged between different rule engines which are partially based on different semantics. Test cases can be used here to test whether a particular logic program (rule set) still has the same intended behaviour when it is executed within another rule engine with different semantics, e.g. rule engine 1 (well-founded), rule engine 2 (stable model). Various semantics such as Clark’s completion, three-valued logics (e.g. well-founded) or belief entailment logics (e.g. stable model or answer set) may or may not have a unique minimal model. Even when there is a unique model-theoretic meaning ascribed to a logic program, the meaning may still be different across the alternative implementation approaches, i.e. they can have different interpretations. Test cases can therefore safeguard the interchange of rule sets between different rule engines with different semantics and provide feedback on possible gaps and risks.
2. Status
Original version proposed by: AdrianPaschke (TU Munich). Current version joint effort by: NRC (HaroldBoley), Jens Dietrich (Massey University) and Adrian Paschke (TU Munich).
More information on test-driven validation of business rules can be found in an ISTA paper http://ibis.in.tum.de/staff/paschke/docs/ista2005-final.pdf.
We currently extend our test case approach with test coverage in an attempt to quantify the quality of the test case and automated test case generation, in analogy to tools like Clover in software engineering. We also integrate test cases into RuleML, where they can be regarded as special, fact-like, integrity constraints ('protecting' a 'content' rule base http://www.ruleml.org/0.9/glossary/#gloss-Protect), possibly carrying "semantic attributes".
3. Links to Related Use Cases
Rule Based Service Level Management and SLAs for Service Oriented Computing: In this use case there is a strong need to validate the rule-based SLA specifications and verify the correctness of contract rules. Typically, SLAs are not designed by rule experts with logic skills but by business practitioners. Predefined test cases can be used to safeguard this authoring process. Furthermore, the contracts are often scattered across organisations and business partners and need to be exchanged and executed in different environments, e.g. within different business rule management systems (BRMS). In a nutshell, SLAs provide one application domain for the test-driven validation approach, but test cases are useful in many other areas and scenarios as well, where rules are designed and engineered by different people and where rules are interchanged.
Information Integration with Rules and Taxonomies: In this use case general, rule-like, integrity constraints are employed. Techniques can thus be transferred between both of these use cases, e.g. RuleML sublanguages, serialization and semantic attributes.
4. Relationship to OWL/RDF Compatibility
As test cases and rules must be consistent, both should be stored and versioned together. In object-oriented software engineering, test cases and code are usually managed in the same source code repository and separated only when the software is deployed into the production environment. To achieve such a high level of integration between tests and rules, both should be serialized in a RIF, e.g. based on RDF stores to make rule bases persistent. Exchanged rules will have to be extended in order to accommodate test cases as well. The test cases are then used in the target system (e.g. rule engine) to validate the target rule engine and the correctness of the execution of the exchanged rules.
5. Examples of Rule Platforms Supporting this Use Case
We have implemented a first reference implementation of the test case approach in the open source backward-reasoning rule engine Mandarax -- see http://ibis.in.tum.de/staff/paschke/docs/ista2005-final.pdf.
We use the test case approach in the ContractLog KR implemented in the rule based Service Level Agreement (RBSLA) project -- see http://ibis.in.tum.de/staff/paschke/rbsla/index.htm -- to safeguard transactional updates (sequences of atomic updates), e.g. in the action part of reactive rules (ECA rules), and to exchange SLA rules in a distributed service oriented environment.
6. Benefits of Interchange
- Test cases can be interchanged together with rules and used to validate the interchanged rule set in the target system/environment.
- Newly received rules can be safely added to an existing rule set, if they pass all existing test cases.
- Test cases can be predefined by experts while rules are written or combined by “normal” practitioners. The test cases then safeguard this latter authoring process.
- Test cases are less abstract than the general purpose rules and therefore easier to comprehend for the end user.
7. Requirements on the RIF
- Test cases should be written in the same language as the rules or logic program that is to be tested.
Test cases as well as rule sets (and rule engines) should be annotated with meta information about their semantics (cf. "semantic attributes" in RuleML).
8. Breakdown
8.1. Actors and their Goals
- Rule designers and engineers write test case to verify and validate rules (akin to X-trem programming in software engineering)
- Business practitioners use test cases to validate the correctness of the logic program after updates
- During rule interchange, test cases are used to validate the correct execution in different target rule engines and logic systems with different semantics.
8.2. Main Sequence
The rule engineer writes a first rule set / logic program, e.g.
a <- not b b <- not a c <- a c <- b
and a test case { { { T: {?c=>true} }}}, which should be informally read as follows querying the rule engine with c should yield true. If the stable model semantics is applied, i.e. the underlying rule engine supports stable models, this test case succeeds. The program and test case are now exchanged to another rule engine, which supports well-founded semantics. Here the test case (query) leads to the answer "unknown" and accordingly the test case fails. Consequently, either the logic program must be extended here, e.g. asserting the fact "a" or "b", or -- if "unknown" is the intended answer -- the test case must be adapted to T2: {?c => unknown} .
8.3. Alternate Sequences
Test case, logic program (and rule engine) can be annotated with information about the semantics which should be applied, e.g. a test case T3: {stable_model: ?c => true; well_founded: ?c=>unknown} . When the test case is now applied on a logic program and rule engine that assume stable model (well-founded) semantics, the "?c=>true" ("?c=>unknown") case will be used.
9. Narratives
9.1. Interchange of rules
Rules and test cases are exchanged via a common RIF and executed in different target engines with different semantics. The test cases can be used to validate the correct behaviour in the target environment.
9.2. Updates of rule sets
Adding or retracting knowledge to or from an existing knowledge base might lead to inconsistencies and conflicts. Test case can be used to detect such inconsistencies or even to anticipate them.
Example:
p <- not p, q
Test case T4: {?p=>true}
If we now assert q to the knowledge base -- ‘’assert(q)’’ -- this update under stable model semantics leads to an inconsistency which can be (automatically) detected via the test case, so that this update can be reversed and the user alerted.
9.3. Transactional update of rule sets
New knowledge, e.g. new facts are added to an existing rule set. For example, a program:
a <- not b b <- not a
and an update: assert {a, b}
“a” and “b” should be asserted to the knowledge base in a transactional update, i.e. both should be added or non of them.
A test case T5: { ?a=>true, ?b=>true}
If only “a” or only “b” is asserted to the knowledge base, the test case fails. If both are added or none of them is, the test case succeeds, which is exactly the intended behaviour of the transactional update.
10. Commentary
Test cases have been successfully applied in software engineering. We have shown that the test case approach can be transferred to (business) rules. Test cases are particularly useful in a scenario where rules are authored by different people and for different rule engines, and where rules are exchanged via a common RIF and freely combined/merged/updated, interpreted with different semantics and executed in different target environments/engines.
A RIF should also comprise means to define test cases and store and exchange them together with the rules.