Design a rule assembly

[aj-stein-nist]

User Story:

As an OSCAL content creator, I need a model designed in OSCAL to define rules, detailed and machine-readable testing procedures for the evaluation of the information system.

NOTE: This is about the design of the internal structure of the rule model, not the final implemented model

Goals:

• Define the internal structure of one or more rules (as opposed to testing-scenarios and conditions and their framing structure

Dependencies:

• Framing context with testing-scenario and conditions (already covered in preparatory work in Support for "Rules" in OSCAL Models #1058)

Acceptance Criteria

• All OSCAL website and readme documentation affected by the changes in this issue have been updated. Changes to the OSCAL website can be made in the docs/content directory of your branch.
• A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

[aj-stein-nist]

Per Dave, try to align with MVP data shape approach from the work by Chris on current CRM design work (and update issue here accordingly, if needed; don't direct link to issue because it is similar approach but very different work).

[aj-stein-nist]

Current draft of the updated outline with the design can be found in this HackMD draft.

https://hackmd.io/5ntD6rJ-RhOX8GTSsREn1g

[aj-stein-nist]

Per the status update meeting, I will create a GitHub issue for a user story to track effort around updating a draft Metaschema model(s) and make content-based examples to solicit community feedback.

[aj-stein-nist]

Per discussion with Dave, we need to resurface the outline referenced in #1339 (comment) and then add a nice explanation here or in the document of how to read that.

Dave and I also agree we need to cover the "review current approaches for security testing processes and tools" goal before moving on to to #1364. So we will wrap that up early next week.

[david-waltermire]

Issue #1391 is follow-on work to this issue.

[david-waltermire]

@aj-stein-nist Please post the relevant notes from the hackmd document into this issue.

[aj-stein-nist]

@aj-stein-nist Please post the relevant notes from the hackmd document into this issue.

Without further ado, here is the updated design document. Reviews and feedback from all most welcome!

Background

As part of usnistgov/OSCAL#1339, the outline below represents the current design of the internal structure of an OSCAL rule, test, and the relationship between them.

How do I read this outline?

This outline is a streamlined representation of the OSCAL information for a rule, test and data structures associating them (testing-scenario and rule-implementation) in a component-definition.

The definitions in this outline are roughly organized in order of use. Some data elements are unspecified at this time and are marked with ??? to indicate where more work is needed.

Each data element includes its name, the cardinality, and data type. Some data elements also contain notes on their use.

Outline

rule

A rule is the specific trait or aspect that is the target of testing and evaluation (by a test).

• @uuid [1]
• title (markup-line) [1]
• description (markup-multiline) [1]
• link [0 to ∞]
• prop [0 to ∞] (NOTE: @name=version should be one of the core allowed-values)

test

A test is the manual, semi-automated, or automated mechanism to analyze, in full or in part, the trait or aspect meets the requirement of the rule.

• @uuid [1]
• title (markup-line) [0 or 1]
• description (markup-multiline) [0 or 1]
• link [0 to ∞]
• prop [0 to ∞] (NOTE: @name=version should be one of the core allowed-values)
• choice:
  • activity???: a manual thing, embedded checklist of steps a party with a responsible role would execute for a test outcome (assessment layer) (empty assembly object)
  • action???: an automated process: a script to run; a call to a service; structured inputs with execution (brave new world territory); a way to identify the "type of thing" (process) and meaningful identifer for "that thing that gets" run (e.g. if OVAL definition, this is an OVAL definition, here it is with ID XYZ1234); complex object details need to be modeled, how do we balance with the props? (empty assembly object)
• ???

(??? indicates there are additional yet to be determined fields we will add to this assembly in future. e.g. parameterization declarations for context to tools used in a test)

test-reference

Used in multiple contexts within this model where a reference to a test is needs to be declared.

• test-uuid
• ???

(??? indicates there are additional yet to be determined fields we will add to this assembly in future. e.g. parameterization for context to tools used in a test)

condition

A mechanism used in a testing-scenario to support logical evaluation of multiple tests for evaluating if the requirements in a rule are met. A choice between a condition, test-reference, and testing-scenario-uuid allows for simplicity in defining simple logic (i.e. test-reference), complex logic (i.e. condition), or reuse of an existing testing-scenario.

• pre-requisite [0 or 1] (use-name of condition)
• @negate (true/false) true -> false, false -> true, error -> error, not-applicable -> not-applicable
• operator (NOTE: default is and, allowed-values are and, or) [0 or 1]
• choice:
  • condition [1 to ∞]
  • test-reference (test-reference) [1 to ∞]
  • testing-scenario-uuid [1 to ∞] Note: the rule binding is ignored in this use case

condition examples

Interleaved conditions and tests like the following:

conditionA
test-reference1
conditionB
test-reference2

can be represented as:

conditionA:
condition: (anonymous wrapper)
  test-reference1:
conditionB:
condition: (anonymous wrapper)
  test-reference2:

Condition Evaluation

1. Does the pre-requisite execute to true? If false or error stop evaluation (result: false -> not-applicable, error -> error)
   • like eager and or failure in popular programming languages & versus &&).
2. Evaluate the child conditions or test-references applying the operator to determine the evaluation result (true, false, error, not-applicable)
   • operator:and can fail fast on first false
   • operator:or can succed fast on first true

testing-scenario

A construct that binds a rule to one or more tests. Dependencies and relationships between the sequence of tests are bound to the testing-scenario through conditions or by directly referencing a test using a test-reference. The latter supports simple cases where complex logic is not needed.

• @uuid
• @rule-uuid
• link [0 to ∞]
• prop [0 to ∞]
• choice:
  • condition [1]
  • test-reference (test-reference) [1]
• ???

(??? indicates there are additional yet to be determined fields we will add to this assembly in future. e.g. parameterization for context to tools used in reference tests).

rule-implementation

A rule-implementation is a means to bind a testing-scenario by reference to a component in a component definition, or a control implementation.

• @uuid [1]
• testing-scenario-uuid [1]
• description (markup-multiline) [1]
• link [0 to ∞]
• prop [0 to ∞]

Notes and follow-on questions

• Do we need to group rules? Ideally NO! There is plenty of opportunity to label rules with props of a given type or make the cohesive "test" of sorts running the collection of tests as defined in the tool.
• How do we handle the steps of an action (are pre-conditions in a testing-scenario enough or do we need linkage inter/intra action)? Use of actions and activities on a test may be a way forward here.

[aj-stein-nist]

Dave from our wrap-up on the call earlier today, just wanted to confirm I added props and links to the testing-scenario assembly in ac03cbf#diff-5b8de85c9d0b5761f2893eca11a33da6b0b4f0dca6c711f7091b43a3050ef359R142-R147.

[aj-stein-nist]

Update: in follow-up work, we should document how these different elements in the draft design above are referenced and linked into a component-definition, as follow through with this in the examples in the follow-up work in #1391.