Establish a Continuous Integration (CI) Environment for the OSCAL Repo

[david-waltermire]

User Story:

As a OSCAL repo maintainer, I need an automated method to ensure timeliness, quality, and consistency in published artifacts.

Goals:

• Define high-level overview of CI/CD process and include this as part of a CONTRIBUTING.md or BUILD.md file
• Determine which items could be rolled up under automated testing
  • Validation of OSCAL-formatted XML against XSD/RNC
  • Validation of OSCAL-formatted XML against Schematron
  • Validation of OSCAL-formatted JSON against JSON schema
• Determine which items could be rolled up under automated artifact generation
  • Conversion of RNC to XSD via Trang
  • Conversion of non-OSCAL XML to OSCAL-formatted XML artifacts via XProc and Calabash
  • Conversion of OSCAL-formatted XML to JSON
• Determine who/what is responsible for automated artifact generation (e.g. CI/CD system vs. individual who executes automated tools locally and manually creates PRs with auto-generated artifacts)
• Select a CI/CD system (e.g. Travis CI, CircleCI, etc)

Dependencies:

Completion of issues #120 and #125.

Acceptance Criteria

• All XML and JSON OSCAL example Catalogs, Profiles, and simple examples are validated by the CI using the appropriate schema (XML Schema, JSON Schema) and other validation rules (Schematron).
• XML and JSON Schema files and documentation is generated automatically by the CI.

[david-waltermire]

We can automated unit tests around profile features as well (#121).

[anweiss]

Listing some possible tools to include as part of CI/CD. These can all be packaged in a Dockerfile which can be built and run as part of a Travis CI build.

Relax NG validation:

• Jing -> https://github.com/relaxng/jing-trang

XSD validation:

• xmllint (based on libxml2) (included with various Linux distros)

Schematron validation:

• https://github.com/NCAR/crux
• https://github.com/phax/ph-schematron

Schematron BDD testing:

• https://github.com/xspec/xspec

JSON schema validation (can also be used to validate schemas against the Core/Validation meta-schema):

• https://github.com/jessedc/ajv-cli
• https://github.com/json-schema-everywhere/pajv

JSON schema processing:

• https://github.com/cloudflare/json-schema-tools

@david-waltermire-nist Also, there already exists a Dockerfile in the root of the docs branch which contains the tool for automatically generating the static assets for pages.nist.gov/OSCAL. When this step is run in a Travis build against the docs branch, we can then use Travis's built-in Pages provider to publish a new commit to the nist-pages branch.

[anweiss]

@wendellpiez do you have any thoughts on the above listing, specifically for XSD validation? I'm currently using xmllint which is a wrapper around libxml 2.9.7, but if you know of a better CLI tool, feel free to add it.

[anweiss]

@david-waltermire-nist @wendellpiez here's a sample .travis.yml file I've been tinkering with -> https://github.com/anweiss/OSCAL/blob/ci/.travis.yml. You can see a sample build output here -> https://travis-ci.org/anweiss/OSCAL/builds/376097537. It bundles up xmllint with the ajv JSON schema validator in a simple Docker image and validates the FedRAMP and 800-53 samples against both the XSD and JSON schema in parallel. This could easily be expanded to execute XSpec tests and generate/publish docs to nist-pages

Note that the FedRAMP XML profiles are failing to validate against the oscal-profile-interim.xsd :)

[anweiss]

@david-waltermire-nist please assign @JJediny to this issue as well

[david-waltermire]

This is being worked as part of #342 as well. We can close both once PR #358 is accepted.

[david-waltermire]

We are using CircleCI to support the CI/CD pipeline. This issue predates the metaschema implementation. We are using the metaschema XSLTs to generate XML and JSON schema, XML <> JSON converters, and documentation. The converters are also used to transcode content between formats automatically. These capabilities have been integrated into the CI/CD process.

All that remains for this issue is to complete documentation for the CI/CD process, which I am currently working on in #374.

[iMichaela]

5/23/2019

See @david-waltermire-nist 's comment above.

[wendellpiez]

@david-waltermire-nist let's be sure there are no broken paths in docs generation; for example the conversion stylesheets have to be in the correct location for the docs generator to be able to convert the samples ... if things have moved this could require adjusting.

[david-waltermire]

PR #374 has been merged. Calling this issue complete.
@wendellpiez The CI/CD is already generating documentation for the pages.nist.gov site. All is working correctly so far.