Metaschema Doc Generator Marks Certain Assemblies Empty When They Aren't

[aj-stein-nist]

Describe the bug

During development in usnistgov/OSCAL#1160, we were running the generate-model-documentation.

Who is the bug affecting?

NIST OSCAL developers working on model updates.

What is affected by this bug?

Proper documentation generation for model reference docs.

When does this occur?

Consistently

How do we replicate the issue?

1. Check out snapshot backup of this branch on the relevant commit: aj-stein-nist/OSCAL@290b497 in the issue-1058-rules-metaschema-draft-backup-bug-report-metaschema-225 branch of my fork
2. Open docs directory cd path/to/OSCAL/docs
3. Check docker-compose container and dev environment is up to date: docker-compose down -v and docker-compose build
4. Run docker-compose run docs /oscal/build/ci-cd/generate-schema.sh -v
5. Run docker-compose run docs /oscal/build/ci-cd/generate-model-documentation.sh -v
6. Run local copy of web site hugo server --enableGitInfo=false --verbose --minify --bind 0.0.0.0

Expected behavior (i.e. solution)

Per usnistgov/OSCAL#1365, it was working before and the assemblies are properly filled out and not empty.

Other Comments

N/A

[aj-stein-nist]

We need to do further testing on this but there was only a slight model change so it is not clear what is happening but as of now on develop it seems to be that lots of assemblies are marked [Empty] that shouldn't be. It could be a model error but all of the recent changes pass with validate-metaschema.sh and other common testing/validation approaches we use as a team. Further investigation is needed.

[aj-stein-nist]

@wendellpiez, I brought this up in a weekly status update as a potential blocker, but not confirmed yet. @david-waltermire-nist asked Wendell and I sync up. I will move this along in the triage progression to "Further Analysis Needed" and determine if it is in fact a bug and prioritize it.

[aj-stein-nist]

Per the above commit message in the rules feature branch of OSCAL at usnistgov/OSCAL@6a134f0, this may be fixed. Can bring this up during the next triage meeting tomorrow or Tuesday of next week and determine a resolution state. 🤷

[aj-stein-nist]

We discussed this in our sprint planning. Dave pointed out it is not entirely resolved. At a11515b in my rules branch, it does still show [Empty] specifically when an assembly recursively points to itself, not as general as before, so part in catalog and condition in CDEF and SSP document instance assemblies (scattered across).

[aj-stein-nist]

I think I need to do more digging because it appears the recursive definition terminating with [Emtpy] approach has been around far longer than I noticed. I am not sure this case as relayed from Dave by me in #225 (comment) is a bug? Need to discuss more with the team.

• October 2021
• May 2022

[aj-stein-nist]

OK, @wendellpiez and I paired this afternoon, and I learned a lot will be able to commit a fix shortly. So, here is a quick write-up of [Empty], where it is valid, where it is not valid, how the bug occurs, and how we can fix (pending some local testing).

When is [Empty] valid in model docs?

It was not clear to me when something is "valid" when it is empty, but in correct OSCAL, there actually is use-case, such as in v1.0.3 current v1.0.4 in the OSCAL profile model is flat. In XML this field even if it acts like a flag and has no empty model. (I was struggling to find a valid example, there are probably others, but I can use this test pending changes).

When is [Empty] invalid in model docs?

The team appears to have cleaned up other edge cases, but as it stands for this bug, recursive model definitions (so in current catalog model: //control/part/part, the nested part definition is not empty, it is just recursive and the model is not repeated in the first nested example. A summary line (currently: (recursive: model like parent part) is left in the summary as a clue to the reader.

So how do we have an extraneous empty?

Why do we have an extraneous , invalid [Empty] in model docs?

We are not checking for the recursive flag on expanded debug Metaschema superset in the XSLT pipeline in triggered by the pipeline as currently defined generated by the model map builder transform and handle in the element map to HTML builder transform.

How do we fix it?

An additional template with higher priority that checks for the recursive attribute on the debug superset element and generates only <div/> with the summary of that element status (that's it's recursive), short-circuit and do not write an expandable/collapsible <details/> block.

PR forthcoming!

[aj-stein-nist]

Wendell and I did some further asynchronous debugging and he recommended a more optimal fix where we tighten an existing template in the element map to HTML builder. This original template match of m:element[empty(* except (m:attribute|m:value))] had actually shifted in how correct it was, separate of the analysis above in #225 (comment) because of the additions of formal-name and description of Metaschema definitions are interpreted through the pipeline.

[aj-stein-nist]

Per discussion in status meeting, we intend for this to bubble from develop to main for document generation, but that requires merging all of develop to main and that is not agreeable or desirable. The preferred way forward, per Dave, is usnistgov/OSCAL#1463. Pointing from here to that issue for continuity.