Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate profile checker Schematron into CI/CD #128

Open
4 tasks
wendellpiez opened this issue Dec 12, 2019 · 7 comments · Fixed by usnistgov/OSCAL#1513
Open
4 tasks

Integrate profile checker Schematron into CI/CD #128

wendellpiez opened this issue Dec 12, 2019 · 7 comments · Fixed by usnistgov/OSCAL#1513
Labels
enhancement The issue adds a new feature, capability, or artifact to the repository. Scope: CI/CD A task issue to modify the repo's continuous integration and continuous deployment capability. User Story The issue is a user story for a development task.

Comments

@wendellpiez
Copy link
Contributor

wendellpiez commented Dec 12, 2019

User Story:

Responding to usnistgov/OSCAL#534 we implemented a simple profile link checker that reports when profiles call controls that can't be found in an imported catalog. It doesn't work on profiles importing profiles but it is better than nothing for detecting simple pointer errors.

The Schematron is in this branch: https://github.com/wendellpiez/OSCAL/tree/Issue534-profile-validation. The PR is usnistgov/OSCAL#539.

(Link to commit: wendellpiez/OSCAL@aa223f3#diff-ae6cae8616ab5f465e28e103376e036e )

Goals:

Help ensure profiles committed to the repo are correct.

This Schematron can also be the basis for other checks to be run on profiles.

Dependencies:

Acceptance Criteria

  • All OSCAL website and readme documentation affected by the changes in this issue have been updated. Changes to the OSCAL website can be made in the docs/content directory of your branch.
  • A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
  • The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

{The items above are general acceptance criteria for all User Stories. Please describe anything else that must be completed for this issue to be considered resolved.}

@wendellpiez wendellpiez added enhancement The issue adds a new feature, capability, or artifact to the repository. User Story The issue is a user story for a development task. labels Dec 12, 2019
@david-waltermire david-waltermire added the Scope: CI/CD A task issue to modify the repo's continuous integration and continuous deployment capability. label Apr 13, 2021
@wendellpiez
Copy link
Contributor Author

Revamping this Schematron in working branch behind usnistgov/OSCAL#970.

@wendellpiez
Copy link
Contributor Author

@wendellpiez
Copy link
Contributor Author

See PR usnistgov/OSCAL#1013 for updated Schematron.

It is located in the OSCAL repo here: src/utils/schematron/oscal-profile.sch

It does basic testing of profiles importing catalogs, confirming that controls and parameters that are addressed by the profile are actually present in the imported catalog.

@aj-stein-nist
Copy link
Contributor

aj-stein-nist commented Oct 17, 2022

Heads up to @david-waltermire-nist and @wendellpiez. I thought I would do a quick one to clear my head. Hit some minor speedbumps.

  1. Not urgent, will not block this work: we want to figure out the path forward for tech debt items on supporting XSLT 3 for any Schematron or Schematron that imports util XSLT functions from a stylesheet using XSLT3 features.
  2. More urgent, and I will file a bug: it seems the Saxon-executed use of a command-line invocated XPath essentially looking for any sad path Schematron result in the SVRL, even with intentionally crafted bad profiles do not bubble up an error. This code seems to evaluate a 0 even in the existence of a bad profile with a non-existent control-id inserted (one of the clear-cut assertions in the profile checker): https://github.com/usnistgov/metaschema/blob/675d385d65c85639d0e2d8e395cb80e4d66d63d3/scripts/include/init-schematron.sh#L60-L81 Will file a bug around this later in the evening.

aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Oct 20, 2022
aj-stein-nist added a commit to aj-stein-nist/oscal-content-forked that referenced this issue Oct 21, 2022
@aj-stein-nist aj-stein-nist linked a pull request Oct 21, 2022 that will close this issue
7 tasks
aj-stein-nist added a commit to aj-stein-nist/oscal-content-forked that referenced this issue Oct 21, 2022
aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Oct 21, 2022
aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Oct 21, 2022
Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Oct 21, 2022
Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/oscal-content-forked that referenced this issue Oct 21, 2022
aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Oct 31, 2022
Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
david-waltermire pushed a commit to usnistgov/OSCAL that referenced this issue Nov 1, 2022
…1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/oscal-content-forked that referenced this issue Nov 1, 2022
@aj-stein-nist
Copy link
Contributor

aj-stein-nist commented Nov 22, 2022

Heads up to @david-waltermire-nist and @wendellpiez. I thought I would do a quick one to clear my head. Hit some minor speedbumps.

1. Not urgent, will not block this work: we want to figure out the path forward for tech debt items on supporting XSLT 3 for any Schematron _or_ Schematron that imports util XSLT functions from a stylesheet using XSLT3 features.

2. More urgent, and I will file a bug: it seems the Saxon-executed use of a command-line invocated XPath essentially looking for any sad path Schematron result in the SVRL, even with intentionally crafted bad profiles do not bubble up an error. This code seems to evaluate a 0 even in the existence of a bad profile with a non-existent control-id inserted (one of the clear-cut assertions in the profile checker): https://github.com/usnistgov/metaschema/blob/675d385d65c85639d0e2d8e395cb80e4d66d63d3/scripts/include/init-schematron.sh#L60-L81 Will file a bug around this later in the evening.

Most of this content has been handled in the interim, but I updated to properly identify the blocker around CI/CD failures rooted in usnistgov/metaschema#240. There may be other issues, but I will need to update accordingly by end of week.

@aj-stein-nist
Copy link
Contributor

Moving to Sprint 61, and leaving open until the updated CI/CD code is merged into the oscal-content repo where it will actually validate profiles.

aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Jan 10, 2023
…snistgov#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
@aj-stein-nist
Copy link
Contributor

This can be merged with very limited effort once CI/CD configuration is resolved. Moving to Sprint 63 as it very close to complete.

aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Feb 6, 2023
…snistgov#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/OSCAL that referenced this issue Jun 29, 2023
…#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/OSCAL that referenced this issue Jun 29, 2023
…#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to aj-stein-nist/OSCAL-forked that referenced this issue Jul 10, 2023
…snistgov#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
aj-stein-nist added a commit to galtm/OSCAL that referenced this issue Sep 28, 2023
…snistgov#1513)

Profile checker Schematron queryBinding from xslt3->xslt2.

We need to do this in the interim or we will need to significantly change
or upgrade the CI/CD Schematron checker from current Schematron/schematron
skeleton to SchXslt or equivalent. The former is mothballed and there was
no addition of XSLT3 support. Potentially investigate further.

Team discussion: https://gitter.im/usnistgov-OSCAL/xslt-etc?at=634dc527dcbf067fdc9c47cd
Schematron issue: Schematron/schematron#20
SchXslt docs on XSLT 3.0 support: https://github.com/schxslt/schxslt/blob/62f1141613c9d4c02d8de86713bb3f3b29d6cf1c/README.md#additional-xslt-elements
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement The issue adds a new feature, capability, or artifact to the repository. Scope: CI/CD A task issue to modify the repo's continuous integration and continuous deployment capability. User Story The issue is a user story for a development task.
Projects
Status: Blocked
3 participants