Timestamps for metadata in profile resolution

[bradh]

The profile resolution spec for Instance metadata (3.4) includes:

<title>{ profile title} RESOLUTION</title>
<last-modified>{ resolution date stamp }</last-modified>
<prop name="catalog-title">{ imported catalog title }</prop>
<prop name="catalog-title">{ imported catalog title }</prop>

In the examples, we have <prop name="resolution-timestamp">2019-12-10T15:21:20.62-05:00</prop>, and use the profile last-modified as the last-modified in the resolved catalog.

Should last-modified really be the profile timestamp, or the resolution timestamp?

[brian-ruf]

@bradh, this is an important question to discuss. My personal opinion is the metadata last-modified field should reflect the date/time the resolved file was created.

The reason for this is even though this profile my not have changed the next time it is resolved, one of the up-stream catalogs or profiles may have.

The current intention for all OSCAL files is for tools to use the id flag on the root element to know if a file has changed, and the last-modified date/time field to know which is the more up-to-date. Requiring a tool to check a different field, to know which resolved profile is newer, breaks this otherwise standard mechanism.

This is just my opinion. I am interested in hearing what others think.

NOTE: Eventually NIST intends to build some form of cryptographic checksums into OSCAL, which will ensure file integrity, and provide a more reliable means of knowing a file has changed; however, the last-modified date/time will still be the primary means of knowing which is newer.

[bradh]

Generally concur on the last-modified - still waiting on additional review for #553 though, so that could move a bit (e.g. if my profile is just "all", would it make sense to use the last-modified from the source catalog instead, since nothing is really changed).

More generally, maybe there are two slightly different needs here:

1. the ability to uniquely identify the (resulting) catalog
2. the ability to work out how we got to this catalog

The first is important for referencing when the source controls change a lot (mine have about 8 versions this year), to address "are we working off the same version?" and "exactly what are you claiming compliance against?" type questions.

The second is more important for traceability, and assessing impacts when changes occur.

Perhaps we need to use the last-modified and id (and checksums, if and when) for the first case, and add more pedigree / lineage information about the inputs and processing time for the resolution process as a different part of the metadata.

[david-waltermire]

I don't think we need to copy around the titles if we preserve a link to the source.

How about the following:

<title>{ profile title}</title>
<last-modified>{ resolution date stamp }</last-modified>
<prop name="profile-last-modified" class="{profile-id}">{ resolved profile }</prop>

The same could be said for the profile's last-modified date, allowing this to be omitted in the example above.

[brian-ruf]

@david-waltermire-nist, I like your example as-is, and think we need to set a last-modified date on the resolved profile catalog that reflects when the resolved profile catalog was created.

The resolved profile represents a snapshot in time across all upstream profiles and catalogs used in its creation. Even though the immediate upstream profile may not change, it's sources farther upstream (additional profiles as well as catalogs) may change from one resolution to the next, thus the resulting resolved profile would be different. If we just propagate or link to the profile's last-modified date, we may incorrectly imply there has been no change.

[wendellpiez]

Currently, metadata/last-modified in the resolved catalog is copied from the the source profile. This is in the spirit of the semantics of last-modified if one respects the idea that it reflects the time of last (substantive) modification of a file's contents, which might (in the case of a baseline represented by an OSCAL profile), predate the creation of its representation in catalog format.

Additionally: the timestamp of profile resolution is added as a property (prop) to metadata:

<prop name="resolution-timestamp" value="2021-04-05T14:46:12.99-04:00"/>

If this is as wanted, @david-waltermire-nist this Issue is closable.

[bradh]

@wendellpiez I live in the world where profiles might change less often than the source content. The Australian ISM (https://www.cyber.gov.au/acsc/view-all-content/ism) is issued monthly. My profiles (e.g. "Essential 8") are much slower to evolve.

[wendellpiez]

We designed profiles with that use case in mind. In this case the timestamp of the resolved profile could indeed be earlier than the 'last-modified' date of a source catalog. Hm.

@david-waltermire-nist this points out to me there is actually a set of dates : the last-modified timestamp on the source profile and those on all its upstream sources including profiles and ultimate source catalog(s). As @bradh points out, the profile being resolved might not be later than all or any of its sources.

Options include:

1. Stamping last-modified as the time of resolution (and capturing source last-modified as metadata)
2. Copying the latest of available last-modified dates in all sources (and ditto?)
3. others?

Interested in opinions on this as it's somewhat crucial when archiving (caching) profile resolutions as catalogs.

[wendellpiez]

Discussing with @david-waltermire-nist we decided the simplest approach is to assign the time resolution is performed (creation of the resolution artifact) to last-modified, inasmuch as that is the "most correct" of the alternatives, plus also the simplest. The link to the source profile is retained for forensic analysis if wanted.

[bradh]

Concur. I could also live with "if the profile was (re-)resolved but nothing changed, keep the old date".

That would be valid (and might be useful for caching) in the case where my profile is extracting a simple subset of a large catalog, and some of the catalog changed in this month's update, but nothing I'm extracting has changed.

[wendellpiez]

That is an awesome idea but I suggest it should be a feature of a resolver, to notify if it finds no change in such a case. (After all, to detect no change entails identifying a copy for comparison.)

[bradh]

Concur. Just want to make sure it is spec-compliant to do so. It might be annoying for the last-modified to be mandated to change on every unique resolution, rather than being an idempotent operation.

[wendellpiez]

Yes. Also what the value is if no file is serialized. Etc.

From the point of view of resolution as a function (mapping a to b), adding a timestamp amounts to providing for an arbitrary value provided at runtime. The timestampiness of it is for us to make of and enforce however we can. Our guideline can say "you won't be lying if you provide the time when the resolution was invoked or its results are serialized" (not the same) but it can also leave it up to the resolver to provide a value that makes sense. (Lexical validity to the dateTime format being a given.)

In fact this discussion is making me think that the resolver spec should provide latitude for conformant behavior here. I can imagine ornamenting the metadata of serialized resolution output in several interesting ways, for example (perhaps capturing more of the upstream metadata), without wishing to require this for any resolver. Similarly for a value of last-modified we could require the basic ("a meaningful last-modified value must be stated") so as to discourage "0001-01-01T00:00:00Z" while leaving it up to the implementation to decide what the right thing is.

Interested in any input from @david-waltermire-nist here. BTW, is there a rule that says a "last-modified" date cannot be in the future?

[david-waltermire]

This was fixed in PR #882.