Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema editor issues and keyword dictionary comparison #335

Open
braingram opened this issue Oct 16, 2024 · 0 comments
Open

Schema editor issues and keyword dictionary comparison #335

braingram opened this issue Oct 16, 2024 · 0 comments

Comments

@braingram
Copy link
Collaborator

There would be significant benefits to adding automated testing against the schemas and keyword dictionary.

Currently schema_editor.py is the tool for comparing the schemas and keyword dictionary but it:

  • fails to correctly parse the schemas (skipping entire branches if they're nested in an allOf)
  • fails to correctly parse the keyword dictionary (it makes different assumptions about the structure compared to the more common keyword_dict.py found in dads-common jwstsdp and jwst-schematic-headers)

Per conversation with @tapastro the useful comparisons are:

  • "data model path" (e.g. meta.observation.date) this is used for the keyword dictionary gui (to make filenames but otherwise not shown) and more importantly for the archive which uses this to check what files need reprocessing triggered by new reference files (since crds uses the "data model path")
  • fits keyword
  • fits hdu
  • title
  • description
  • enum (and perhaps more generally type)

In addition to the undocumented format of the keyword dictionary json the structure of the dictionary repository differs from the datamodels. Keywords in the dictionary are separated by instrument/model whereas datamodel schemas are not similarly formally separated. This complicates the comparison of some keywords (like FILTER which has different allowed values for each instrument/mode in the keyword dictionary but has an all-encompassing enum in the datamodel schema).

I propose that we compare the schemas and keyword dictionary using a tool that more closely matches the keyword_dict.py for parsing the keyword dictionary and uses the schema parsing built into stdatamodels for the datamodel schema parsing. The result would be a comparison tool that more closely matches the actual uses of these files (and is likely much simpler).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant