Merge pull request #1080 from spacetelescope/DR-user-guide-update

CCD-1508 Dr user guide update

CHANGES.rst

@@ -1,3 +1,14 @@
+11.18.3.dev0+g46fc854b.d20240903 (2024-10-02)
+=============================================
+
+General
+-------
+
+- client.api.get_default_context falls back to observatory as only argument to
+  maintain compatibility with crds_server < 13.0.0 (`#1078
+  <https://github.com/spacetelescope/crds/issues/1078>`_)
+
+
 12.0.0 (2024-09-25)
 ===================
 
@@ -8,9 +19,6 @@ Documentation
   <https://github.com/spacetelescope/crds/issues/1068>`_)
 
 
-12.0.0 (unreleased)
-====================
-
 General
 -------
 

changes/1080.doc.rst

@@ -0,0 +1 @@
+Doc updates for user guide to switch from Operational to Latest

changes/CCD-1508.doc.rst

@@ -0,0 +1 @@
+Updated various doc files and images pertaining to the switch from 'operational' to 'latest' in CRDS code.

crds/bestrefs/stale.py

@@ -56,7 +56,7 @@ class StaleByClassScript(cmdline.Script, cmdline.UniqueErrorsMixin):
 `stale` is the initial attempt at determining how current the calibration is for
 products in the JWST archive. For this report, "staleness" is defined as those
 dataset ids whose context is not current yet appear in affected dataset reports
-for all operational context changes between the exposure's context and the
+for all latest context changes between the exposure's context and the
 current context.
     """
 

documentation/crds_users_guide/source/basic_use.rst

@@ -109,7 +109,7 @@ care about:
 Overriding the Default Context
 ------------------------------
 
-It's possible to use past or future/experimental CRDS contexts rather than the pipeline's default operational context as follows:
+It's possible to use past or future/experimental CRDS contexts rather than the pipeline's default latest context as follows:
 
 .. tabs::
 

documentation/crds_users_guide/source/command_line_tools.rst

@@ -211,7 +211,7 @@ New Context
 
 crds.bestrefs always computes best references with respect to a context which
 can be explicitly specified with the `--new-context` parameter.  If `--new-context`
-is not specified, the default operational context is determined by consulting
+is not specified, the default latest context is determined by consulting
 the CRDS server or looking in the local cache.
 
 ...........
@@ -604,7 +604,7 @@ file against the file it replaces looking for new or missing table rows.
   -a, --dont-parse      Skip slow mapping parse based checks,  including mapping duplicate entry checking.
   -e, --exist           Certify reference files referred to by mappings exist.
   -p, --dump-provenance  Dump provenance keywords.
-  -x COMPARISON_CONTEXT, --comparison-context COMPARISON_CONTEXT   Pipeline context defining comparison files.  Defaults to operational context,  use 'none' to suppress.
+  -x COMPARISON_CONTEXT, --comparison-context COMPARISON_CONTEXT   Pipeline context defining comparison files.  Defaults to latest context,  use 'none' to suppress.
   -y COMPARISON_REFERENCE, --comparison-reference COMPARISON_REFERENCE  Comparison reference for tables certification.
   -s, --sync-files      Fetch any missing files needed for the requested difference from the CRDS server.
   -v, --verbose         Set log verbosity to True,  nominal debug level.
@@ -809,7 +809,7 @@ explained above:
   .. code-block:: bash
 
       $ crds list --cat jwst_nirspec_dark_0036.fits
-      CRDS - INFO - Symbolic context 'jwst-operational' resolves to 'jwst_0167.pmap'
+      CRDS - INFO - Symbolic context 'jwst-latest' resolves to 'jwst_0167.pmap'
       File:  '/grp/crds/jwst/references/jwst/jwst_nirspec_dark_0036.fits'
       {'A1_COL_C': '8.9600000e+002',
       'A1_CONF1': '2.1846000e+004',
@@ -870,7 +870,7 @@ and regressions can be printed or stored.
    .. code-block:: bash
 
        $ crds list --dataset-headers jcl403010 --first-id --minimize-header
-       CRDS - INFO - Symbolic context 'hst-operational' resolves to 'hst_0462.pmap'
+       CRDS - INFO - Symbolic context 'hst-latest' resolves to 'hst_0462.pmap'
        CRDS - INFO - Dataset pars for 'JCL403010:JCL403ECQ' with respect to 'hst_0462.pmap'
        {'APERTURE': 'WFC1',
         'ATODCORR': 'OMIT',
@@ -914,7 +914,7 @@ Sometimes it's desirable to know the individual exposures CRDS associates with a
   .. code-block:: bash
 
       $ crds list --dataset-headers jcl403010 --id-expansions-only
-      CRDS - INFO - Symbolic context 'hst-operational' resolves to 'hst_0462.pmap'
+      CRDS - INFO - Symbolic context 'hst-latest' resolves to 'hst_0462.pmap'
       JCL403010:JCL403ECQ
       JCL403010:JCL403EEQ
       JCL403010:JCL403EGQ
@@ -930,7 +930,7 @@ Sometimes it's desirable to know the individual exposures CRDS associates with a
 
   .. code-block:: bash
 
-      $ python m crds.list --operational-context
+      $ python m crds.list --latest-context
       jwst_0204.pmap 
 
 lists the context which has been *commanded* as default on the CRDS server.
@@ -974,7 +974,7 @@ contexts:
       jwst_0208.pmap
       jwst_0209.pmap
       
-      $ crds list --resolve-contexts  --contexts jwst-miri-dark-operational 
+      $ crds list --resolve-contexts  --contexts jwst-miri-dark-latest
       jwst_miri_dark_0012.rmap
       
       $ crds list --resolve-contexts --contexts jwst-niriss-superbias-2016-01-01T00:00:00

documentation/crds_users_guide/source/database.rst

@@ -44,7 +44,7 @@ or mapping file from the specified observatory:
              'rejected': 'false',
              'sha1sum': 'b684c9123da94ff5b9efb72c15316af050ecca62',
              'size': '42390720',
-             'state': 'operational',
+             'state': 'latest',
              'type': 'reference',
              'uploaded_as': '20151022_g2.0_wfc2-orampq_d_bia.fits',
              'useafter_date': '2015-10-22 23:30:54'}
@@ -105,7 +105,7 @@ or mapping file from the specified observatory:
              'rejected': 'false',
              'sha1sum': 'adc1e6cdf2491a393e439b8f820f29056b41e2ff',
              'size': '872417289',
-             'state': 'operational',
+             'state': 'latest',
              'type': 'reference',
              'uploaded_as': 'roman_dark_wfi01_wfi_image_highlatitudespecsurvey_vold.asdf',
              'useafter_date': '1900-01-01 00:00:00'}
@@ -143,12 +143,12 @@ If `fields` are specified as *None*,  info on all available fields is returned.
            {'lcb12060j_drk.fits': {
                'sha1sum': '56cfd1107bda5d82cb49a301a50edb45cb64ded6',
                'size': '10549440',
-               'state': 'operational'
+               'state': 'latest'
                },
             'n3o1022fj_drk.fits': {
                 'sha1sum': 'cecf11300015df8f39913b638138d8c67de77a02',
                 'size': '10526400',
-                'state': 'operational'
+                'state': 'latest'
                 }
            }
 
@@ -165,12 +165,12 @@ If `fields` are specified as *None*,  info on all available fields is returned.
            {'jwst_miri_dark_0072.fits': {
                'sha1sum': '34c432f1204618f46ed6591e0cb0f959981ba702',
                'size': '10549440',
-               'state': 'operational'
+               'state': 'latest'
                },
             'jwst_miri_dark_0073.fits': {
                 'sha1sum': 'ed6591e0cb0f959981ba70234c432f1204618f46',
                 'size': '10526400',
-                'state': 'operational'
+                'state': 'latest'
                 }
            }
 
@@ -187,12 +187,12 @@ If `fields` are specified as *None*,  info on all available fields is returned.
            {'roman_wfi_dark_0295.asdf': {
                'sha1sum': '2432e01bd0058a485d83e35e74d2701611a191f0',
                'size': '1006635000',
-               'state': 'operational'
+               'state': 'latest'
                },
             'roman_wfi_flat_0227.asdf': {
                 'sha1sum': '527f54ed8b8e53ff2e92425d506a23233deae044',
                 'size': '200542817',
-                'state': 'operational'
+                'state': 'latest'
                 }
             }
 
@@ -201,7 +201,7 @@ Download CRDS catalog for SQLite queries
 ----------------------------------------
 
 The CRDS catalog stores metadata about references not captured in the .rmap files. It also contains
-the history of CRDS context use, the effective dates at which particular contexts where operational in
+the history of CRDS context use, the effective dates at which particular contexts where the latest in
 the pipeline.
 
 You can download a SQLite-3 snapshot of the CRDS catalog like this:
@@ -236,9 +236,9 @@ SQLite3 database file.  The SQLite database can typically be accessed like this:
     
     sqlite> .mode tabs
     sqlite> .headers on
-    sqlite> select * from crds_hst_context_history where state="operational" limit 1;
+    sqlite> select * from crds_hst_context_history where state="latest" limit 1;
     id    name    start_date        context     state          description
-    2        2013-07-02 15:44:53    hst.pmap    operational    set by system
+    2        2013-07-02 15:44:53    hst.pmap    latest    set by system
     \.\.\.
     
 The CRDS catalog contains the following meta-data: 

documentation/crds_users_guide/source/environment.rst

@@ -255,9 +255,9 @@ CRDS Context
 
 The CRDS context file defines a version of CRDS rules used to assign best references.
 
-The CRDS context used to evaluate CRDS best references defaults to `{observatory}`-operational, e.g. `jwst-operational`.  This is an indirect name for the context in use or soon-to-be in use in the archive pipeline.
+The CRDS context used to evaluate CRDS best references defaults to `{observatory}`-latest, e.g. `jwst-latest`.  This is an indirect name for the context in use or soon-to-be in use in the archive pipeline.
 
-During development `-operational` corresponds to the latest context which is sufficiently mature for broad use and is automatic.
+During development `-latest` corresponds to the latest context which is sufficiently mature for broad use and is automatic.
 
 The context used can be overridden to some specific historical or experimental context by setting
 the **CRDS_CONTEXT** environment variable:
@@ -434,7 +434,7 @@ contains project specific sub-directories::
 
 - *references* contains reference files themselves
 
-- *config* contains system configuration information like operational context and bad files
+- *config* contains system configuration information like latest context and bad files
 
 Inidivdual branches of a cache can be overriden to locate that branch outside the directory
 tree specified by `CRDS_PATH`. The remaining directories can be overriden as well or derived

documentation/crds_users_guide/source/file_submissions.rst

@@ -246,19 +246,19 @@ Edit Context
 .pmaps are derived by default.
 
 In this way CRDS deliveries normally chain from one context to the next in a
-linear flow which can advance ahead of the *Operational Context* indefinitely
+linear flow which can advance ahead of the *Latest Context* indefinitely
 to support last minute testing prior to being used for real calibrations.  In
 almost all cases, eventually the current *Edit Context* is adopted for use in
-the archive pipeline and effectively becomes the *Operational Context*.
+the archive pipeline and effectively becomes the *Latest Context*.
 
-Operational Context
+Latest Context
 !!!!!!!!!!!!!!!!!!!
 
-*Operational Context* is the .pmap which is nominally in use by the pipeline.
-While it's common to make new files operational as each context is added, it's
-possible for the *Operational Context* to lag behind the *Edit Context* when
-new files are being added but need additional testing in OPS.   Deriving
-from the *Operational Context* is a crude kind of reversion since CRDS
+*Latest Context* is the .pmap which is nominally in use by the pipeline.
+While it's common to make new files the latest as each context is added, it's
+possible for the *Latest Context* to lag behind the *Edit Context* when
+new files are being added but need additional testing in OPS. Deriving
+from the *Latest Context* is a crude kind of reversion since CRDS
 effectively branches around any existing subsequent contexts.
 
 Recent
@@ -550,7 +550,7 @@ Manual .pmap update
 
 5. From here onward,  this should be a normal file submission,  with
 corresponding processes to archive the files,  Set Context the default
-OPERATIONAL context,  and sync the pipeline's CRDS cache.
+LATEST context,  and sync the pipeline's CRDS cache.
 
 6. Use Set Context to update the **EDIT context** to this .pmap
 as the default starting point for subsequent file submissions.
@@ -568,7 +568,7 @@ When Generate Contexts is OFF and a .pmap is manually updated, the Set Context
 page should generally be used to update the Editing Context so that future
 submissions will derive from the new .pmap by default.
 
-The Set Context page can be used to update either the Operational or Editing
+The Set Context page can be used to update either the Latest or Editing
 Context.  When updating the Editing Context, you may need to open the context
 selection accordion and type in the name of the new .pmap in User Specified.
 Verify that the correct .pmap is being set.
@@ -598,10 +598,10 @@ also supports reversing the decision and marking it good once more.
 The CRDS procedure for marking files bad requires three steps:
 
 1. Create a clean context which does not contain any prospective bad files.
-2. Make the clean context operational using Set Context.
+2. Make the clean context latest using Set Context.
 3. Mark the prospective bad files actually bad using Mark Bad Files.
 
-This procedure maintains the invariant that the operational pipeline context
+This procedure maintains the invariant that the latest pipeline context
 contains no known bad files.  The designation as bad files does not take effect
 until the pipeline CRDS cache is synchronized with the server.
 
@@ -671,7 +671,7 @@ files to CRDS.
 Add references can be used to undo the effects of Delete References in a
 perhaps distant descendant context containing other changes.  Add references
 can also be used to add tested references from a branched context into the
-current operational context.
+current latest context.
 
 Files are specified for Add References by listing their names in the Added
 Files field of the input form, separated by spaces, commas, and/or newlines.

documentation/crds_users_guide/source/library_use.rst

@@ -106,7 +106,7 @@ reference file paths:
                       
           context :   The name of the pipeline context mapping which should be
                   used to define best reference lookup rules,  or None.  If 
-                  `context` is None,  use the latest operational pipeline mapping.
+                  `context` is None,  use the latest pipeline mapping.
                   
                   str
                   
@@ -147,7 +147,7 @@ reference type:
         filling in appropriate reference type keywords.
     
         Define best references using either .pmap `context` or the default
-        CRDS operational context if context=None.
+        CRDS latest context if context=None.
     
         If `reftypes` is defined, assign bestrefs to only the listed
         reftypes, otherwise assign all reftypes.
@@ -181,7 +181,15 @@ crds.get_default_context()
 ..........................
 
 `get_default_context()` returns the name of the pipeline mapping which is 
-currently in operational use.
+currently the Latest Context
+!!!!!!!!!!!!!!!!!!!
+
+*Latest Context* is the .pmap which is nominally in use by the pipeline.
+While it's common to make new files the latest as each context is added, it's
+possible for the *Latest Context* to lag behind the *Edit Context* when
+new files are being added but need additional testing in OPS. Deriving
+from the *Latest Context* is a crude kind of reversion since CRDS
+effectively branches around any existing subsequent contexts. use.
 
 The default context defines the matching rules used to determine best 
 reference files for a given set of parameters:

documentation/crds_users_guide/source/overview.rst

@@ -125,7 +125,7 @@ A number of additional servers exist to support development and I&T for JWST and
 The CRDS web servers provide these functions:
 
 	1. Cataloging and display of information about CRDS files.
-	2. Tabular display of the current operational rules in the archive pipeline.
+	2. Tabular display of the latest rules in the archive pipeline.
 	3. Maintains and displays history of contexts used by the archive pipelines.
 	4. Supports the CRDS client library.
 	5. File submissions and archiving.

documentation/crds_users_guide/source/pipeline_management.rst

@@ -35,7 +35,7 @@ Set (Default CRDS) Context
 
 When an instrument team is satisfied that new references are working as
 intended within CRDS, they request a DMS operator to update the default /
-"operational" CRDS context by using the CRDS server *Set Context* page
+"latest" CRDS context by using the CRDS server *Set Context* page
 described below.  This step establishes the specified context as the default
 used by the pipeline as well as by other end-users who sync to the CRDS server
 and do not explicitly override it.
@@ -46,20 +46,20 @@ Set Context
 This section describes the website page used by operators to update the default
 CRDS context.
 
-*Set Context* enables setting the operational, edit, and versions contexts.  
+*Set Context* enables setting the latest, edit, and versions contexts.
 
 .. figure:: images/web_set_context.png
    :scale: 50 %
    :alt: set context inputs
 
 CRDS enables contexts to be distributed before their adoption as the pipeline
-default.  Set Context is used to select the default pipeline (operational)
+default.  Set Context is used to select the default pipeline (latest)
 context.
 
-Setting the operational context changes state on the CRDS server which must be
+Setting the latest context changes state on the CRDS server which must be
 subsequently sync'ed to remote pipelines and users.
 
-Setting the *operational* context creates a new entry at the top of the Context
+Setting the *latest* context creates a new entry at the top of the Context
 History.
 
 Setting the *edit* context sets the default starting point for future file

documentation/crds_users_guide/source/rmap_syntax.rst

@@ -514,7 +514,7 @@ new hook functions can only be added with a new release of CRDS code.   hook fun
 never be modified after use in operations since that would change the meaning of historical .rmaps.  Instead,  a new
 hook function should be added and the .rmap header modified to assign it.
 
-hook functions can be 'unplugged' in an operational .rmap by setting the value of the hook to 'none'.  Removing the
+hook functions can be 'unplugged' in the latest .rmap by setting the value of the hook to 'none'.  Removing the
 'hooks' section of the .rmap header, or removing individual hook names, currently results in reversion to <100 series
 .rmap behavior and the original implicit hook functions.