NXxpcs.nxdl.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="nxdlformat.xsl" ?>
<!--
# NeXus - Neutron and X-ray Common Data Format
#
# Copyright (C) 2008-2021 NeXus International Advisory Committee (NIAC)
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 3 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
# For further information, see http://www.nexusformat.org
-->
<definition name="NXxpcs" extends="NXobject" type="group"
  category="application"
  xmlns="http://definition.nexusformat.org/nxdl/3.1"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"
  >
  <symbols>
    <doc>
      The symbol(s) listed here will be used below to coordinate datasets with the same shape.
    </doc>
    <symbol name="nP">
      <doc>Number of points</doc>
    </symbol>
  </symbols>
  <doc>
    X-ray Photon Correlation Spectroscopy (XPCS) data (results).

    The purpose of ``NXxpcs`` is to document and communicate an accepted vernacular for various XPCS results data
    in order to support development of community software tools.  The definition presented here only
    represents a starting point and contains fields that a common software tool should support for
    community acceptance.

    Additional fields may be added to XPCS results file (either formally or informally).  It is expected
    that this XPCS data will be part of multi-modal data set that could involve e.g., :ref:`NXcanSAS` or
    1D and/or 2D data.
  </doc>
  <group type="NXentry" name="entry">
    <field name="definition">
      <doc> Official NeXus NXDL schema to which this file conforms </doc>
      <enumeration>
        <item value="NXxpcs"/>
      </enumeration>
    </field>

    <field name="entry_identifier">
      <doc>
        Unique identifier for the experiment.
      </doc>
    </field>
    <field name="entry_identifier_uuid" minOccurs="0" maxOccurs="1">
      <doc>
        (optional) UUID identifier for this entry.
      </doc>
    </field>
    <field name="scan_number" type="NX_INT">
      <doc>
        Scan number (must be an integer).

        NOTE: Link to collection_identifier.
      </doc>
    </field>
    <field name="start_time" type="NX_DATE_TIME">
      <doc>
        Starting time of experiment, such as "2021-02-11 11:22:33.445566Z".
      </doc>
    </field>
    <field name="end_time" type="NX_DATE_TIME" minOccurs="0" maxOccurs="1">
      <doc>
        Ending time of experiment, such as "2021-02-11 11:23:45Z".
      </doc>
    </field>

    <group type="NXdata" name="data">
      <doc>
        The results data captured here are most commonly required for high throughput, equilibrium dynamics experiments. Data (results)
        describing on-equilibrium dynamics consume more memory resources so these data are separated.
      </doc>
      <field name="frame_sum" type="NX_NUMBER" units="NX_COUNT" minOccurs="0" maxOccurs="1">
        <doc>
          Two-dimensional summation along the frames stack.

          sum of intensity v. time (in the units of "frames")
        </doc>
      </field>
      <field name="frame_average" type="NX_NUMBER" units="NX_COUNT" minOccurs="0" maxOccurs="1">
        <doc>
            Two-dimensional average along the frames stack.

            average intensity v. time (in the units of "frames")
        </doc>
      </field>
      <field name="g2" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
            normalized intensity auto-correlation function, see Lumma, Rev. Sci. Instr. (2000), Eq 1

            ..  math:: g_2(\boldsymbol Q,t) = \frac{ \langle I(\boldsymbol Q,t\prime) I(\boldsymbol Q,t\prime + t) \rangle }{ \langle I(\boldsymbol Q,t\prime)\rangle^2 }; t > 0

            Typically, :math:`g_2` is a quantity calculated for a group of pixels representing a specific
            region of reciprocal space.  These groupings, or bins, are generically described as :math:`q`. Some
            open-source XPCS libraries refer to these bins as "rois", which are not to be confused with
            EPICS AreaDetector ROI. See usage guidelines for q_lists and roi_maps within a mask.  [#]_

            In short, :math:`g_2` should be ordered according to the roi_map value.  In principle, any format is acceptable if
            the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one
            of the following two formats:

            * iterable list of linked files (or keys) for each :math:`g_2` with 1 file (key) per :math:`q`, where `q` is called by the nth roi_map value
            * 2D array [#]_ with shape (:math:`g_2`, :math:`q`), where `q` is represented by the nth roi_map value, not the value `q` value

            Note it is expected that "g2" and all quantities following it will be of the same length.

            Other formats are acceptable with sufficient axes description.

            See references below for related implementation information:

            .. [#] mask: ``NXxpcs:/entry/instrument/masks-group``
            .. [#] NeXus 2-D data and axes: https://manual.nexusformat.org/classes/base_classes/NXdata.html#nxdata
        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <doc>
            storage_mode describes the format of the data to be loaded

            We encourage the documentation of other formats not represented here.

            * one array representing entire data set ("one_array")
            * data exchange format with each key representing one ``q`` by its corresponding roi_map value ("data_exchange_keys")
          </doc>
          <enumeration>
            <item value="one_array"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
      </field>
      <field name="g2_derr" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
            error values for the :math:`g_2` values.

            The derivation of the error is left up to the implemented code. Symmetric error will be
            expected (:math:`\pm` error).  The data should be in the same format as ``g2``.
        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
      </field>
      <field name="G2_unnormalized" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
            unnormalized intensity auto-correlation function.

            Specifically, ``g2`` without the denominator.  The data should be in the same format as ``g2``.

        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
      </field>
      <field name="delay_difference" type="NX_INT" units="NX_INT" minOccurs="0" maxOccurs="1">
        <doc>
          delay_difference (also known as delay or lag step)

          This is quantized difference so that the "step" between two consecutive
          frames is one frame (or step ``= dt = 1 frame``)

          It is the "quantized" delay time corresponding to the ``g2`` values.

          The unit of delay_differences is ``NX_INT`` for units of frames (i.e., integers) preferred,
          refer to :ref:`NXdetector` for conversion to time units.
        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
      </field>
    </group>

    <group type="NXdata" name="twotime" minOccurs="0">
    <doc>
      The data (results) in this section are based on the two-time intensity correlation function derived from a time series of scattering images.
    </doc>
      <field name="two_time_corr_func" type="NX_NUMBER" units="NX_ANY" minOccurs="0" maxOccurs="1">
        <doc>
          two-time correlation of speckle intensity for a given q-bin or roi (represented by the nth roi_map value)

          See Fluerasu, Phys Rev E (2007), Eq 1 and Sutton, Optics Express (2003) for an early
          description applied to X-ray scattering:

          .. math:: C(\boldsymbol Q, t_1, t_2) = \frac{ \langle I(\boldsymbol Q, t_1)I(\boldsymbol Q, t_2)\rangle }{ \langle I(\boldsymbol Q,t_1)\rangle \langle I(\boldsymbol Q,t_2)\rangle }

          in which time is quantized by frames. In principle, any data format is acceptable if
          the data and its axes are self-describing as per NeXus recommendations. However, the data is preferred in one
          of the following two formats:

          * iterable list of linked files (or keys) for each q-bin called by the nth roi_map value. data for each bin is a 2D array
          * 3D array with shape (frames, frames, q) or (q, frames, frames), where :math:`q` is represented by the nth roi_map value, not the value `q` value


          The computation of this result can be customized.  These customizations can affect subsequently derived results (below).  The
          following attributes will be used to manage the customization.

          * Other normalization methods may be applied, but the method will not be specified in this
            definition. Some of these normalization methods result in a baseline value of ``0``, not ``1``.

          * The various software libraries use different programming languages.  Therefore, we need to
            specify the ``time = 0`` origin location of the 2D array for each :math:`q`.

          * A method to reduce data storage needs is to only record half of the 2D array by populating
            array elements above or below the array diagonal.


        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <doc>
            storage_mode describes the format of the data to be loaded

            We encourage the documention of other formats represented here.
          </doc>
          <enumeration>
            <item value="one_array_q_first"/>
            <item value="one_array_q_last"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
        <attribute name="baseline_reference" type="NX_INT">
          <doc>
            baseline is the expected value of a full decorrelation

            The baseline is a constant value added to the functional form of the auto-correlation
            function. This value is required.
          </doc>
          <enumeration>
            <item value="0"/>
            <item value="1"/>
          </enumeration>
        </attribute>
        <attribute name="time_origin_location" type="NX_CHAR">
          <doc>
            time_origin_location is the location of the origin
          </doc>
          <enumeration>
            <item value="upper_left"/>
            <item value="lower_left"/>
          </enumeration>
        </attribute>
        <attribute name="populated_elements" type="NX_CHAR">
          <doc>
            populated_elements describe the elements of the 2D array that are populated with data
          </doc>
          <enumeration>
            <item value="all"/>
            <item value="upper_half"/>
            <item value="lower_half"/>
          </enumeration>
        </attribute>
      </field>
      <field name="g2_from_two_time_corr_func" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
          frame weighted average along the diagonal direction in ``two_time_corr_func``

          The data format and description should be consistent with that found in "/NXxpcs/entry/data/g2"

          * iterable list of linked files for each :math:`g_2` with 1 file per :math:`q`
          * 2D array with shape (:math:`g_2`, :math:`q`)

          Note that delay_difference is not included here because it is derived from the shape of
          extracted :math:`g_2` because all frames are considered, which is not necessarily the case for :math:`g_2`.

          The computation of this result can be customized.  The customization can affect the fitting required to extract quantitative results.  The
          following attributes will be used to manage the customization.

        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array_q_first"/>
            <item value="one_array_q_last"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
        <attribute name="baseline_reference" type="NX_INT">
          <enumeration>
          <item value="0"/>
          <item value="1"/>
        </enumeration>
        </attribute>
        <attribute name="first_point_for_fit" type="NX_INT">
          <doc>
            first_point_for_fit describes if the first point should or should not be used in fitting the functional form of the dynamics to extract quantitative time-scale information.

            The first_point_for_fit is True ("1") or False ("0"). This value is required.
          </doc>
          <enumeration>
          <item value="0"/>
          <item value="1"/>
        </enumeration>
        </attribute>
      </field>
      <field name="g2_err_from_two_time_corr_func" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
            error values for the :math:`g_2` values.

            The derivation of the error is left up to the implemented code. Symmetric error will be
            expected (:math:`\pm` error).
        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array_q_first"/>
            <item value="one_array_q_last"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
      </field>
      <field name="g2_from_two_time_corr_func_partials" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
          subset of frame weighted average along the diagonal direction in ``two_time_corr_func``

          Time slicing along the diagonal can be very sophisticated.  This entry currently assumes
          equal frame-binning. The data formats are highly dependent on the implantation of various analysis libraries.
          In principle, any data format is acceptable if the data and its axes are self describing as per NeXus
          recommendations. However, the data is preferred in one of the following two formats:

          * iterable list of linked files (or keys) for each partial :math:`g_2` of each q-bin represented by the roi_map value
          * 3D array with shape (:math:`g_2`, :math:`q`, nth_partial)

          Note that delay_difference is not included here because it is derived from the shape of
          extracted :math:`g_2`.

        </doc>
        <attribute name="storage_mode" type="NX_CHAR">
          <enumeration>
            <item value="one_array"/>
            <item value="data_exchange_keys"/>
            <item value="other"/>
          </enumeration>
        </attribute>
        <attribute name="baseline_reference" type="NX_INT">
          <enumeration>
          <item value="0"/>
          <item value="1"/>
        </enumeration>
        </attribute>
      </field>
      <field name="g2_err_from_two_time_corr_func_partials" type="NX_NUMBER" units="NX_ARBITRARY_UNITS" minOccurs="0" maxOccurs="1">
        <doc>
            error values for the :math:`g_2` values.

            The derivation of the error is left up to the implemented code. Symmetric error will be
            expected (:math:`\pm` error).
        </doc>
      </field>
    </group>

    <group type="NXinstrument" name="instrument">
      <doc>
          XPCS instrument Metadata.

          Objects can be entered here directly or linked from other
          objects in the NeXus file (such as within ``/entry/instrument``).
      </doc>
      <group type="NXbeam" name="incident_beam">
        <field name="incident_energy" type="NX_FLOAT" units="NX_ENERGY">
          <doc>Incident beam line energy (either keV or eV).</doc>
        </field>
        <field name="incident_energy_spread" type="NX_FLOAT" units="NX_ENERGY" minOccurs="0" maxOccurs="1">
          <doc>
            Spread of incident beam line energy (either keV or eV). This quantity is otherwise known
            as the energy resolution, which is related to the longitudinal coherence length.
          </doc>
        </field>
        <field name="incident_polarization_type" minOccurs="0" maxOccurs="1">
          <doc>
            Terse description of the incident beam polarization.

            The value can be plain text, such as ``vertical``, ``C+``,
            ``circular left``.
          </doc>
        </field>
        <field name="extent" type="NX_FLOAT" units="NX_LENGTH" minOccurs="0" maxOccurs="1">
          <doc>Size (2-D) of the beam at this position.</doc>
          <!-- FIXME: (h, v) or (v, h)?  State this in the docs FOR EPICS AD, likeky v, h.  But seems CSX is (h,v) if looking
          from the source's perspective at the face of the detector - see fig 11 and fig 12 of cxidb documentation.  this
          is also relevant for the next section, which is just describing the 2D array V, H  is python/bluesky-->
        </field>
      </group>

      <group type="NXdetector" name="detector">
        <doc>
          XPCS data is typically produced by area detector (likely EPICS AreaDetector) as a stack of 2D images. Sometimes
          this data is represented in different ways (sparse arrays or photon event list), but this detail
          is left to the analysis software.  Therefore, we only include requirements based on full array data.

          We note that the image origin (pixel coordinates (0,0)) are found at the top left of a single 2D image array. This
          is the standard expected by Coherent X-ray Imaging Data Bank. [#]_
          See CXI version 1.6 and Maia, Nature Methods (2012).  This seems to be consistent with matplotlib and
          the practiced implementation of EPICS AreaDetector.  However, some exceptions may exists in the CXI
          documentation (See Fig 11 vs Fig 12).

          Additionally, not all NXdetector dependencies are inherited from AreaDetector or other control systems. ``frame_time`` is used to
          convert ``delay_difference`` to seconds.  ``frame_time`` field could be missing from AreaDetector or may
          either be `acquire_period` or `acquire_time`, depending on the detector model and the local implementation.

          .. [#] Coherent X-ray Imaging Data Bank: https://cxidb.org/cxi.html
        </doc>
        <field name="description" minOccurs="0" maxOccurs="1">
          <doc>Detector name.</doc>
        </field>
        <field name="distance" type="NX_NUMBER" units="NX_LENGTH" minOccurs="0" maxOccurs="1">
          <doc>Distance between sample and detector.</doc>
        </field>
        <field name="count_time" type="NX_NUMBER" units="NX_TIME">
          <doc>Exposure time of frames, s.</doc>
        </field>
        <field name="frame_time" type="NX_NUMBER" units="NX_TIME">
          <doc>
            Exposure period (time between frame starts) of frames, s
          </doc>
        </field>
        <field name="beam_center_x" type="NX_NUMBER" units="NX_LENGTH">
          <doc>
            Position of beam center, x axis, in detector's coordinates.
          </doc>
        </field>
        <field name="beam_center_y" type="NX_NUMBER" units="NX_LENGTH">
          <doc>
            Position of beam center, y axis, in detector's coordinates.
          </doc>
        </field>
        <field name="x_pixel_size" type="NX_NUMBER" units="NX_LENGTH" minOccurs="0" maxOccurs="1">
          <!--
            made this optional in case of single photon xy-time lists
          -->
          <doc>
            Length of pixel in x direction.
          </doc>
        </field>
        <field name="y_pixel_size" type="NX_NUMBER" units="NX_LENGTH" minOccurs="0" maxOccurs="1">
          <doc>
            Length of pixel in y direction.
          </doc>
        </field>
      </group>

      <group type="NXnote" name="masks" minOccurs="0" maxOccurs="1">
          <doc>
              Data masks or mappings to regions of interest (roi) for specific :math:`Q` values

              Fields in this ``masks`` group describe regions of interest
              in the data by either a mask to select pixels or to associate
              a *map* of rois with a (one-dimensional) *list* of values.

              "roi_maps" provide for representation of pixel binning that are arbitrary and irregular,
              which is geometry scattering agnostic and most flexible. The maps work as a labeled array for N rois.

              "Dynamic" represents quantities directly related to XPCS and NXxcps/entry/data and
              NXxpcs/entry/two_time.

              "Static" refers to finer binning used for computation not strictly used for the final
              XPCS results. Implementation of _static_ binning is left for individual libraries to
              document.  We encourage usage of :ref:`NXcanSAS` to represent standard SAXS results or
              development of new NeXus definitions for GI-SAXS or other reciprocal space
              intensity mapping.
          </doc>
          <field name="dynamic_roi_map" type="NX_DIMENSIONLESS">
            <doc>
            roi index array or labeled array

            The values of this mask index (or map to) the :math:`Q` value from the
            the ``dynamic_q_list`` field. Not that the value of ``0`` represents in-action. XPCS computations
            are performed on all pixels with a value > 0.

            The ``units`` attribute should be set to ``"au"``
            indicating arbitrary units.
            </doc>
          </field>
          <field name="dynamic_q_list" type="NX_NUMBER" units="NX_PER_LENGTH" minOccurs="0">
            <doc>
            1-D list of :math:`Q` values, one for each roi index value.

            List order is determined by the index value of the associated roi map starting at ``1``.

            The only requirement for the list is that it may be iterable. Some expected formats are:

            * iterable list of floats (i.e., :math:`Q(r)`)
            * iterable list of tuples (i.e., :math:`Q(r)`, :math:`\varphi`), but preferable use the seperate :math:`\varphi` field below
            * iterable list of tuples (e.g., (H, K, L); (qx, qy, qz); (horizontal_pixel, vertical_pixel))
            * iterable list of integers (for Nth roi_map value) or strings

            This format is chosen because results plotting packages are not common and simple I/O is required by end user.
            The lists can be accessed as lists, arrays or via keys
            </doc>
          </field>
          <field name="dynamic_phi_list" type="NX_NUMBER" units="NX_PER_LENGTH" minOccurs="0">
            <doc>
            Array of :math:`\varphi` value for each pixel.

            List order is determined by the index value of the associated roi map starting at ``1``.
            </doc>
          </field>
          <field name="static_roi_map" type="NX_DIMENSIONLESS" minOccurs="0">
            <doc>
            roi index array.

            The values of this mask index the :math:`|Q|` value from the
            the ``static_q_list`` field.

            The ``units`` attribute should be set to ``"au"``
            indicating arbitrary units.
            </doc>
          </field>
          <field name="static_q_list" type="NX_NUMBER" units="NX_PER_LENGTH" minOccurs="0">
            <doc>
            1-D list of :math:`|Q|` values, 1 for each roi.
            </doc>
          </field>
      </group>
    </group>

    <group type="NXsample" name="sample" minOccurs="0">
    <!--
      Describes the minimum requirements regarding equilibrium sample conditions. NXsample
      permits other quantities (e.g., applied fields, crystallographic information, name, etc) that
      may optionally be include for equilibrium conditions (which is not exclusively equilibrium
      dynamics from XPCS analysis).

      For non-equilibrium sample conditions (i.e., changing sample or process conditions
      during the XPCS measurement) will require either a new entry or an additional atttribute.
      -->
      <field name="temperature_set" type="NX_NUMBER" units="NX_TEMPERATURE" minOccurs="0">
        <doc>
          Sample temperature setpoint, (C or K).
        </doc>
      </field>
      <field name="temperature" type="NX_NUMBER" units="NX_TEMPERATURE" minOccurs="0">
        <doc>
          Sample temperature actual, (C or K).
        </doc>
      </field>
      <group type="NXpositioner" name="position_x" minOccurs="0" />
      <group type="NXpositioner" name="position_y" minOccurs="0" />
      <group type="NXpositioner" name="position_z" minOccurs="0" />
    </group>

    <group type="NXnote" name="ROI" minOccurs="0" maxOccurs="unbounded">
      <doc>
        **THIS FIELD IS SCHEDULED FOR DELETION on or about March 1, 2022.** Contact abarbour@bnl.gov or jemian@anl.gov to object.

        Region(s) of interest.

        NAME: The NeXus convention, to use all upper case
        to indicate the name (here ``roi``), is left to the file
        writer.  In our case, follow the suggested name
        pattern and sequence: roi_1, roi_2, roi_3, ...
        Start with ``roi_1`` if the first one, otherwise
        pick the next number in this sequence.
      </doc>
    </group>

    <group type="NXnote" name="NOTE" minOccurs="0" maxOccurs="unbounded">
      <doc>
        Any other notes.

        NAME: The NeXus convention, to use all upper case
        to indicate the name (here ``NOTE``), is left to the file
        writer.  In our case, follow the suggested name
        pattern and sequence: note_1, note_2, note_3, ...
        Start with ``note_1`` if the first one, otherwise
        pick the next number in this sequence.
      </doc>
    </group>
  </group>
</definition>