feat(4D): Add 4D dynamic volume rendering and new pre-clinical 4d pt/ct mode

[lscoder]

---

Alireza:

💯 👏👏
This work is published under the set aside funding U24 for enhancing the Open Health Imaging Foundation (OHIF) Viewer to support preclinical 4D imaging datasets and integrating it with the Preclinical Imaging XNAT-enabled Informatics (PIXI) platform at Washington University in St. Louis by Dr. Kooresh Shoghi.

Big shoutout to @lscoder and @NeilMacPhee for kicking off the work on this fantastic PR! 💯 👏👏

---

Breaking Changes

leftPanelDefaultClosed is now called leftPanelClosed, same for right

New Features

New Mode (4D PT/CT)

This work introduces a new preclinical 4D PET/CT mode that allows visualization, navigation, and analysis of 4D preclinical imaging data, including features like multi-modality image fusion, drawing 2D/3D ROIs across time points, and exporting voxel data within ROIs over time.

Basic 4D support via enhanced CINE player in basic viewer

You can also view your 4D data in the basic viewer. We have enhanced our CINE player to automatically detect the 4D series and show the splitting tag

(Note the CINE player in the bottom of the viewport)

CleanShot.2024-04-09.at.15.28.25.mp4

dd.mp4

New Chart Viewport

We have added a new Chart viewport that you can utilize to show relevant information. To use this type of viewport you need to create an instance and add it to the DICOMMetadataStore so that a displaySet is created after. Take a look at how we do so in the extensions/cornerstone-dynamic-volume/src/actions/updateSegmentationsChartDisplaySet.ts

We will simplify this component usage in the future

const instance = {
    SOPClassUID: ChartDataSOPClassUid,
    Modality: CHART_MODALITY,
    SOPInstanceUID: utils.guid(),
    SeriesDate: seriesDate,
    SeriesTime: seriesTime,
    SeriesInstanceUID: SEG_CHART_INSTANCE_UID,
    StudyInstanceUID: segmentationsData[0].StudyInstanceUID,
    StudyDescription: segmentationsData[0].StudyDescription,
    SeriesNumber: 100,
    SeriesDescription: 'Segmentation chart series data',
    chartData: {
      series,
      axis: { ...segmentationsData[0].chartData.axis },
    },
  };

  const seriesMetadata = {
    StudyInstanceUID: instance.StudyInstanceUID,
    StudyDescription: instance.StudyDescription,
    SeriesInstanceUID: instance.SeriesInstanceUID,
    SeriesDescription: instance.SeriesDescription,
    SeriesNumber: instance.SeriesNumber,
    SeriesTime: instance.SeriesTime,
    SOPClassUID: instance.SOPClassUID,
    Modality: instance.Modality,
  };

DicomMetadataStore.addSeriesMetadata([seriesMetadata], true);
DicomMetadataStore.addInstances([instance], true);

and use a hanging protocol to render it (extensions/cornerstone-dynamic-volume/src/getHangingProtocolModule.ts)

displaySetSelectors: {

chartDisplaySet: {
      seriesMatchingRules: [
        {
          attribute: 'Modality',
          constraint: {
            equals: {
              value: 'CHT',
            },
          },
          required: true,
        },
      ],
    },
}


function getSeriesChartViewport() {
  return {
    viewportOptions: {
      viewportId: 'seriesChart',
    },
    displaySets: [
      {
        id: 'chartDisplaySet',
        options: {
          // This dataset does not require the download of any instance since it is pre-computed locally,
          // but interleaveTopToBottom.ts was not loading any series because it consider that all viewports
          // are a Cornerstone viewport which is not true in this case and it waits for all viewports to
          // have called interleaveTopToBottom(...).
          skipLoading: true,
        },
      },
    ],
  };
}

Workflow Step Service

This service allows you to manage your workflow in smaller steps. It provides a structured way to define and navigate through different stages
or phases of a larger process or workflow. Each step can have its own configuration, layout, toolbar buttons, and other settings tailored to the specific requirements of that stage.

Anatomy of a Workflow Step

The anatomy of a workflow step refers to the different components or properties that define and configure each individual step within the workflow. Each step can be customized with various settings to tailor the user interface, available tools, and behavior of the application for that specific stage of the workflow. Here are the key components that make up a workflow step:

• id: A unique identifier for the step

• name: A human-readable name or title for the step, which can be displayed in the user interface to help users understand the current stage of the workflow.

• hangingProtocol: The hanging protocol configuration specifies the protocol and stage ID to be used for displaying the images. This ensures that the appropriate data viewports and presentation are used for the current workflow step.

• layout: The layout configuration defines the arrangement and visibility of various panels or viewports within the application's user interface for the specific step. This can include specifying which panels should be visible on the left or right side of the screen, as well as any options for panel visibility or behavior.

• toolbarButtons: Each step can define a set of toolbar buttons that should be available and displayed in the application's toolbar during that step. Remember the button definitions should already be registered to toolbarService beforehand, here we are just referencing the buttons id in each section.

• info : An optional description or additional information about the current workflow step can be provided. which
  will be displayed as tooltip in the UI.

• Step Callbacks or Commands: Some workflow steps may require specific actions or commands to be executed when the step is entered or exited. These callbacks or commands can be defined within the step configuration and can be used to update the application's state, perform data processing, or trigger other relevant actions. For instance you have access to onEnter hook to run a command right after the step is entered.

For instance, a simplified example of our pre-clinical 4D workflow steps configuration might look like this:

const dynamicVolume = {
  sopClassHandler:
    "@ohif/extension-cornerstone-dynamic-volume.sopClassHandlerModule.dynamic-volume",
  leftPanel:
    "@ohif/extension-cornerstone-dynamic-volume.panelModule.dynamic-volume",
  toolBox:
    "@ohif/extension-cornerstone-dynamic-volume.panelModule.dynamic-toolbox",
  export:
    "@ohif/extension-cornerstone-dynamic-volume.panelModule.dynamic-export",
}

const cs3d = {
  segmentation:
    "@ohif/extension-cornerstone-dicom-seg.panelModule.panelSegmentation",
}



const steps = [
  {
    id: "dataPreparation",
    name: "Data Preparation",
    layout: {
      panels: {
        left: [dynamicVolume.leftPanel],
      },
    },
    toolbarButtons: {
      buttonSection: "primary",
      buttons: ["MeasurementTools", "Zoom", "WindowLevel", "Crosshairs", "Pan"],
    },
    hangingProtocol: {
      protocolId: "default4D",
      stageId: "dataPreparation",
    },
    info: "In the Data Preparation step...",
  },
  {
    id: "roiQuantification",
    name: "ROI Quantification",
    layout: {
      panels: {
        left: [dynamicVolume.leftPanel],
        right: [
          [dynamicVolume.toolBox, cs3d.segmentation, dynamicVolume.export],
        ],
      },
      options: {
        leftPanelClosed: false,
        rightPanelClosed: false,
      },
    },
    toolbarButtons: [
      {
        buttonSection: "primary",
        buttons: [
          "MeasurementTools",
          "Zoom",
          "WindowLevel",
          "Crosshairs",
          "Pan",
        ],
      },
      {
        buttonSection: "dynamic-toolbox",
        buttons: ["BrushTools", "RectangleROIStartEndThreshold"],
      },
    ],
    hangingProtocol: {
      protocolId: "default4D",
      stageId: "roiQuantification",
    },
    info: "The ROI quantification step ...",
  },
  {
    id: "kineticAnalysis",
    name: "Kinetic Analysis",
    layout: {
      panels: {
        left: [dynamicVolume.leftPanel],
        right: [],
      },
    },
    toolbarButtons: {
      buttonSection: "primary",
      buttons: ["MeasurementTools", "Zoom", "WindowLevel", "Crosshairs", "Pan"],
    },
    hangingProtocol: {
      protocolId: "default4D",
      stageId: "kineticAnalysis",
    },
    onEnter: [
      {
        commandName: "updateSegmentationsChartDisplaySet",
        options: { servicesManager },
      },
    ],
    info: "The Kinetic Analysis step ...",
  },
]

Integration

After you have defined your workflow steps, you can integrate them into your application by using the workflowStepsService.

These steps should be called on onSetupRouteComplete in your mode factory.

Note: onModeEnter is too soon to call these steps as the mode is not yet fully initialized.

onSetupRouteComplete: ({ servicesManager }) => {
  workflowStepsService.addWorkflowSteps(workflowSettings.steps);
  workflowStepsService.setActiveWorkflowStep(workflowSettings.steps[0].id);
},

check out the modes/preclinical-4d/src/index.tsx for a complete example.

User Interface

We have developed a simple dropdown UI element that you can use to navigate between the different steps of your workflow. This dropdown can be added to the toolbar like below:

toolbarService.addButtons([
  {
    id: 'ProgressDropdown',
    uiType: 'ohif.progressDropdown',
  },
])
toolbarService.createButtonSection('secondary', ['ProgressDropdown']);

It will appear in the secondary location in the toolbar.

Re-usable Segmentation Panel and Stackable Panels

We've cleaned up a bunch of duplicated code related to the segmentation panel that was being repeatedly used with various settings. You can now work with the original segmentation panel in your modes and tweak its behavior. Currently, there's just one PanelSegmentation implementation, and tmtv, viewer, and pre-clinical all make use of this panel with distinct behaviors.

For instance we modify the behaviour of the seg panel in tmtv mode like

 customizationService.addModeCustomizations([
        {
          id: 'segmentation.panel',
          segmentationPanelMode: 'expanded',
          addSegment: false,
          onSegmentationAdd: () => {
            commandsManager.run('createNewLabelmapFromPT');
          },
        },
      ]);

You can now stack different panel modules on top of each other. Just provide the ids in an array

For instance in the tmtv mode we have

layoutTemplate: () => {
          return {
            id: ohif.layout,
            props: {
              leftPanels: [ohif.thumbnailList],
              leftPanelClosed: true,
              rightPanels: [[tmtv.toolbox, cs3d.segPanel, tmtv.export], tmtv.petSUV],
              viewports: [
                {
                  namespace: cs3d.viewport,
                  displaySetsToDisplay: [ohif.sopClassHandler],
                },
              ],
            },
          };
        },

Note the rightPanels: [[tmtv.toolbox, cs3d.segPanel, tmtv.export], tmtv.petSUV], specially the first item which is now an array [tmtv.toolbox, cs3d.segPanel, tmtv.export]

So go ahead and only create parts of the panels that you want specific to your workflow and use OHIF's implementation on more general panels.

New Window Level Component

There is a new window level component that you can use to view the histogram of the viewport window level, and then manipulate it. Use it like a panel sub component and stack it in your panels. We plan to make it more general so that you can put it in a modal or etc

CleanShot.2024-04-09.at.16.39.44.mp4

Using webworkers for computationally heavy task

We have started to move computationally heavy tasks to workers, and as start you can look how we calculate histogram in such way

extensions/cornerstone/src/components/ViewportWindowLevel/histogramWorker.js

[netlify]

✅ Deploy Preview for ohif-platform-docs canceled.

Name	Link
🔨 Latest commit	82b3c13
🔍 Latest deploy log	https://app.netlify.com/sites/ohif-platform-docs/deploys/66160ac03687e50008ca67c6

[netlify]

✅ Deploy Preview for ohif-dev canceled.

Name	Link
🔨 Latest commit	82b3c13
🔍 Latest deploy log	https://app.netlify.com/sites/ohif-dev/deploys/66160ac0febb39000884959f

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.37%. Comparing base (dc0b183) to head (82b3c13).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3664   +/-   ##
=======================================
  Coverage   44.37%   44.37%           
=======================================
  Files          80       80           
  Lines        1334     1334           
  Branches      327      327           
=======================================
  Hits          592      592           
  Misses        589      589           
  Partials      153      153           

Files	Coverage Δ
platform/app/src/appInit.js	0.00% <ø> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc0b183...82b3c13. Read the comment docs.