Converting Siemens Vida data

[BrainStormCenter]

Radiology recently installed a new Siemens Vida scanner. Apparently it exports in the enhanced dicom format, which doesn’t work with any of my software. I’m using Chris Rorden's dcm2niiX version v1.0.20181005 Clang10.0.0 (64-bit MacOS) - without success. Every image seems to be a separate stack. Here is a link to the data I was given.
https://drive.google.com/open?id=1fIHE4rq15sjDvwdjatSlUVzh6EjqCeEu

Jason

[neurolabusc]

Jason-

Thanks for sharing. These images are clear violations of the DICOM standard, and include many problems with Type 1 and Type 2 attributes. This is not a problem with my software, but an issue with the vendor. I suggest you contact your Research Collaboration manager and ask them to honor their DICOM conformance statement. Siemens has a nice system for sending problematic images to their engineers.

Your Mosaic images are missing a Type 1 Attribute: Image Position Patient (0020,0032). They can not be accurately positioned - at best they will have poor starting estimates, at worst they may cause left right confusion with clinical consequences. The DICOM standard is very complex, and it is common for vendors to have difficulty getting everything correct the first time. I am surprised these issues persist - I first reported several of these issues last November.

All the images have a clear violation of a Type 2 Attribute: Study Time (0008,0030) defined as "Time the Study started". In your example they appear to have inserted the acquisition time, that varies between slices. I think dcm2niix reports this clearly: slices not stacked: Study Date/Time (0008,0020 / 0008,0030) varies 20181003124358.394531250000 ~= 20181003124355.937500000000.

I have uploaded a desperate kludge that can salvage your non-mosaic images. Compiling this version with the myIgnoreStudyTime compiler directive will ignore the varying study time. This will process your images, albeit in theory it will fail to discriminate legitimately different images, so use with caution. However, this version will fail to de-interlace your mosaics nor provide accurate image orientation for the mosaics - this is not a fault of my software, this information does not exist in the pseudo-DICOMs your system created. The minimal compile for this special version is
g++ -O3 -I. main_console.cpp nii_foreign.cpp nii_dicom.cpp jpg_0XC3.cpp ujpeg.cpp nifti1_io_core.cpp nii_ortho.cpp nii_dicom_batch.cpp -o dcm2niix -DmyDisableOpenJPEG -DmyIgnoreStudyTime

[BrainStormCenter]

Chris,

Thanks for taking the time to investigate this issue and to work up a desperate kludge, which I will try being mindful of the issues you mention. Thankfully, this is pilot data we collected before scanning real subjects for our current grant, which was supposed to begin on Monday. As I'm a researcher, not a clinician, I'll talk to the Chair of Radiology about how I contact the Research Collaboration manager regarding DICOM conformance. Hopefully this can be addressed fairly quickly. As the Vida system is fairly new and not widely used, yet, I'll keep you posted on developments for the sake of documentation, if that’s ok.

Cheers,
Jason

[neurolabusc]

Jason-
I have uploaded a new version that auto-detects XA10A data.

1. If ImplementationVersionName (0002,0013) includes XA10A, images with different Study Times (0008,0030) will be combined as long as they share a common SeriesInstanceUID (0020,000e). This supersedes my DmyIgnoreStudyTime and has the benefit of not causing issues with non-XA10A data.
2. Siemens considers XA10A a secondary capture images intended for quality assurance only. The mosaics do not include slice orientation, image resolution, slice thickness, number of slices in mosaic, phase encoding steps or many of the other attributes considered crucial to image processing. To help those who have mistakenly saved XA10A data as mosaics, I have added a new feature that is enabled with the mySaveXA10Mosaics directive. The minimal compile is:

g++ -O3 -I.  main_console.cpp nii_foreign.cpp nii_dicom.cpp jpg_0XC3.cpp ujpeg.cpp nifti1_io_core.cpp nii_ortho.cpp nii_dicom_batch.cpp  -o dcm2niix -DmyDisableOpenJPEG -DmySaveXA10Mosaics

When compiled this way, the program will pause when it encounters an XA10A mosaic and request that the user enter the number of slices in the mosaic

Siemens XA10 Mosaics are not primary images and lack vital data.
 See https://github.com/rordenlab/dcm2niix/issues/236
INPUT REQUIRED FOR Anonymous.MR._.17.6.2018.10.03.12.43.29.437.23003264.dcm
PLEASE ENTER NUMBER OF SLICES IN MOSAIC:

For your dataset, you would enter '32' at this stage, and the software will de-interlace your mosaic. You will still need to manually enter crucial data into your NIfTI file (spatial and temporal parameters). If you have taken an anatomical scan, entering this information is probably sufficient to get a good coregistration and allow you to salvage fMRI and resting state data. Unfortunately, slice angulation is crucial for diffusion data, so XA10A diffusion data can not be salvaged that is not acquired orthogonal to the scanner bore.

Note this is another desperate kludge. When compiled with mySaveXA10Mosaics, the software must be run from the terminal interactively. If this executable gets called from any other script or program (FSLeyes, MRIcroGL, etc) the software will hang whenever it encounters a XA10A mosaic.

I am closing this issue - we are getting everything we can out of these images. This is a Siemens issue and not an issue with my software.

[neurolabusc]

PUBLIC SERVICE ANNOUNCEMENT

Since before DICOM (e.g. Siemens Vision format), users of Siemens equipment have learned that saving data in MOSAIC format creates smaller files that can be transferred and loaded faster. These assumptions do not carry forward to the Siemens XA10 DICOM images introduced with the Vida. Siemens considers the XA10A mosaics secondary capture images intended for quality assurance only. These mosaics lack many of the parameters required for subsequent analyses. I can think of no circumstances where a Vida user should consider saving data in this format.

When saved in the non-mosaic enhanced format, the Siemens XA10 DICOM format is clean, fast and generates smaller files than prior Siemens scanners. However, users should be aware that these images currently do not include the rich information found from previous Siemens equipment (e.g. Trio, Prisma, Skyra, etc) such as the CSA header. Due to this, the BIDS files generated when converting XA10A data may not include many parameters you expect for slice timing, un-distortion, etc. You will want to be careful to document these parameters on the scanner console for all archival data.

[BrainStormCenter]

Chris, Thank you. Really. I sincerely appreciate the time and effort you spent on this issue, it goes way above and beyond expectations. I’ll try the software in just a little bit and hopefully get something out of the data. However, as you mentioned, not all the data will be salvageable, given that this is a Siemens issue. Hopefully, we will be able to work with someone from Siemens this upcoming week and get the issue resolved. I hope. Cheers, Jason Sent from my phone; please pardon any communication catastrophes. On Oct 6, 2018, at 9:08 AM, Chris Rorden <[email protected]<mailto:[email protected]>> wrote: Jason- I have uploaded a new version that auto-detects XA10A data. 1. If ImplementationVersionName (0002,0013) includes XA10A, images with different Study Times (0008,0030) will be combined as long as they share a common SeriesInstanceUID (0020,000e). This supersedes my DmyIgnoreStudyTime and has the benefit of not causing issues with non-XA10A data. 2. Siemens considers XA10A a secondary capture images intended for quality assurance only. The mosaics do not include slice orientation, image resolution, slice thickness, number of slices in mosaic, phase encoding steps or many of the other attributes considered crucial to image processing. To help those who have mistakenly saved XA10A data as mosaics, I have added a new feature that is enabled with the mySaveXA10Mosaics directive. The minimal compile is: g++ -O3 -I. main_console.cpp nii_foreign.cpp nii_dicom.cpp jpg_0XC3.cpp ujpeg.cpp nifti1_io_core.cpp nii_ortho.cpp nii_dicom_batch.cpp -o dcm2niix -DmyDisableOpenJPEG -DmySaveXA10Mosaics When compiled this way, the program will pause when it encounters an XA10A mosaic and request that the user enter the number of slices in the mosaic Siemens XA10 Mosaics are not primary images and lack vital data. See #236 INPUT REQUIRED FOR Anonymous.MR._.17.6.2018.10.03.12.43.29.437.23003264.dcm PLEASE ENTER NUMBER OF SLICES IN MOSAIC: For your dataset, you would enter '32' at this stage, and the software will de-interlace your mosaic. You will still need to manually enter crucial data into your NIfTI file (spatial and temporal parameters). If you have taken an anatomical scan, entering this information is probably sufficient to get a good coregistration and allow you to salvage fMRI and resting state data. Unfortunately, slice angulation is crucial for diffusion data, so XA10A diffusion data can not be salvaged that is not acquired orthogonal to the scanner bore. Note this is another desperate kludge. When compiled with mySaveXA10Mosaics, the software must be run from the terminal interactively. If this executable gets called from any other script or program (FSLeyes, MRIcroGL, etc) the software will hang whenever it encounters a XA10A mosaic. I am closing this issue - we are getting everything we can out of these images. This is a Siemens issue and not an issue with my software. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#236 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AQySS__saiIcWy3o1XSsN5InPmjYgTOAks5uiLl0gaJpZM4XKySp>.

[neurolabusc]

For users of the current Siemens Vida software: it is critical that you export data from the scanner in enhanced mode without the Siemens online anonymization. Any other combination will lead to data that does not have the critical tags for analyses.

• Data can be saved in Enhanced, Interoperability or Mosaic mode. Siemens notes we highly recommend that the Enhanced DICOM format be used. This is because this format retains far more information in the header. Furthermore, the Interoperability mode can increment the series number for every volume in a 4D series.
• The Siemens de-identification can have unexpected consequences, such as generating a unique study time for every slice and inserting inaccurate Acquisition Times (which can impact slice time estimates). Futhermore, important tags can be stripped from the data. Siemens recommends the use an offline/in-house anonymization software instead.
• Be aware that many DICOM anonymization and storage tools pre-date the development of the enhanced DICOM format. I would recommend validating any tool, and retaining the original format as much as possible (e.g. retain explicit VR rather than converting to implicit VR).

[BrainStormCenter]

As someone whom Chris has helped deal with this issue - I cannot stress enough the importance of this warning and advice. ⚠️ For the sake of clarity and documentation: Does the current version of dcm2niix work with the both the standard dicom and this NEW enhanced dicom formats? Cheers, Jason On Oct 13, 2018, at 4:05 PM, Chris Rorden <[email protected]<mailto:[email protected]>> wrote: For users of the current Siemens Vida software: it is critical that you export data from the scanner in enhanced mode without the Siemens online anonymization. Any other combination will lead to data that does not have the critical tags for analyses. * Data can be saved in Enhanced, Interoperability or Mosaic mode. Siemens notes we highly recommend that the Enhanced DICOM format be used. This is because this format retains far more information in the header. Furthermore, the Interoperability mode can increment the series number for every volume in a 4D series. * The Siemens de-identification can have unexpected consequences, such as generating a unique study time for every slice and inserting inaccurate Acquisition Times (which can impact slice time estimates). Futhermore, important tags can be stripped from the data. Siemens recommends the use an offline/in-house anonymization software instead. * Be aware that many DICOM anonymization and storage tools pre-date the development of the enhanced DICOM format. I would recommend validating any tool, and retaining the original format as much as possible (e.g. retain explicit VR rather than converting to implicit VR). [image002]<https://user-images.githubusercontent.com/8930807/46909844-2fe41e80-cf07-11e8-9490-5fb930bbb01c.jpg> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#236 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AQySSwqM2uRix-QwZ6pQEEirSPq6tZWlks5uklWugaJpZM4XKySp>.

[neurolabusc]

• enhanced not-anonymized data: dcm2niix will deal with this well. The DICOM images have the critical tags but are still missing some of the helpful tags that we get from other Siemens systems. Specifically, details like read-out time, phase encoding polarity. I would suggest making protocol names that make these properties explicit, so if you acquire two DWI scans with reverse phase encoding (so you can use TOPUP), you might want to call them 'DWI_AP' and 'DWI_PA' so you can identify them later.

• mosaic images: these do not have slice orientation, number of slices, or diffusion parameters. No software can extract these accurately. You might be able to salvage fMRI/resting state with some manual operations. The DWI are useless.

• Interoperability mode: the latest commits of dcm2niix should auto-detect the Vida format and should be able to identify series saved with multiple series numbers.

• Siemens de-identified (anonymized) images in any format: The problems listed above for each format are compounded. Some missing parameters could in theory be entered by hand. However, this is likely to be arduous, particularly for DWI sequences.

[neurolabusc]

The current stable release (v1.0.20211006) will generate a false alarm referring to this issue when converting XA30 data:

Warning: Siemens XA exported as classic not enhanced DICOM (issue 236)

Explanation: Previously seen XA10 and XA11 enhanced DICOM saved all 2D slices from an entire 4D series in a single enhanced DICOM file. With XA30 it appears that it is possible that the system generates one enhanced DICOM 3D file for each volume of a 4D series. Since dcm2niix detects more than one file in the series, it generates a warning.

This warning will be silenced in future releases. XA30 users should remain vigilant, as we have little experience with this new format.

[captainnova]

Actually, the false warning will appear with a lot (most?) of enhanced DICOM from XA10, 11, and 20 too. All of the Siemens enhanced DICOM I have seen (from XA10, 11 + 20 so far) has been of the 1 file per volume form.

I'm not sure how I missed this in prerelease testing.

[mharms]

I had a site export a session collected under XA30 (Prisma) to their local XNAT through the Network option on their console. The result seems to be an odd mixture of "Enhanced" and "Classic" DICOMs. Here's a sampling of the (0002,0002) field, as reported by dcmdump:

6_Localizer_aligned_i06001.dcmdump.txt:(0002,0002) UI =MRImageStorage                          #  26, 1 MediaStorageSOPClassUID
7_DistortionMap_AP.dcmdump.txt:(0002,0002) UI =EnhancedMRImageStorage                  #  28, 1 MediaStorageSOPClassUID
8_DistortionMap_PA.dcmdump.txt:(0002,0002) UI =EnhancedMRImageStorage                  #  28, 1 MediaStorageSOPClassUID
9_T1w_MPR.dcmdump.txt:(0002,0002) UI =MRImageStorage                          #  26, 1 MediaStorageSOPClassUID
10_T2w_SPC.dcmdump.txt:(0002,0002) UI =MRImageStorage                          #  26, 1 MediaStorageSOPClassUID
14_rfMRI_REST_AP.dcmdump.txt:(0002,0002) UI =EnhancedMRImageStorage                  #  28, 1 MediaStorageSOPClassUID
16_rfMRI_REST_PA.dcmdump.txt:(0002,0002) UI =EnhancedMRImageStorage                  #  28, 1 MediaStorageSOPClassUID
18_dMRI_b0_AP.dcmdump.txt:(0002,0002) UI =MRImageStorage                          #  26, 1 MediaStorageSOPClassUID
20_dMRI_dir176_PA.dcmdump.txt:(0002,0002) UI =MRImageStorage                          #  26, 1 MediaStorageSOPClassUID

I discovered this because I noticed that I was still getting the following warning (using v1.0.20220720):
Warning: Siemens XA exported as classic not enhanced DICOM (issue 236)

Sure enough, the series that generate that warning are indeed the ones with (0002,0002) = "MRImageStorage", rather than "EnhancedMRImageStorage".

Does anyone have any insight into what is going on? (i.e., What is Siemens doing?!?)

[mharms]

To expand a little further, this mixing of "Classic" and "Enhanced" DICOMs from the export of the series within a given session has important implications for how one interprets the DICOM fields, since Siemens populates information for the two different types of DICOMs differently.

Here's an example for the "ImageType" information:

[mharms@login2 NIFTI]$ grep ImageType 14_rfMRI_REST_AP.json
	"ImageType": ["ORIGINAL", "PRIMARY", "FMRI", "NONE"],
	"ImageTypeText": ["ORIGINAL", "PRIMARY", "M", "MB", "ND", "NORM"],
[mharms@login2 NIFTI]$ grep ImageType 20_dMRI_dir176_PA.json
	"ImageType": ["ORIGINAL", "PRIMARY", "DIFFUSION", "NONE", "MB", "ND", "NORM", "MFSPLIT"],
	"ImageTypeText": ["ORIGINAL", "PRIMARY", "DIFFUSION", "NONE"],

Series 14 (rfMRI) was exported as "Enhanced" DICOM, whereas series 20 (dMRI) was exported as "Classic" DICOM (as shown in my previous post). For the former, the information about whether Prescan Normalize was applied ("NORM") is contained in the ImageTypeText field (0021,1175), whereas for the latter its contained in the ImageType field (0008,0008).

This obviously places extra burden on the user to know where to look for a given piece of information, and requires that users be aware of the fact that Siemens Network export option on XA30 appears to export some acqusitions as "Classic" DICOMs and others as "Enhanced" DICOMs.

[neurolabusc]

@mharms your comments of Prescan Normalize and disambiguating it from NonlinearGradientCorrection are related to issue 597 and dcm2niix does inspect both ImageType and ImageTypeText to determine the values. I did talk to my Siemens Research Collaboration Manager about this discrepancy. Since ImageType is public tag, with clearly defined values, Siemens has decided not to put non-standard values in the private tag ImageTypeText. Therefore, moving forward values that Siemens used to put in ImageType will be put in the private tag ImageTypeText to improve compliance with the DICOM standard. Siemens XA is a moving target, but at least it seems to be moving in the right direction. I am not sure why different series are behaving differently on this, it might reflect a variance in how product, WIP or research sequences are exported.

I worked closely with Siemens engineers to make sure the enhanced XA20 scans include rich sequence data, hence there was a quantum leap in details provided between XA11 and XA20. I also do not know if XNAT modifies the data. Since I do not have XA30, I am not sure whether the classic DICOMs are still impoverished relative to enhanced DICOMs. I do know that Siemens recommendation is to export research data as enhanced.

Do my comments resolve your issue? It seems to me the dcm2niix behavior is appropriate, and raising a warning (not an error) for XA data saved as classic DICOMs still seems appropriate.

[mharms]

The fundamental issue that I'm trying to bring to light for the community is NOT about differences between the ImageType and ImageTypeText fields per se. But rather that the Network export option is sending some series as "Classic" and some as "Enhanced", all collected within the same imaging session. Note that the Network export option does not give you any options (other than destination), so users seemingly have no control over how data gets exported when using a Network send. I think it is highly unlikely that XNAT itself is the source of this behavior.

I'd be very curious to get some comment on this behavior from a Siemens engineer, if you could solicit their feedback, or if one of them follows this repo. Also, maybe @eauerbach could comment on whether the choice between "Classic" vs. "Enhanced" DICOM exports is something that the CMRR sequences exercise any control over.

We appear to be getting all the info under XA30 that we had under VE11, even for the series exported as "Classic". That may be because even the "Classic" DICOMs under XA30 include the (0021, *) fields, one of which includes the "ASCII CSA" information (which dcm2niix is obviously querying). So, while the Warning message is probably still appropriate (that's what brought this to light), it may not be as relevant under XA30 as it was for earlier XA versions.

I'm not saying that dcm2niix is doing anything incorrectly. This Issue just seemed like an appropriate one to document this behavior and continue the conversation about how XA30 is operating.

BTW, in your post above, you wrote:

"Siemens has decided not to put non-standard values in the private tag ImageTypeText."

I don't think you intended to include that 'not' in that statement. i.e., standard DICOM tags for ImageType are now used for "Enhanced" DICOMs (but not for "Classic" DICOMs under XA30, per above). Non-standard information (i.e., the info previously included in ImageType under "Classic" DICOMs of VE11) now goes into ImageTypeText for "Enhanced" DICOMs under XA30.

[neurolabusc]

Mea culpa, I meant to write Siemens has decided to put non-standard values in the private tag ImageTypeText rather than the public ImageType tag. I would suggest you communicate with the Siemens Research Collaboration Manager associated with the site where the data was acquired. It is their role to resolve issues and communicate with the engineers. I do not believe the RCM associated with my center has much direct experience with XA30.

[eauerbach]

On the developer side, my experience with XA30 so far has been that Enhanced (multi-frame) simply replaces Mosaic. There is no special control over the output format in the sequence code, and in ICE it is actually still called Mosaic in some of the functors. Maybe I am missing something, but I have only seen the Enhanced format used for 3D volumes in exactly the same situations where Mosaic would have been used previously. I have never seen the 4D volume format mentioned upthread; maybe that was discontinued after XA10/11 or so.

In the session list posted above the only questionable case is the two diffusion scans at the end. However, remember that in previous versions, Mosaic was restricted for diffusion: the Mosaic checkbox in the UI would only be selectable under certain conditions, like if 6 or more directions and 2 or more b-values were selected (among other things, as far as I recall). The checkbox is gone now, but I wouldn't be surprised if the same cases that didn't allow Mosaic previously are resulting in non-Enhanced images now.

[neurolabusc]

The dcm_qa_xa30 validation datasets for XA30 provide enhanced datasets for ((0002,0002) UI =EnhancedMRImageStorage ) all modalities (T1, fMRI, DWI, ASL). Regardless of whether the Siemens data could historically have been saved as mosaic (e.g. DWI, fMRI, ASL) or not not (T1), the validation data is always enhanced DICOM.

As an aside, while Philips and Canon save an entire 4D series in a single file, with Siemens each 3D volume is saved as a separate file.

[mharms]

Thanks @eauerbach, that's helpful, esp. the reminder about the old Mosaic checkbox on diffusion scans -- except there is no reason (as far as I can tell) that the dMRI data in question wouldn't have allowed Mosaic under VE11. Plus, per what @neurolabusc wrote regarding the dcm_qa_xa30 validation dataset, it appears possible to get dMRI DICOMs with all the expected/"typical" information exported as "Enhanced" DICOMs under XA30. Which simply returns us to the question of why Siemens would choose to export the dMRI series as "Classic" DICOMs instead if using the Network export option.

I did notice that the DWI in the validation dataset was collected in MDDW mode, whereas the data that I'm looking at was acquired in Free mode. I wonder if that factors in somehow...

Also, I'm now wondering if a File system export with "Image Conversion" set to "Interoperability" mode (rather than "Enhanced") generates "Classic" DICOMs for all series, or if it does the same sort of thing as the Network export in which some series are exported as "Classic" DICOMs, and other series as "Enhanced".

[neurolabusc]

@mharms can we close this issue? It seems like dcm2niix supports XA30 as well as can be hoped. While values can be stored in either the public ImageType (0008,0008) and private ImageTypeText (0021,1175) tags, the behavior of dcm2niix is documented.

[mharms]

Sure. The main open questions in my mind are:

1. Why Siemens choose to export some series as "Enhanced" DICOMs, and others as "Classic" when using the Network export option on XA30? That would probably require direct input from someone in Siemens.
2. If you use "Interoperability" mode for a File system export under XA30, what exactly do you get? e.g., Do you get the same mix of "Enhanced" and "Classic" DICOMs that you get with a Network export?

If I learn more regarding (2), I'll re-open this Issue and post my findings.

[neurolabusc]

An update based on new XA 30 sample data. The XA30 systems can export DICOMs in one of two modes:

• Interoperability: assures that the data is compatible to any DICOM node, but enhanced features cannot be guaranteed.
• Enhanced: preserves the enhanced features of the source images.

dcm2niix users should be encouraged to use Enhanced mode which generates one file per 3D volume, provides more sequence details, and requires less overall disk space. The Interoperability mode creates classic DICOMs (one file per 2D slice). Some EPI sequences (fMRI, ASL) may also increment the series number with each volume, and these list MFSPLIT in the ImageType tag (0008,0008). I am adding a commit that attempts to accurately extract slice times for MFSPLIT (traditionally, dcm2niix only extracts slice times from a series with multiple volumes as slice timing routines are designed for 4D datasets).

[mharms]

An update based on new XA 30 sample data. The XA30 systems can export DICOMs in one of two modes:

* Interoperability: assures that the data is compatible to any DICOM node, but enhanced features cannot be guaranteed.

* Enhanced: preserves the enhanced features of the source images.

Those two "modes" are only exposed for selection to the user if they are exporting the data via the "File system" option (e.g., to disk, or a USB drive). If you export via the separate Network option (e.g., to a local XNAT, or presumably PACS), no such option exists, and rather, as is commented above, Siemens appears to export different data types in different formats, with dMRI data being exported (unfortunately) as "Classic" DICOMs (one DICOM per slice per volume).

So, when we are talking about exporting XA30, I think we should say there are three "modes" of exporting data.