dwi2response: Unhandled Python exception ValueError: could not convert string to float: N/A

[GerardYu]

Hi there, i've encounter the error below when running dwi2response. It appears that this error only occurs on 1 out of 91 of my subjects. If its helpful, i've uploaded beddytensors.mif here https://drive.google.com/file/d/1omAdS-2td2PwVZjIPEVlmEB4FWdQvYjB/view?usp=sharing

junhong@junhong-Precision-5820-Tower:/bcs/junhong/MAP/data/017/time1/DTImrt2$ dwi2response dhollander beddytensors.mif wm_response.txt gm_response.txt csf_response.txt -nthreads 20 -force
dwi2response:
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response:
dwi2response: Generated scratch directory: /bcs/junhong/MAP/data/017/time1/DTImrt2/dwi2response-tmp-KZJOBX/
dwi2response: Importing DWI data (/bcs/junhong/MAP/data/017/time1/DTImrt2/beddytensors.mif)...
dwi2response: Changing to scratch directory (/bcs/junhong/MAP/data/017/time1/DTImrt2/dwi2response-tmp-KZJOBX/)
dwi2response: Computing brain mask (dwi2mask)...
dwi2response: -------
dwi2response: 3 unique b-value(s) detected: 0,1000,3000 with 14,60,60 volumes
dwi2response: -------
dwi2response: Preparation:
dwi2response: * Eroding brain mask by 3 pass(es)...
dwi2response: [ mask: 49228 -> 32396 ]
dwi2response: * Computing signal decay metric (SDM):
dwi2response: * b=0...
dwi2response: * b=1000...
dwi2response: * b=3000...
dwi2response: * Removing erroneous voxels from mask and correcting SDM...
dwi2response: [ mask: 32396 -> 32393 ]
dwi2response: -------
dwi2response: Crude segmentation:
dwi2response: * Crude WM versus GM-CSF separation (at FA=0.2)...
dwi2response: [ 32393 -> 17770 (WM) & 14623 (GM-CSF) ]
dwi2response: * Crude GM versus CSF separation...
dwi2response: [ 14623 -> 10032 (GM) & 4591 (CSF) ]
dwi2response: -------
dwi2response: Refined segmentation:
dwi2response: * Refining WM...
dwi2response: [ WM: 17770 -> 16257 ]
dwi2response: * Refining GM...
dwi2response: [ GM: 10032 -> 5785 ]
dwi2response: * Refining CSF...

dwi2response: [ERROR] Unhandled Python exception:
dwi2response: [ERROR] ValueError: could not convert string to float: N/A
dwi2response: [ERROR] Traceback:
dwi2response: [ERROR] /home/junhong/mrt/mrtrix3/bin/dwi2response:118 (in execute())
dwi2response: [ERROR] alg.execute()
dwi2response: [ERROR] /home/junhong/mrt/mrtrix3/lib/mrtrix3/dwi2response/dhollander.py:196 (in execute())
dwi2response: [ERROR] statrefcsfcount = image.statistics('refined_csf.mif', mask='refined_csf.mif').count
dwi2response: [ERROR] /home/junhong/mrt/mrtrix3/lib/mrtrix3/image.py:252 (in statistics())
dwi2response: [ERROR] result.append(ImageStatistics(float(line[0]), float(line[1]), float(line[2]), float(line[3]), float(line[4]), float(line[5]), int(line[6])))

[jdtournier]

I think this is what happens when the mask is (almost) empty. It looks like the refined_csf.mif image ends up with a single voxel for some reason. This means the standard deviation is reported as N/A, which Python then (rightly) refuses to convert to float. We should try to fix that special case so the standard deviation is properly reported as NaN or None, and try to provide a more helpful error message at this point.

Regarding why you'd end up with this single-voxel mask, I've had a look through your data: it seems there is one voxel where the mean b=1000 signal is abnormally low (very close to zero). This causes the value of log(b₀/b₁₀₀₀) to be abnormally high, and when this is thresholded later using the automatic threshold determination, this single outlier throws off the determination of the threshold value (because the best correlation between mask and original image is the mask with the single outlier in it). I think we'll need to come up with a more robust way to handle these edge cases...

In the meantime, one simple trick that seems to fix it is to rectify the data to ensure it's non-negative:

mrcalc beddytensors.mif 0 -max beddytensors_rect.mif

On my system, this seems to allow dwi2response to run successfully, and the output responses seems sensible.

[GerardYu]

thanks that works!

[Mtay316]

Hi, I have the same problem. My brain image is with tumour which seems like the signal intensity in the tumour (middle of the brain) is really low (100 wish). First I ran:
dwi2response dhollander dwi.mif wm.txt gm.txt csf.txt

And I got this error

mtay316@MD412890 Test_02 % dwi2response dhollander dwi.mif wm.txt gm.txt csf.txt 
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated scratch directory: /Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-QF063C/
dwi2response: Importing DWI data (/Volumes/CAMRI/Kio_project/Test_02/dwi.mif)...
dwi2response: Changing to scratch directory (/Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-QF063C/)
dwi2response: Computing brain mask (dwi2mask)...
dwi2response: -------
dwi2response: 2 unique b-value(s) detected: 0,1000 with 1,30 volumes
dwi2response: -------
dwi2response: Preparation:
dwi2response: * Eroding brain mask by 3 pass(es)...
dwi2response:   [ mask: 535731 -> 431349 ]
dwi2response: * Computing signal decay metric (SDM):
dwi2response:  * b=0...
dwi2response:  * b=1000...
dwi2response: * Removing erroneous voxels from mask and correcting SDM...
dwi2response:   [ mask: 431349 -> 431349 ]
dwi2response: -------
dwi2response: Crude segmentation:
dwi2response: * Crude WM versus GM-CSF separation (at FA=0.2)...

dwi2response: [ERROR] Unhandled Python exception:
dwi2response: [ERROR]   ValueError: could not convert string to float: N/A
dwi2response: [ERROR] Traceback:
dwi2response: [ERROR]   /usr/local/bin/dwi2response:118 (in execute())
dwi2response: [ERROR]     alg.execute()
dwi2response: [ERROR]   /usr/local/mrtrix3/lib/mrtrix3/dwi2response/dhollander.py:152 (in execute())
dwi2response: [ERROR]     statcrudenonwmcount = image.statistics('_crudenonwm.mif', mask='_crudenonwm.mif').count
dwi2response: [ERROR]   /usr/local/mrtrix3/lib/mrtrix3/image.py:252 (in statistics())
dwi2response: [ERROR]     result.append(ImageStatistics(float(line[0]), float(line[1]), float(line[2]), float(line[3]), float(line[4]), float(line[5]), int(line[6])))

Then I read your explanation and thought it might be because of the low signal intensity from the tumour. So I applied your mentioned command:

mrcalc dwi.mif 0 -max bdwi.mif

And then after running dwi2response command, I got this error:

dwi2response dhollander bdwi.mif wm.txt gm.txt csf.txt
dwi2response: 
dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
dwi2response: 
dwi2response: Generated scratch directory: /Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-WXOI4H/
dwi2response: Importing DWI data (/Volumes/CAMRI/Kio_project/Test_02/bdwi.mif)...
dwi2response: Changing to scratch directory (/Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-WXOI4H/)
dwi2response: Computing brain mask (dwi2mask)...
dwi2response: -------
dwi2response: 2 unique b-value(s) detected: 0,1000 with 1,30 volumes
dwi2response: -------
dwi2response: Preparation:
dwi2response: * Eroding brain mask by 3 pass(es)...
dwi2response:   [ mask: 532624 -> 413063 ]
dwi2response: * Computing signal decay metric (SDM):
dwi2response:  * b=0...
dwi2response:  * b=1000...
dwi2response: * Removing erroneous voxels from mask and correcting SDM...
dwi2response:   [ mask: 413063 -> 413063 ]
dwi2response: -------
dwi2response: Crude segmentation:
dwi2response: * Crude WM versus GM-CSF separation (at FA=0.2)...
dwi2response:   [ 413063 -> 413013 (WM) & 50 (GM-CSF) ]
dwi2response: * Crude GM versus CSF separation...
dwi2response:   [ 50 -> 20 (GM) & 30 (CSF) ]
dwi2response: -------
dwi2response: Refined segmentation:
dwi2response: * Refining WM...
dwi2response:   [ WM: 413013 -> 347890 ]
dwi2response: * Refining GM...
dwi2response:   [ GM: 20 -> 13 ]
dwi2response: * Refining CSF...
dwi2response:   [ CSF: 30 -> 10711 ]
dwi2response: -------
dwi2response: Final voxel selection and response function estimation:
dwi2response: * CSF:
dwi2response:  * Selecting final voxels (10.0% of refined CSF)...
dwi2response:    [ CSF: 10711 -> 1071 ]
dwi2response:  * Estimating response function...
dwi2response: * GM:
dwi2response:  * Selecting final voxels (2.0% of refined GM)...

dwi2response: [ERROR] mrcalc refined_gm.mif safe_sdm.mif 2.09190726 -subtract -abs 1 -add 0 -if - | mrthreshold - - -bottom 0 -ignorezero | mrcalc refined_gm.mif - 0 -if - -datatype bit | mrconvert - voxels_gm.mif -axes 0,1,2 (dhollander.py:220)
dwi2response: [ERROR] Information from failed command:
dwi2response:
              mrcalc: computing: (refined_gm.mif ? (|(safe_sdm.mif - 2.09191)| + 1) : 0)... [==================================================]
              mrthreshold: [ERROR] value supplied for option "bottom" is out of bounds (valid range: 1 to 9223372036854775807, value supplied: 0)
              mrcalc: [ERROR] Could not interpret string "-" as either an image path or a numerical value
              mrcalc: [ERROR] As image: 
              mrcalc: [ERROR] no filename supplied to standard input (broken pipe?)
              mrcalc: [ERROR] error opening image "-"
              mrcalc: [ERROR] As numerical value: 
              mrcalc: [ERROR] error converting string "-" to complex float (no valid conversion)
              mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
              mrconvert: [ERROR] error opening image "-"
dwi2response:
dwi2response: [ERROR] For debugging, inspect contents of scratch directory: /Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-WXOI4H/
dwi2response: Scratch directory retained; location: /Volumes/CAMRI/Kio_project/Test_02/dwi2response-tmp-WXOI4H/

Can you hep me what this error mean and how I can fix it?

Regards,
Maryam

[Lestropie]

Hi Maryam,

The specific error occurs because it is trying to select the bottom 2% of just 13 voxels, which is ... zero.
The origin of the problem is quite a lot earlier:

dwi2response: Crude segmentation:
dwi2response: * Crude WM versus GM-CSF separation (at FA=0.2)...
dwi2response:   [ 413063 -> 413013 (WM) & 50 (GM-CSF) ]

I am assuming that the fraction of your image that actually consists of GM & CSF is more than 0.01%. So you would want to first take a look at the FA image produced from your data and see if that indicates that there is some larger problem at hand (i.e. why is it that almost all of the GM & CSF in the image has an FA greater than 0.2?), and second (assuming no such fundamental flaw exists) consider using the -fa option to use a higher threshold for crude separation of WM from GM/CSF for your particular data.

Rob