NF improve converter and utils interface that can be handled #11

htwangtw · 2021-04-15T15:19:46Z

Changes to converter.py:

add default help string when nipype1 provides none
inputs_drop allow converter dropping input specs that is not compatible with pydra
ensure name_source grabbed from nipype1 is a list
allow using callable when encountering nipype1 genfile in input spec

Remaining issues for future PRs:

implement parser for argstr: a series of float ("%f %f") and %g
parser for plain text output (possibly a pydra general function; see AvScale)
expand test data type (ConvertXFM)
Some dropped input spec should be implemented as pydra task
- fwhm to sigma conversion (Smooth)
- number pair to proper input (ExtractROI)

To-do:

fix docstring path issue
find test data?

update from upstream

* 'master' of https://github.com/nipype/pydra-fsl: MNT: Find namespace packages FIX: Find packages correctly Update README.md using lru_cache for all_interface adding specs dir to the python path in the converter Update tools/converter.py updating readme using all as a default option for interface_name in cli; checking name of modules and available interfaces updating readme adding cli to the converter

Still need to work on the test * fix-converter: fix yml module name input

Still need to understand how test conversion works. Need to add customised components to converter

FIX: Add help_string by default

Pydra callable only applies for output manipulation, but fslmaths requires a output file name. Nipype autogenerate a out_file name when there's no input. Not sure how to resolve this in pydra.

Allow converter to drop a given list of nipype interface parameter. Need to add test to check the dropped parameter that's not a requirement of the fsl command.

… fslmaths * 'fslmaths' of https://github.com/htwangtw/pydra-fsl: ADD converter yml field inputs_drop BK fslmaths requires out_file field FIX: Add help_string by default TEST/BK test generation unsuccessful ADD template for all FSL utils ADD test in ImageMaths spec fix yml module name input FIX converter module name BK utils yml test converter ADD first try on fslmaths interface

Possible implementation: pydra callable Need to check how to do it the pydra way and possibly implement in the converter

Logging interfaces with tests passed for now: * copygeom * extractroi * filterregressor * imagemeants * imagestats * smooth

attribute name_source in nipype can be string or list. Make sure when pass to the converter, it's a list

Remaining issues: * implement parser for argstr: a series of float ("%f %f") and %g * parser for plain text output (possibly a pydra general function; see AvScale) * expand test data type (ConvertXFM)

Quotation mark in the converter doctest generator produces error for filerregressor.py

djarecka · 2021-05-13T03:38:57Z

@htwangtw - I was checking the test that is failing - you should probably use "sep": " ,", in the input_spec, however it won't work with "argstr": "-f '{filter_columns}'". There is one thing to fix in pydra, and I'm planning to work on this tomorrow, but I'm also not completely sure why are you using '{filter_columns}'. Does the command line requires '' around 1,2,3?

htwangtw · 2021-05-13T12:06:06Z

Thanks for looking into this!

I was writing the test case according to the doc:

Usage: 
fsl_regfilt -i <input> -d <design> -f <component numbers or filter threshold> -o <out> [options]

...
Optional arguments (You may optionally specify one or more of):
	-m,--mask	mask image file name
	-f,--filter	filter out part of the regression model, e.g. -f "1,2,3"

It might be fine to remove the "?

djarecka · 2021-05-13T14:20:47Z

@effigies - I'm confused about nipype interface FAST: in_files use InputMultiPath and have no sep defined. Shouldn't this mean that the input value is always returned as a list?

The test shows that it returns a single value, but I'm not sure why this is a single value if no sep was defined.

satra · 2021-05-13T14:49:35Z

because of this: https://github.com/nipy/nipype/blob/6c060304f380c46b2f05c5afdc7171dbbdfadc58/nipype/interfaces/base/core.py#L850

djarecka · 2021-05-13T15:37:58Z

oh, ok, so there is default sep value. Do you think we should have in pydra?

djarecka · 2021-05-15T01:57:13Z

@htwangtw - I've fix the bug in pydra: nipype/pydra#466
but I'm still not sure if the command should have "" around 1,2,3

htwangtw · 2021-05-15T11:11:21Z

I will update my pydra version to the latest and have a look. I will have a look into the original command

htwangtw · 2021-05-20T14:15:40Z

The command itself doesn't require " for the input. That was added because that was how this interface was written in nipype. I will add this to my list of "Interface Needing Manual Edit After Conversion"

FIX - * FAST doc test BREAK - * fNIRT - assertion error with output file * SUSAN - run time error; possibly fixable with more suitable test data

htwangtw · 2021-05-27T14:17:13Z

Note for @djarecka: Interfaces I worked on all passed the test - this PR is ready to merge if we only want to handle the util module. I can open issues for the remaining issue in the preprocess module. What do you think?

Edit: SUSAN and FNIRT are failing locally now - might be related to my FSL version.

htwangtw and others added 24 commits February 23, 2021 15:21

Merge pull request #1 from nipype/master

658a955

update from upstream

ADD first try on fslmaths interface

f5450d1

test converter

e0438f8

merge

b9f0f75

BK utils yml

ba9ef77

FIX converter module name

5ca66ca

fix yml module name input

637c525

ADD ImageMaths

8e33341

Still need to work on the test * fix-converter: fix yml module name input

ADD test in ImageMaths spec

d854f99

ADD template for all FSL utils

997f1aa

TEST/BK test generation unsuccessful

7833167

Still need to understand how test conversion works. Need to add customised components to converter

FIX: Add help_string by default

2b23146

Merge pull request #2 from effigies/fslmaths

430bcda

FIX: Add help_string by default

BK fslmaths requires out_file field

c5a8b22

Pydra callable only applies for output manipulation, but fslmaths requires a output file name. Nipype autogenerate a out_file name when there's no input. Not sure how to resolve this in pydra.

ADD converter yml field inputs_drop

b27f701

Allow converter to drop a given list of nipype interface parameter. Need to add test to check the dropped parameter that's not a requirement of the fsl command.

ADD first try on fslmaths interface

81a1d2b

BREAK utils.smooth need a fwhm to sigma conversion

2ae3f0b

Possible implementation: pydra callable Need to check how to do it the pydra way and possibly implement in the converter

NF add some of the util interface

6a32251

Logging interfaces with tests passed for now: * copygeom * extractroi * filterregressor * imagemeants * imagestats * smooth

NF interface utils.smooth

3a303f9

FIX let converter detect nipype inconsistency

b0b9cd9

attribute name_source in nipype can be string or list. Make sure when pass to the converter, it's a list

ADD all utils interfaces the current converter can handle

0cde464

Remaining issues: * implement parser for argstr: a series of float ("%f %f") and %g * parser for plain text output (possibly a pydra general function; see AvScale) * expand test data type (ConvertXFM)

ADD empty test data and fix doc test

495f97a

Quotation mark in the converter doctest generator produces error for filerregressor.py

htwangtw mentioned this pull request Apr 22, 2021

Doc string not created correctly when quotation marks are present #12

Closed

htwangtw added 5 commits May 27, 2021 13:30

Update pydra version and re run the converter

b71b654

FIX fsl_regfilt doctest

ffea51f

manually change doc

cbfda88

FIX FAST doc test

df4bc0d

FIX/BREAK/TEST preprocess interface failed

0746d11

FIX - * FAST doc test BREAK - * fNIRT - assertion error with output file * SUSAN - run time error; possibly fixable with more suitable test data

TEST FAST doc test works on my enviornment

f43cccc

djarecka merged commit 51480b3 into nipype:master Jun 10, 2021

yibeichan mentioned this pull request Jul 7, 2022

input_drops in yml file #34

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NF improve converter and utils interface that can be handled #11

NF improve converter and utils interface that can be handled #11

htwangtw commented Apr 15, 2021 •

edited

Loading

djarecka commented May 13, 2021

htwangtw commented May 13, 2021

djarecka commented May 13, 2021

satra commented May 13, 2021

djarecka commented May 13, 2021

djarecka commented May 15, 2021

htwangtw commented May 15, 2021

htwangtw commented May 20, 2021

htwangtw commented May 27, 2021 •

edited

Loading

NF improve converter and utils interface that can be handled #11

NF improve converter and utils interface that can be handled #11

Conversation

htwangtw commented Apr 15, 2021 • edited Loading

djarecka commented May 13, 2021

htwangtw commented May 13, 2021

djarecka commented May 13, 2021

satra commented May 13, 2021

djarecka commented May 13, 2021

djarecka commented May 15, 2021

htwangtw commented May 15, 2021

htwangtw commented May 20, 2021

htwangtw commented May 27, 2021 • edited Loading

htwangtw commented Apr 15, 2021 •

edited

Loading

htwangtw commented May 27, 2021 •

edited

Loading