Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python CLI changes #2678

Merged
merged 84 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
cd8bbfb
Created custom callables for different datatypes on cmd
ankitasanil Feb 6, 2023
075db52
Used callable types in commands - mrtrix_cleanup and dwicat
ankitasanil Feb 6, 2023
8d91a38
Changes done as instructed in the review
ankitasanil Feb 7, 2023
ebd36cf
Used callable types in few more commands
ankitasanil Feb 7, 2023
2988229
Python API: Integrate argument types across commands
Lestropie Feb 7, 2023
481462f
Changed the syntax as per https://github.com/ankitasanil/mrtrix3/pull…
ankitasanil Feb 7, 2023
54bca48
Python API: Used Python list comprehension as per https://github.com/…
ankitasanil Feb 7, 2023
cd33930
population_template: Refine cmdline interface
Lestropie Feb 7, 2023
9f3d624
Change interfaces for robust linear registration estimators
Lestropie Feb 10, 2023
51ccc67
Python API: Updated class names as per https://github.com/ankitasanil…
ankitasanil Feb 13, 2023
47fec95
Modifications in existing code to handle new argument types
ankitasanil Feb 13, 2023
b72c211
Modifications in existing code to handle new argument types (contd.)
ankitasanil Feb 14, 2023
6dcfe53
Added inheritance for TracksIn checks as per (https://github.com/anki…
ankitasanil Feb 14, 2023
935a8aa
Python API: Ability to handle piped commands
ankitasanil Mar 7, 2023
e749fe9
Changes for output image piping
ankitasanil Mar 14, 2023
a830bf4
Python CLI: Tweaks to custom types
Lestropie Jul 12, 2023
4560880
Python CLI: Code cleanup
Lestropie Jul 13, 2023
2894b1d
Move handling of envvar MRTRIX_PRESERVE_TMPFILE
Lestropie Aug 10, 2023
294ea57
Fix pylint warnings for python_cmd_changes branch
Lestropie Aug 10, 2023
70c154e
Python: Move some functions from path to utils module
Lestropie Aug 10, 2023
e9393d7
First functional version of image piping with Python scripts
Lestropie Aug 10, 2023
913ce00
mrtrix3.app: Terminate if piping image to stdout
Lestropie Aug 10, 2023
2e1efdb
mrtrix3.app: Improvements to __print_full_usage__ behaviour
Lestropie Aug 11, 2023
ff56d40
dwibiascorrect ants: Change command-line options
Lestropie Aug 11, 2023
a847938
mrtrix3.app: Fix __print_full_usage__ for subparser algorithms
Lestropie Aug 11, 2023
94abafa
Docs: Improved metavar usage in Python scripts
Lestropie Aug 11, 2023
dda5db7
Merge branch 'dev' into python_cmd_changes
Lestropie Aug 11, 2023
1a9759e
dwi2mask consensus: Generalise -algorithm interface
Lestropie Aug 11, 2023
a22382f
dwi2mask legacy: Make use of Python command image piping
Lestropie Aug 11, 2023
7f27e89
Python API: Ranged integers and floats
Lestropie Jan 30, 2024
18e6850
Python: Broad fixing of typed command-line argument usage
Lestropie Feb 4, 2024
d511e3d
Python API: Multiple related changes
Lestropie Feb 13, 2024
b31c9d9
New unit test for Python CLI typing
Lestropie Feb 14, 2024
7791a38
Fix Python help pages for bound integers & floats
Lestropie Feb 18, 2024
4da90a0
Python API: Multiple API/CLI changes
Lestropie Feb 18, 2024
4c74759
Testing: Verify piped image support for Python scripts
Lestropie Feb 19, 2024
ff86c44
Python API: Fix __print_full_usage__ for algorithm wrappers
Lestropie Feb 19, 2024
24f1c32
Fix Python issues identified by latest tests
Lestropie Feb 19, 2024
6778522
Merge remote-tracking branch 'origin/dev' into python_cmd_changes
Lestropie Feb 28, 2024
a483d00
Python: Code formatting & documentation regeneration
Lestropie Feb 28, 2024
9822d58
Python CLI: Better metavar handling
Lestropie Feb 28, 2024
d158891
clang-format and clang-tidy fixes for #2678
Lestropie Feb 28, 2024
d5a4596
Add unit testing of Python CLI
Lestropie Feb 28, 2024
4c331e2
Merge remote-tracking branch 'origin/dev' into python_cmd_changes
Lestropie Feb 28, 2024
9cee5ae
dwicat: Use F-strings
Lestropie Feb 28, 2024
d783f8c
population_template: Resolve merge conflicts
Lestropie Feb 28, 2024
55929b0
5ttgen: Algorithms return path of final image to test using 5ttcheck
Lestropie Feb 29, 2024
db3ca00
Resolution of pylint errors for #2678
Lestropie Feb 29, 2024
ff983de
Python CLI: Fixes to typing & alternative typing interface
Lestropie Mar 1, 2024
553a4fb
Python CLI: Move relevant functions / classes into Parser class
Lestropie Mar 1, 2024
e25e977
Fix for population_template for #2678
Lestropie Mar 1, 2024
23f436d
Python API: Fixes to typed CLI
Lestropie Mar 4, 2024
45738a2
Python CLI: Default flags to None
Lestropie Mar 4, 2024
2e6c28a
Merge remote-tracking branch 'origin/dev' into python_cmd_changes
Lestropie Mar 5, 2024
e98ecc6
Testing: Finalise Python CLI testing
Lestropie Mar 5, 2024
40b7a7c
population_template: Syntax fixes
Lestropie Mar 5, 2024
8a0d639
Python CLI: Fix for __print_usage_markdown__
Lestropie Mar 5, 2024
e384c18
Add argument to specify environment for bash tests
daljit46 Mar 5, 2024
7790fd3
testing: Set PYTHONPATH for non-cpp-standalone unit tests
Lestropie Mar 5, 2024
d9587e6
Testing: DO not use sed in Python CLI tests
Lestropie Mar 6, 2024
8065803
Testing: Add C++ CLI evaluation
Lestropie Mar 11, 2024
be75618
CI: Fix PYTHONPATH setting in MSYS2
Lestropie Mar 11, 2024
08bc526
CI: Fix CLI dunder function checks on MSYS2
Lestropie Mar 11, 2024
e014b24
Merge remote-tracking branch 'origin/python_cmd_changes' into fix_pyt…
Lestropie Mar 11, 2024
4c9247b
CI: Fix C++ CLI tests on MSYS2
Lestropie Mar 11, 2024
f57bbd8
Merge pull request #2846 from MRtrix3/fix_python_cli_ci_test
Lestropie Mar 11, 2024
d6a84ba
CI: Final fixes for addition of C++ CLI tests
Lestropie Mar 12, 2024
9ce93f3
Merge remote-tracking branch 'origin/dev' into python_cmd_changes
Lestropie May 13, 2024
65ba856
Testing: Greater testing of Python image piping support
Lestropie May 13, 2024
99dd927
mask2glass: Ensure input image is 3D
Lestropie May 13, 2024
7115593
Testing: Verify that Python commands handle paths with whitespaces
Lestropie May 13, 2024
4095d76
Testing: Import new test data for new mask2glass tests
Lestropie May 13, 2024
05b68d5
Python: Multiple fixes following merge of split tests & CLI changes
Lestropie May 14, 2024
69d3590
Merge remote-tracking branch 'origin/dev' into python_cmd_changes
Lestropie May 15, 2024
92f9a89
5ttgen hsvs: Fixes to whitespace handling changes
Lestropie May 16, 2024
fc4b798
population_template: Fix handling of output directory paths
Lestropie May 16, 2024
f34730d
dwi2mask consensus: Fix for whitespace in scratch directory path
Lestropie May 16, 2024
2a55a43
Testing: Whitespaces in scratch directory paths
Lestropie May 17, 2024
ef5b8c5
mrtrix3.app: Fix argparse exception on existing output filesystem path
Lestropie May 18, 2024
40fdc8e
Testing: Fix execution of bash unit tests
Lestropie May 18, 2024
b3d98c1
Testing: Add comments to CLI tests
Lestropie May 18, 2024
4d8f766
Python CLI & testing fixes
Lestropie May 18, 2024
de5ebe7
Docs: Auto update for changes to -scratch option
MRtrixBot May 20, 2024
c2b2e76
Merge branch 'dev' into python_cmd_changes
Lestropie May 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,6 @@
.check_syntax.tmp
.check_syntax2.tmp
build/
CMakeLists.txt.user
CMakeLists.txt.user
testing/data/tmp*
testing/data/*-tmp-*
7 changes: 4 additions & 3 deletions cmake/BashTests.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ function(add_bash_test)
message(FATAL_ERROR "bash not found")
endif()

set(singleValueArgs FILE_PATH PREFIX WORKING_DIRECTORY)
set(singleValueArgs FILE_PATH PREFIX WORKING_DIRECTORY ENVIRONMENT)
set(multiValueArgs EXEC_DIRECTORIES LABELS)
cmake_parse_arguments(
ARG
Expand All @@ -20,6 +20,7 @@ function(add_bash_test)
set(prefix ${ARG_PREFIX})
set(working_directory ${ARG_WORKING_DIRECTORY})
set(exec_directories ${ARG_EXEC_DIRECTORIES})
set(environment ${ARG_ENVIRONMENT})
set(labels ${ARG_LABELS})

# Regenerate tests when the test script changes
Expand Down Expand Up @@ -47,11 +48,11 @@ function(add_bash_test)
)
set_tests_properties(${test_name}
PROPERTIES
ENVIRONMENT "PATH=${exec_directories}"
ENVIRONMENT "PATH=${exec_directories};${environment}"
)
if(labels)
set_tests_properties(${test_name} PROPERTIES LABELS "${labels}")
endif()

message(VERBOSE "Add bash tests commands for ${test_name}: ${line}")
message(VERBOSE "Added bash test command ${test_name}")
endfunction()
5 changes: 4 additions & 1 deletion cmd/mrregister.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -584,8 +584,11 @@ void run() {
case 2:
affine_estimator = Registration::LP;
break;
default:
case 3:
affine_estimator = Registration::None;
break;
default:
assert(false);
}
}

Expand Down
6 changes: 2 additions & 4 deletions core/app.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -696,10 +696,8 @@ std::string markdown_usage() {
s += std::string(REFERENCES[i]) + "\n\n";
s += std::string(MRTRIX_CORE_REFERENCE) + "\n\n";

s += std::string("---\n\nMRtrix ") + mrtrix_version + ", built " + build_date +
"\n\n"
"\n\n**Author:** " +
AUTHOR + "\n\n**Copyright:** " + COPYRIGHT + "\n\n";
s += std::string("**Author:** ") + AUTHOR + "\n\n";
s += std::string("**Copyright:** ") + COPYRIGHT + "\n\n";

return s;
}
Expand Down
15 changes: 14 additions & 1 deletion core/image_io/pipe.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,19 @@ void Pipe::unload(const Header &) {
}
}

bool Pipe::delete_piped_images = true;
// ENVVAR name: MRTRIX_PRESERVE_TMPFILE
// ENVVAR This variable decides whether the temporary piped image
// ENVVAR should be preserved rather than the usual behaviour of
// ENVVAR deletion at command completion.
// ENVVAR For example, in case of piped commands from Python API,
// ENVVAR it is necessary to retain the temp files until all
// ENVVAR the piped commands are executed.
namespace {
bool preserve_tmpfile() {
const char *const MRTRIX_PRESERVE_TMPFILE = getenv("MRTRIX_PRESERVE_TMPFILE");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function is not thread safe [concurrency-mt-unsafe]

    const char* const MRTRIX_PRESERVE_TMPFILE = getenv("MRTRIX_PRESERVE_TMPFILE");
                                                ^

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function is not thread safe [concurrency-mt-unsafe]

  const char *const MRTRIX_PRESERVE_TMPFILE = getenv("MRTRIX_PRESERVE_TMPFILE");
                                              ^

return (MRTRIX_PRESERVE_TMPFILE != nullptr && to<bool>(std::string(MRTRIX_PRESERVE_TMPFILE)));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: implicit conversion 'const char *' -> bool [readability-implicit-bool-conversion]

Suggested change
return (MRTRIX_PRESERVE_TMPFILE != nullptr && to<bool>(std::string(MRTRIX_PRESERVE_TMPFILE)));
return ((MRTRIX_PRESERVE_TMPFILE != nullptr) && to<bool>(std::string(MRTRIX_PRESERVE_TMPFILE)));

}
} // namespace
bool Pipe::delete_piped_images = !preserve_tmpfile();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: variable 'delete_piped_images' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

bool Pipe::delete_piped_images = !preserve_tmpfile();
           ^


} // namespace MR::ImageIO
42 changes: 21 additions & 21 deletions docs/reference/commands/5ttgen.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Usage
Description
-----------

5ttgen acts as a 'master' script for generating a five-tissue-type (5TT) segmented tissue image suitable for use in Anatomically-Constrained Tractography (ACT). A range of different algorithms are available for completing this task. When using this script, the name of the algorithm to be used must appear as the first argument on the command-line after '5ttgen'. The subsequent compulsory arguments and options available depend on the particular algorithm being invoked.
5ttgen acts as a "master" script for generating a five-tissue-type (5TT) segmented tissue image suitable for use in Anatomically-Constrained Tractography (ACT). A range of different algorithms are available for completing this task. When using this script, the name of the algorithm to be used must appear as the first argument on the command-line after "5ttgen". The subsequent compulsory arguments and options available depend on the particular algorithm being invoked.

Each algorithm available also has its own help page, including necessary references; e.g. to see the help page of the 'fsl' algorithm, type '5ttgen fsl'.
Each algorithm available also has its own help page, including necessary references; e.g. to see the help page of the "fsl" algorithm, type "5ttgen fsl".

Options
-------
Expand All @@ -39,9 +39,9 @@ Additional standard options for Python scripts

- **-nocleanup** do not delete intermediate files during script execution, and do not delete scratch directory at script completion.

- **-scratch /path/to/scratch/** manually specify the path in which to generate the scratch directory.
- **-scratch /path/to/scratch/** manually specify an existing directory in which to generate the scratch directory.

- **-continue <ScratchDir> <LastFile>** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.
- **-continue ScratchDir LastFile** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.

Standard options
^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -107,16 +107,16 @@ Usage

5ttgen freesurfer input output [ options ]

- *input*: The input FreeSurfer parcellation image (any image containing 'aseg' in its name)
- *input*: The input FreeSurfer parcellation image (any image containing "aseg" in its name)
- *output*: The output 5TT image

Options
-------

Options specific to the 'freesurfer' algorithm
Options specific to the "freesurfer" algorithm
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- **-lut** Manually provide path to the lookup table on which the input parcellation image is based (e.g. FreeSurferColorLUT.txt)
- **-lut file** Manually provide path to the lookup table on which the input parcellation image is based (e.g. FreeSurferColorLUT.txt)

Options common to all 5ttgen algorithms
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -130,9 +130,9 @@ Additional standard options for Python scripts

- **-nocleanup** do not delete intermediate files during script execution, and do not delete scratch directory at script completion.

- **-scratch /path/to/scratch/** manually specify the path in which to generate the scratch directory.
- **-scratch /path/to/scratch/** manually specify an existing directory in which to generate the scratch directory.

- **-continue <ScratchDir> <LastFile>** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.
- **-continue ScratchDir LastFile** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.

Standard options
^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -204,12 +204,12 @@ Usage
Options
-------

Options specific to the 'fsl' algorithm
Options specific to the "fsl" algorithm
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- **-t2 <T2 image>** Provide a T2-weighted image in addition to the default T1-weighted image; this will be used as a second input to FSL FAST
- **-t2 image** Provide a T2-weighted image in addition to the default T1-weighted image; this will be used as a second input to FSL FAST

- **-mask** Manually provide a brain mask, rather than deriving one in the script
- **-mask image** Manually provide a brain mask, rather than deriving one in the script

- **-premasked** Indicate that brain masking has already been applied to the input image

Expand All @@ -225,9 +225,9 @@ Additional standard options for Python scripts

- **-nocleanup** do not delete intermediate files during script execution, and do not delete scratch directory at script completion.

- **-scratch /path/to/scratch/** manually specify the path in which to generate the scratch directory.
- **-scratch /path/to/scratch/** manually specify an existing directory in which to generate the scratch directory.

- **-continue <ScratchDir> <LastFile>** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.
- **-continue ScratchDir LastFile** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.

Standard options
^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -319,9 +319,9 @@ Additional standard options for Python scripts

- **-nocleanup** do not delete intermediate files during script execution, and do not delete scratch directory at script completion.

- **-scratch /path/to/scratch/** manually specify the path in which to generate the scratch directory.
- **-scratch /path/to/scratch/** manually specify an existing directory in which to generate the scratch directory.

- **-continue <ScratchDir> <LastFile>** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.
- **-continue ScratchDir LastFile** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.

Standard options
^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -393,11 +393,11 @@ Usage
Options
-------

- **-template** Provide an image that will form the template for the generated 5TT image
- **-template image** Provide an image that will form the template for the generated 5TT image

- **-hippocampi** Select method to be used for hippocampi (& amygdalae) segmentation; options are: subfields,first,aseg
- **-hippocampi choice** Select method to be used for hippocampi (& amygdalae) segmentation; options are: subfields,first,aseg

- **-thalami** Select method to be used for thalamic segmentation; options are: nuclei,first,aseg
- **-thalami choice** Select method to be used for thalamic segmentation; options are: nuclei,first,aseg

- **-white_stem** Classify the brainstem as white matter

Expand All @@ -413,9 +413,9 @@ Additional standard options for Python scripts

- **-nocleanup** do not delete intermediate files during script execution, and do not delete scratch directory at script completion.

- **-scratch /path/to/scratch/** manually specify the path in which to generate the scratch directory.
- **-scratch /path/to/scratch/** manually specify an existing directory in which to generate the scratch directory.

- **-continue <ScratchDir> <LastFile>** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.
- **-continue ScratchDir LastFile** continue the script from a previous execution; must provide the scratch directory path, and the name of the last successfully-generated file.

Standard options
^^^^^^^^^^^^^^^^
Expand Down
Loading
Loading