Update nightly #136

anthonyprinaldi · 2024-07-03T14:36:22Z

Merge all recent changes from dev (ACE related) into nightly release
This is in preparation for testing the new Python version in the nightly branch

This fix makes sure that the right atlas is selected by the registration script, instead of the workflow. This is needed for when using half brain registration.

Add registration overwrite ability to single workflow.

Clear reg_final/ and clar_allen_reg/ when user choses to rerun registration

Add parser flag to skip segmentation when the segmentation has already been run.

Seg skipping was only added to comparison workflow and not single workflow.

In Tiff to Nifti allow the user to specify a value of percentile to correct extreme artifacts in the image. Default no correction.

Let ACE use the percentil correction in Tiff to Nifti conversion to crop extreme intensities.

This was the problem with the registration function. We also no longer need to perform the percentile correction.

If labels are specified, use the user specified argument. Otherwise, come up with the labels based on other arguments.

sw_batch_size is hard coded to 4. batch_size is now taken from user input.

This allows us to specify the number of MC iterations instead of hard coding the value.

Need to move results to CPU before doing numpy stackin. Otherwise results in an error

New parser argument to select number of GPU to be used.

Deal with outputs when batch size is greater than 1.

Ensure that the variable of the outside loop doesn't conflict with the list comprehension

Need to keep original values of val_outputs for future loop iterations.

Statements originally included for debugging no longer needed.

Add flag for forcing rerun conversion. By default conversion will be skipped if it already completed and the right downsample arg is used.

The stacked patches are only beign zero padded in the height and width dimension right now. We need to pad the depth dimension so that all patches loaded to the model are the same size.

…t and width

The warp script does not have ability to specify custom allen labels (in the case of half-brain registratin). Add the flag that overwrite the default allen atlas labels.

The output of MC ace models were being saved with additional uncertaintiy information. To handle this, we need to filter the output seg folder for specific files. These files re named different for regular vs MC ace models.

Sometimes a None string is passed through the CLI that we must check for, otherwise the script will try and load the path None. We check if the label path is not empty and not None

Ace heatmap used to be for whole brain only. This commit adds logic to load in half brain atlas labels if half brain is passed to the ACE workflow. This also adds a flag for heatmap that masks out voxels that our outside of the brain, as determined by the atlas.

The model output by default will determine if a voxel is a neuron with 0.5 threshold. We now add an argument so we can change this threshold to be whatever we want.

Previously, creating output storage arrays based on the assumed patch size of 512^3. We now take the patch size from the input so that it can work with any patch size.

Function will plot the p-values from ACE clusterwise analysis and overlay it onto a brain template from the atlas.

ACE flow will now plot a p-value overlaid on an atalas after clusterwise analysis is complete.

Add validate clusters and instance segmentation output to ace workflow

Second Round of ACE Fixes.

Add ACE Finetuning Script

anthonyprinaldi added 30 commits February 27, 2024 19:07

[fix] Change default atlas labels for ace reg to None

0db3290

This fix makes sure that the right atlas is selected by the registration script, instead of the workflow. This is needed for when using half brain registration.

[feature] Add vox and warp to single ace workflow

92562eb

[fix] Add reg checker to single workflow

15b39b4

Add registration overwrite ability to single workflow.

[fix] Clear reg dirs when overwrite

42202b1

Clear reg_final/ and clar_allen_reg/ when user choses to rerun registration

[feature] Add ability to skip segmentation

0a9e5ce

Add parser flag to skip segmentation when the segmentation has already been run.

[fix] Add seg skipping for single ace workflow

11574c1

Seg skipping was only added to comparison workflow and not single workflow.

[feature] Add percentile thresholding for conversion

863be85

In Tiff to Nifti allow the user to specify a value of percentile to correct extreme artifacts in the image. Default no correction.

[feature] Add extreme value percentile correction to ACE

1e8256d

Let ACE use the percentil correction in Tiff to Nifti conversion to crop extreme intensities.

[wip] Update printing statement

3956999

[wip] Try removing c3d data type conversion

0669d97

This was the problem with the registration function. We also no longer need to perform the percentile correction.

[fix] Remove overwriting labels argument

971a483

If labels are specified, use the user specified argument. Otherwise, come up with the labels based on other arguments.

[fix] Update batch size and sw batch size

974dd76

sw_batch_size is hard coded to 4. batch_size is now taken from user input.

[feature] Chane MC flag to integer

446d2f7

This allows us to specify the number of MC iterations instead of hard coding the value.

[fix] Move MC results to cpu

e34fcad

Need to move results to CPU before doing numpy stackin. Otherwise results in an error

[feature] Add GPU indexing for parallel running of ace workflow

bb9a2de

New parser argument to select number of GPU to be used.

[fix] Small bugfix for ensemble model

092b50c

Deal with outputs when batch size is greater than 1.

[fix] Change variable used in list comprehension

b18d48a

Ensure that the variable of the outside loop doesn't conflict with the list comprehension

[fix] Overwriting val_outputs in loop

f8c7f30

Need to keep original values of val_outputs for future loop iterations.

[fix] Remove print statements

c129b12

Statements originally included for debugging no longer needed.

[feature] Add option to skip conversion

5bdff24

Add flag for forcing rerun conversion. By default conversion will be skipped if it already completed and the right downsample arg is used.

[fix] Add zero padding to the depth dimension of patches

a8d0e55

The stacked patches are only beign zero padded in the height and width dimension right now. We need to pad the depth dimension so that all patches loaded to the model are the same size.

[fix] Fix the calculation for the number of patches that fit in heigh…

65f8d0f

…t and width

[feature] Add labels flag for warping clar to allen

22819dd

The warp script does not have ability to specify custom allen labels (in the case of half-brain registratin). Add the flag that overwrite the default allen atlas labels.

[fix] Add logic to handle MC ACE models

1a32000

The output of MC ace models were being saved with additional uncertaintiy information. To handle this, we need to filter the output seg folder for specific files. These files re named different for regular vs MC ace models.

[fix] Add check for None label in warp

76657f1

Sometimes a None string is passed through the CLI that we must check for, otherwise the script will try and load the path None. We check if the label path is not empty and not None

[feature] Add parser argument for binarization threshold of model output

39e3c57

The model output by default will determine if a voxel is a neuron with 0.5 threshold. We now add an argument so we can change this threshold to be whatever we want.

[fix] Create new arrays based on input size

630d048

Previously, creating output storage arrays based on the assumed patch size of 512^3. We now take the patch size from the input so that it can work with any patch size.

[feature] Add function for pvalue plotting

41427cd

Function will plot the p-values from ACE clusterwise analysis and overlay it onto a brain template from the atlas.

[feature] Add pvalue plotting to ACE

9bc71ac

ACE flow will now plot a p-value overlaid on an atalas after clusterwise analysis is complete.

anthonyprinaldi and others added 30 commits August 15, 2024 12:23

[fix] Udpdate ace stats default values

370fca4

[format] Organize imports

622dd86

[fix] Update parser details and usage

f1de6e8

[fix] Handle args better

91ad5dd

[fix] Modularize image loading

11743d8

[fix] Modularize transforms

318aedd

[fix] Modularize dataloaders

3e43a99

[fix] Modularize model building

14d13f9

[fix] Modularize optimizers

8668506

[fix] Modularize train loop

fd3b27e

[format] Format entire script

b72c372

[fix] Remove dynamic unet no longer used

5f64fe3

[fix] Correct default model filepath based on docker/sing structure

127291d

[fix] Add torchio req for finetune

6525fe7

[feat] Add ace_finetune to miracl seg CLI

69f1f38

[fix] Remove bug in data transforms

e0349de

[fix] Update model_state default to trained

e66eeac

[feat] Update default config values and add options

d011642

[docs] Add ACE finetune usage to docs

b73f00e

[fix] Add new ace features to CLI usage

e2d8595

[docs] Add new ace parser flags to docs

7705c19

[fix] Spelling mistake

f2515ca

[fix] Increase verbosity of parser

f7785ea

[docs] Add instance seg documentation

4a758e9

[docs] Add neuron_count documentation

ff144c4

[docs] Update index for segmentation

6ab228f

[docs] Update ace workflow output files

83ca5bc

Add validate clusters and instance segmentation output to ace workflow

[feat/fix] Merge pull request #152 from AICONSlab/ar-ace_fixes_v2

5f1f071

Second Round of ACE Fixes.

Merge branch 'dev' into ar/ace-finetune

a769354

[feat] Merge pull request #153 from AICONSlab/ar/ace-finetune

3ca8443

Add ACE Finetuning Script

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update nightly #136

Update nightly #136

anthonyprinaldi commented Jul 3, 2024

Update nightly #136

Are you sure you want to change the base?

Update nightly #136

Conversation

anthonyprinaldi commented Jul 3, 2024