Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docs 0.1.0dev #75

Merged
merged 3 commits into from
Mar 25, 2024
Merged

Docs 0.1.0dev #75

merged 3 commits into from
Mar 25, 2024

Conversation

nvnieuwk
Copy link
Member

@nvnieuwk nvnieuwk commented Mar 22, 2024 •
edited by mvheetve
Loading

To create the docs, install mkdocs with pip install mkdocs && pip install mkdocs-material

Run mkdocs serve to create a test web page and go to http://127.0.0.1:8000/structural/ to see your web page

Mkdocs documentation: https://www.mkdocs.org/
Mkdocs material documentation: https://squidfunk.github.io/mkdocs-material/reference/

@nvnieuwk nvnieuwk requested a review from mvheetve March 22, 2024 13:34
mvheetve
mvheetve reviewed Mar 25, 2024
View reviewed changes
README.md Outdated Show resolved Hide resolved
mvheetve
mvheetve reviewed Mar 25, 2024
View reviewed changes
docs/index.md Outdated Show resolved Hide resolved
mvheetve
mvheetve reviewed Mar 25, 2024
View reviewed changes
docs/index.md Outdated Show resolved Hide resolved
mvheetve
mvheetve approved these changes Mar 25, 2024
View reviewed changes
Copy link
Collaborator

@mvheetve mvheetve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few comments, but fine by me!

@nvnieuwk nvnieuwk merged commit a4d35e6 into dev Mar 25, 2024
1 of 2 checks passed
@nvnieuwk nvnieuwk deleted the docs-0.1.0dev branch March 25, 2024 13:11
@nvnieuwk nvnieuwk mentioned this pull request Mar 29, 2024
Closed
nvnieuwk added a commit that referenced this pull request Apr 3, 2024
@nvnieuwk @mvheetve
v0.1.0 pipeline release (#80)
dec66ea 
* Birth of a pipeline (#2)

* cleaned up the code and docs before starting

* cleaned up the github actions

* cleaned some more code

* removed unused pipeline code

* added an easily adjustable input checker

* removed some CI

* added the variant callers

* linting

* tests update

* ci update

* linting

* linting

* linting

* turned off actions

* added read counts + delly updates

* added scatter count to delly call

* added versions and multiqc

* updated the manta workflow

* updated the whamg workflow

* added versions to whamg

* added some additional test data

* updated the test data

* updated the test data

* updated the test data

* updated the test data

* overal updates and improvements

* small updates

* small updates

* metrics part 1

* metrics part 2

* Module  evidence qc (#3)

* make bincov matrix part 1

* added ploidy and parts of calcmedcov

* added WGD

* removed delly and added a reheader to MANTA

* added all code to the module (some parts are still broken)

* small updates

* small ploidy matrix fix

* small updates to evidenceQC + re-added delly

* Module gather batch evidence (#4)

* added groundwork for the module

* added groundwork for the module

* more work done

* removed commented lines

* Continue module gather batch evidence (#19)

* printsvevidence update (not done yet)

* restructured the pipeline to match nf-core v2.6

* updated module paths

* some small "improvements"

* Template update for nf-core/tools version 2.6

* additional fixes to the template

* removed wrong file

* Added clarifying comments to GSE

* updated some of the modules to 4.3.0.0

* fixed some issues caused by gatk4.3.0.0

* latest additions

* Refactoring (#20)

* refactored gather sample evidence

* refactor the rest

* New series of fixes and features (#21)

* update whamg

* update local tests

* updated whamg officially

* updated whamg + fixed svtk standardize

* small work on scramble (still no correct data for this tool)

* updated schema

* improved whamg bed file integration

* added a merge vcfs subworkflow

* small fix

* update test config

* added the samplesheet validation and conversion script

* hotfix dev test (#22)

* Add smoove & tests (#23)

* Template update for nf-core/tools version 2.6

* hotfix dev test

* fixed a lot of issues

* added smoove

* updated the schema

* fix samplesheet and schema

* add better outdir folders

* add tests

* tests fix

* fix secrets

* fix typo

* add more tests

* add template yml

* Template update for nf-core/tools version 2.7.2

* fixes after template merge

* linting

* more linting

* even more linting

* black

* Refactor the pipeline to prepare for subworkflow export to nf-core (#24)

* refactored the pipeline

* remove unused modules

* removed unused workflows

* Add gridss (#25)

* refactored the pipeline

* remove unused modules

* removed unused workflows

* add gridss

* add stub to local modules

* fix stubs

* fully add gridss

* add tests

* copy fix?

* add gridss test to ci

* exclude gridss vcfs from metrics (not supported)

* update min nextflow version to fix caching issue

* update gridss

* remove TODOs [no ci]

* Add genotyping (#26)

* update scattering

* update delly

* bump commit

* add --output_callers parameter

* small fix to --output_callers

* fix schema and test config

* removed reheadering from manta

* update bcftools/sort

* update bcftools/sort

* update headers after jasmine merge

* small fix

* finished genotyping

* fix for paragraph/multigrmpy

* updated the samplesheet conversion

* removed some unused modules

* add merging of samples from the same family

* removed scattering from whamg

* Updated jasmine (added support for bgzip to the module)

* fix prettier

* fix tests

* fix tests again and remove black CI

* add linting ci to nf-core.yml

* Adopted SVINSSEQ as uniform ins sequence info key

* disabled whamg for now

* add missing format header

* add patch for delly

* fix header typo

* remove local test profile

* updated test profiles and added big tests

* add delly args

* add new args to schema and config

* Add annotation + viola + join checks + cleanup (#29)

* add join checks

* fix delly mismatches

* Added viola

* fix header issues in single caller runs

* header fix?

* appease eclint

* add stub to reverse_bed

* fix for gridss?

* appease eclint again!

* typo

* fix permissions

* fix shebang

* fix shebang again

* disable gridss

* fix tests

* remove old code from gatk-sv

* Add VEP to the pipeline

* add some QOL parameters

* update tests

* QOL changes (variables and joins)

* disable scattering for delly

* remove unused gatk modules

* update modules

* made BED files optional

* add metro map and update readme

* forgot to save svg

* fix linting

* fix vep test

* free up CI space?

* use another cache for vep

* edit PosCon3 samplesheet_big.csv (#30)

* Add bcftools sort (#31)

* add join checks

* fix delly mismatches

* Added viola

* fix header issues in single caller runs

* header fix?

* appease eclint

* add stub to reverse_bed

* fix for gridss?

* appease eclint again!

* typo

* fix permissions

* fix shebang

* fix shebang again

* disable gridss

* fix tests

* remove old code from gatk-sv

* Add VEP to the pipeline

* add some QOL parameters

* update tests

* QOL changes (variables and joins)

* disable scattering for delly

* remove unused gatk modules

* update modules

* made BED files optional

* add metro map and update readme

* forgot to save svg

* fix linting

* fix vep test

* free up CI space?

* use another cache for vep

* add bcftools sort

* fix smoove output (#33)

* fix smoove output

* update according to other modules

* Fully implement annotation (#35)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* first steps

* fully implement the annotation

* update test

* offer a lamb to the linting gods

* bug fix

* offer a virgin to the linting gods (hope this is enough)

* Add dynamic VCFanno config creation

* go on a pilgrimage to the mountain of the linting gods and make a blood sacrifice

* fix and update annotate test to stub

* update vep container on stub runs

* add postannotations support to the configs

* simplified the vcfanno config code and made it smarter

* added new fields from the annotsv vcf to the default configs

* added a param for the default annotsv file name

* fix small bug with param

* fix small bug with param

* Add all default configs

* sacrificing my firstborn child to the linting gods

* Moved the unused configs to a separate folder

* update configs

* update vep config

* add option to annotate on small variants with AnnotSV (real test files should still be added)

* Remove unused VEP parameters

* fix issues

* add tests for small variants vcfs

* add an option to also output the annotsv outputs

* Merge wrongly placed commits (#40)

* fixes for big tests

* metro map update

* Template update for nf-core/tools version 2.8

* black linting

* update modules

* update modules

* fix local modules

* bump

* eclint

* update genome in docs

* update vcfanno configs retina and rpe

* fix rpe bed file in config

* Cnv support (#50)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* first steps

* fully implement the annotation

* update test

* offer a lamb to the linting gods

* bug fix

* offer a virgin to the linting gods (hope this is enough)

* Add dynamic VCFanno config creation

* go on a pilgrimage to the mountain of the linting gods and make a blood sacrifice

* fix and update annotate test to stub

* update vep container on stub runs

* add postannotations support to the configs

* simplified the vcfanno config code and made it smarter

* added new fields from the annotsv vcf to the default configs

* added a param for the default annotsv file name

* fix small bug with param

* fix small bug with param

* Add all default configs

* sacrificing my firstborn child to the linting gods

* Moved the unused configs to a separate folder

* update configs

* update vep config

* add option to annotate on small variants with AnnotSV (real test files should still be added)

* Remove unused VEP parameters

* fix issues

* add tests for small variants vcfs

* add an option to also output the annotsv outputs

* Merge wrongly placed commits (#40)

* fixes for big tests

* metro map update

* Template update for nf-core/tools version 2.8

* black linting

* update modules

* update modules

* fix local modules

* bump

* eclint

* update genome in docs

* update vcfanno configs retina and rpe

* fix rpe bed file in config

* adjust test to new data

* remove unused data from workflows

* remove simple event annotation

* update modules config

* remove unused modules

* fix compile error

* another compile error

* remove comments in config

* guess what! another compile error

* small fixes to annotsv

* update viola

* circumvent a annotsv bug

* add gridss genotyping

* filter out variants with 0/0 genotype from gridss

* move ref variants filtering to before annotsv

* update tests

* fix linting and add meta to references

* black

* install expansionhunter

* fix merge conflict issues

* fix linting

* fix config

* fix stub

* add expansionhunter

* some concurrency debugging

* add nf-validation support

* revert to using linkedhashmap for now

* fix jasmine issue

* update tests

* fix typos

* fix linting

* there you go editorconfig

* let's make it pretty

* update nf-test config

* fix even more tests

* add concat output and fix tests

* fix schema

* now be nice please

* i'm pretty again

* update big test samplesheet [no ci]

* fix max_time pattern [no ci]

* add bcftools sort to single runs

* only apply viola on gridss VCFs and change prefixes

* remove unused comment [no ci]

* fix input pattern

* improve the efficiency of viola

* black

* added a sort step to gridss

* updated the gridss label

* patch gridss module

* remove estimate_read_length

* remove unused code

* small typo

* add -ends to vcfanno

* add full test config

* fix full test description

* update samplesheet

* add exists tags to the samplesheet schema

* specify advised resources for gridss

* possible fix for gridss OOM?

* remove code duplication

* remove an unused parameter

* added a check for callers support being higher than the amount of sv callers

* update metro map

* increase gridss memory

* try to optimize gridss part 2

* try to optimize gridss part 3

* oops

* oops 2

* oops 3

* update to advised amounts

* increase viola time

* discourage the use of gridss for now

* typo

* linting fix

* rename SV subworkflow

* fix expansionhunter catalog and add qdnaseq ref

* add qdnaseq

* add a check for the qdnaseq reference

* fixed some issues with qdnaseq

* add cram merging and index creation

* update resources for qdnaseq

* update resources for qdnaseq

* update resources for qdnaseq

* fix resources qdnaseq

* add wisecondorx

* updated the metro map

* fix background of metro map

* update resolution

* update resolution

* remove old samplesheet conversion script

* moved available callers to a groovy script

* removed the manual check for file exists (now fully uses nf-validation)

* fix a bug with concat when a cnv caller has been supplied

* added some comments for the last fix

* update optional input handling

* fix small issue with annotsv annotations

* fix a no process matching selector warning

* revert last changes

* temp fix for warning

* improve vep cache handling + update modules

* typo

* fix container declaration

* fix for manta issue?

* undo last commit

* version bump expansionhunter

* added support for manta configs

* made qdnaseq references gender specific

* update manta resources

* add contig input to manta

* moved the contigs file creation to the manta subwf

* fix vep params in schema

* fix linting with no template update

* fix nf-test tests

* fix eclint

* fix eclint

* change gender to sex

* typo

* added the sex arg to expansionhunter

* fix nf-test config for v0.8.0

* revert nf-test to 0.7.3

* prettier

* set default sex to female instead of male

* fix tests samplesheet

* Fix vep container issue (#52)

* add dynamic container fetching for vep

* linting fix

* moved the function to the process itself

* fix conda issue

* eclint

* fix params schema

* fix linting

* fix for vep issue?

* fix for missing singularity container

* Add filtering on annotations + fix linting (#55)

* Add filtering on annotations

* fix linting

* begone trailing whitespace

* add annotation filter to test

* CNV VCFs (#58)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* Family merging (#61)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* add z-score to VCF header

* small fixes to sex defining

* Add proper testing + fix some issues found (#64)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* small clarification in the header

* Fix annotsv TCL error (#69)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* split annotsv input files by svtype and merge them back

* small clarification in the header

* fix bcftools concat issue

* fix linting

* don't lint .prettierignore

* Add Svync standardization + other fixes (#72)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* split annotsv input files by svtype and merge them back

* small clarification in the header

* fix bcftools concat issue

* fix linting

* don't lint .prettierignore

* deprecate gridss for now

* implement delly svync

* implement manta svync

* implement smoove svync

* add better headering to merge callers

* small fix to bcftools annotate

* fix headers for family merging

* remove unused assets

* fix some stability issues

* simplify expansionhunter handling

* reheader with bcftools/reheader again

* fix the headering once and for all

* add a quick view statement

* change view to dump

* rename dump tag

* extra dump

* fix join issue

* fix header issues with annotsv

* add missing header line for annotsv

* fix callers info field

* remove some views

* add views for hpc debugging

* a workaround for double variants in annotsv

* a workaround for double variants in annotsv try 2

* remove view statements

* fix join error?

* fix some linting issues

* don't test gridss

* drop pinned nf-test version

* disable gridss tests for real

* fix some tests

* update subwf tests

* update pipeline tests

* fix annotate tests?

* fix annotate tests again?

* add views to annotation test

* fix join error + fix test jasmine

* revert join error changes?

* another try to fix weird join error

* join error fix try 3

* join error fix try 4

* join error fix try 5

* join error fix try 6

* join error fix try 7

* Template v2.13.1 sync + update all modules (#73)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* update pipeline name

* update test-data repo name

* update test data config name

* update pipeline name

* update pipeline name

* update nf-core yaml

* Template update for nf-core/tools version 2.13.1

* Merge branch 'TEMPLATE' into template-v2.13.1-sync

* fix some linting issues

* update all modules

* fix subwf in modules json

* remove unused code

* remove unused code

* fix jasminesv

* fix eclint

* prettier

* have nf-test in the CI

* fix samtools convert stub

* prettier

* fix tabix outputs + tests

* update test snapshots

* remove views

* rearrange joins

* revert last commit

* another try 🤞

* another try 2 🤞

* try adding by

* fix wrong versions yml

* patch bcftools concat

* fix bcftools concat versions

* fix delly svync config

* use midnighter's approach of joining channels

* fix linting

* fix test

* Create beautiful docs with mkdocs (#74)

* create beautiful docs

* some small fixes

* set dev version to beta version

* Docs 0.1.0dev (#75)

* update docs

* fix linting

* update description of pipeline

* docs-output (#76)

* docs

* parameters.md adapted

* relative links

* comments for docs-output PR

* parameters.md correction

* Last fixes before v0.1.0 release (#77)

* fix tests

* some esthetic fixes

* move the logo in the metro map

* made the metro map logo a bit smaller

* update parameter docs

* bump to version 0.1.0

---------

Co-authored-by: mvheetve <[email protected]>
nvnieuwk added a commit that referenced this pull request Jul 19, 2024
@nvnieuwk @mvheetve
Release PR 0.2.0 (#94)
b4e4028 
* Birth of a pipeline (#2)

* cleaned up the code and docs before starting

* cleaned up the github actions

* cleaned some more code

* removed unused pipeline code

* added an easily adjustable input checker

* removed some CI

* added the variant callers

* linting

* tests update

* ci update

* linting

* linting

* linting

* turned off actions

* added read counts + delly updates

* added scatter count to delly call

* added versions and multiqc

* updated the manta workflow

* updated the whamg workflow

* added versions to whamg

* added some additional test data

* updated the test data

* updated the test data

* updated the test data

* updated the test data

* overal updates and improvements

* small updates

* small updates

* metrics part 1

* metrics part 2

* Module  evidence qc (#3)

* make bincov matrix part 1

* added ploidy and parts of calcmedcov

* added WGD

* removed delly and added a reheader to MANTA

* added all code to the module (some parts are still broken)

* small updates

* small ploidy matrix fix

* small updates to evidenceQC + re-added delly

* Module gather batch evidence (#4)

* added groundwork for the module

* added groundwork for the module

* more work done

* removed commented lines

* Continue module gather batch evidence (#19)

* printsvevidence update (not done yet)

* restructured the pipeline to match nf-core v2.6

* updated module paths

* some small "improvements"

* Template update for nf-core/tools version 2.6

* additional fixes to the template

* removed wrong file

* Added clarifying comments to GSE

* updated some of the modules to 4.3.0.0

* fixed some issues caused by gatk4.3.0.0

* latest additions

* Refactoring (#20)

* refactored gather sample evidence

* refactor the rest

* New series of fixes and features (#21)

* update whamg

* update local tests

* updated whamg officially

* updated whamg + fixed svtk standardize

* small work on scramble (still no correct data for this tool)

* updated schema

* improved whamg bed file integration

* added a merge vcfs subworkflow

* small fix

* update test config

* added the samplesheet validation and conversion script

* hotfix dev test (#22)

* Add smoove & tests (#23)

* Template update for nf-core/tools version 2.6

* hotfix dev test

* fixed a lot of issues

* added smoove

* updated the schema

* fix samplesheet and schema

* add better outdir folders

* add tests

* tests fix

* fix secrets

* fix typo

* add more tests

* add template yml

* Template update for nf-core/tools version 2.7.2

* fixes after template merge

* linting

* more linting

* even more linting

* black

* Refactor the pipeline to prepare for subworkflow export to nf-core (#24)

* refactored the pipeline

* remove unused modules

* removed unused workflows

* Add gridss (#25)

* refactored the pipeline

* remove unused modules

* removed unused workflows

* add gridss

* add stub to local modules

* fix stubs

* fully add gridss

* add tests

* copy fix?

* add gridss test to ci

* exclude gridss vcfs from metrics (not supported)

* update min nextflow version to fix caching issue

* update gridss

* remove TODOs [no ci]

* Add genotyping (#26)

* update scattering

* update delly

* bump commit

* add --output_callers parameter

* small fix to --output_callers

* fix schema and test config

* removed reheadering from manta

* update bcftools/sort

* update bcftools/sort

* update headers after jasmine merge

* small fix

* finished genotyping

* fix for paragraph/multigrmpy

* updated the samplesheet conversion

* removed some unused modules

* add merging of samples from the same family

* removed scattering from whamg

* Updated jasmine (added support for bgzip to the module)

* fix prettier

* fix tests

* fix tests again and remove black CI

* add linting ci to nf-core.yml

* Adopted SVINSSEQ as uniform ins sequence info key

* disabled whamg for now

* add missing format header

* add patch for delly

* fix header typo

* remove local test profile

* updated test profiles and added big tests

* add delly args

* add new args to schema and config

* Add annotation + viola + join checks + cleanup (#29)

* add join checks

* fix delly mismatches

* Added viola

* fix header issues in single caller runs

* header fix?

* appease eclint

* add stub to reverse_bed

* fix for gridss?

* appease eclint again!

* typo

* fix permissions

* fix shebang

* fix shebang again

* disable gridss

* fix tests

* remove old code from gatk-sv

* Add VEP to the pipeline

* add some QOL parameters

* update tests

* QOL changes (variables and joins)

* disable scattering for delly

* remove unused gatk modules

* update modules

* made BED files optional

* add metro map and update readme

* forgot to save svg

* fix linting

* fix vep test

* free up CI space?

* use another cache for vep

* edit PosCon3 samplesheet_big.csv (#30)

* Add bcftools sort (#31)

* add join checks

* fix delly mismatches

* Added viola

* fix header issues in single caller runs

* header fix?

* appease eclint

* add stub to reverse_bed

* fix for gridss?

* appease eclint again!

* typo

* fix permissions

* fix shebang

* fix shebang again

* disable gridss

* fix tests

* remove old code from gatk-sv

* Add VEP to the pipeline

* add some QOL parameters

* update tests

* QOL changes (variables and joins)

* disable scattering for delly

* remove unused gatk modules

* update modules

* made BED files optional

* add metro map and update readme

* forgot to save svg

* fix linting

* fix vep test

* free up CI space?

* use another cache for vep

* add bcftools sort

* fix smoove output (#33)

* fix smoove output

* update according to other modules

* Fully implement annotation (#35)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* first steps

* fully implement the annotation

* update test

* offer a lamb to the linting gods

* bug fix

* offer a virgin to the linting gods (hope this is enough)

* Add dynamic VCFanno config creation

* go on a pilgrimage to the mountain of the linting gods and make a blood sacrifice

* fix and update annotate test to stub

* update vep container on stub runs

* add postannotations support to the configs

* simplified the vcfanno config code and made it smarter

* added new fields from the annotsv vcf to the default configs

* added a param for the default annotsv file name

* fix small bug with param

* fix small bug with param

* Add all default configs

* sacrificing my firstborn child to the linting gods

* Moved the unused configs to a separate folder

* update configs

* update vep config

* add option to annotate on small variants with AnnotSV (real test files should still be added)

* Remove unused VEP parameters

* fix issues

* add tests for small variants vcfs

* add an option to also output the annotsv outputs

* Merge wrongly placed commits (#40)

* fixes for big tests

* metro map update

* Template update for nf-core/tools version 2.8

* black linting

* update modules

* update modules

* fix local modules

* bump

* eclint

* update genome in docs

* update vcfanno configs retina and rpe

* fix rpe bed file in config

* Cnv support (#50)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* first steps

* fully implement the annotation

* update test

* offer a lamb to the linting gods

* bug fix

* offer a virgin to the linting gods (hope this is enough)

* Add dynamic VCFanno config creation

* go on a pilgrimage to the mountain of the linting gods and make a blood sacrifice

* fix and update annotate test to stub

* update vep container on stub runs

* add postannotations support to the configs

* simplified the vcfanno config code and made it smarter

* added new fields from the annotsv vcf to the default configs

* added a param for the default annotsv file name

* fix small bug with param

* fix small bug with param

* Add all default configs

* sacrificing my firstborn child to the linting gods

* Moved the unused configs to a separate folder

* update configs

* update vep config

* add option to annotate on small variants with AnnotSV (real test files should still be added)

* Remove unused VEP parameters

* fix issues

* add tests for small variants vcfs

* add an option to also output the annotsv outputs

* Merge wrongly placed commits (#40)

* fixes for big tests

* metro map update

* Template update for nf-core/tools version 2.8

* black linting

* update modules

* update modules

* fix local modules

* bump

* eclint

* update genome in docs

* update vcfanno configs retina and rpe

* fix rpe bed file in config

* adjust test to new data

* remove unused data from workflows

* remove simple event annotation

* update modules config

* remove unused modules

* fix compile error

* another compile error

* remove comments in config

* guess what! another compile error

* small fixes to annotsv

* update viola

* circumvent a annotsv bug

* add gridss genotyping

* filter out variants with 0/0 genotype from gridss

* move ref variants filtering to before annotsv

* update tests

* fix linting and add meta to references

* black

* install expansionhunter

* fix merge conflict issues

* fix linting

* fix config

* fix stub

* add expansionhunter

* some concurrency debugging

* add nf-validation support

* revert to using linkedhashmap for now

* fix jasmine issue

* update tests

* fix typos

* fix linting

* there you go editorconfig

* let's make it pretty

* update nf-test config

* fix even more tests

* add concat output and fix tests

* fix schema

* now be nice please

* i'm pretty again

* update big test samplesheet [no ci]

* fix max_time pattern [no ci]

* add bcftools sort to single runs

* only apply viola on gridss VCFs and change prefixes

* remove unused comment [no ci]

* fix input pattern

* improve the efficiency of viola

* black

* added a sort step to gridss

* updated the gridss label

* patch gridss module

* remove estimate_read_length

* remove unused code

* small typo

* add -ends to vcfanno

* add full test config

* fix full test description

* update samplesheet

* add exists tags to the samplesheet schema

* specify advised resources for gridss

* possible fix for gridss OOM?

* remove code duplication

* remove an unused parameter

* added a check for callers support being higher than the amount of sv callers

* update metro map

* increase gridss memory

* try to optimize gridss part 2

* try to optimize gridss part 3

* oops

* oops 2

* oops 3

* update to advised amounts

* increase viola time

* discourage the use of gridss for now

* typo

* linting fix

* rename SV subworkflow

* fix expansionhunter catalog and add qdnaseq ref

* add qdnaseq

* add a check for the qdnaseq reference

* fixed some issues with qdnaseq

* add cram merging and index creation

* update resources for qdnaseq

* update resources for qdnaseq

* update resources for qdnaseq

* fix resources qdnaseq

* add wisecondorx

* updated the metro map

* fix background of metro map

* update resolution

* update resolution

* remove old samplesheet conversion script

* moved available callers to a groovy script

* removed the manual check for file exists (now fully uses nf-validation)

* fix a bug with concat when a cnv caller has been supplied

* added some comments for the last fix

* update optional input handling

* fix small issue with annotsv annotations

* fix a no process matching selector warning

* revert last changes

* temp fix for warning

* improve vep cache handling + update modules

* typo

* fix container declaration

* fix for manta issue?

* undo last commit

* version bump expansionhunter

* added support for manta configs

* made qdnaseq references gender specific

* update manta resources

* add contig input to manta

* moved the contigs file creation to the manta subwf

* fix vep params in schema

* fix linting with no template update

* fix nf-test tests

* fix eclint

* fix eclint

* change gender to sex

* typo

* added the sex arg to expansionhunter

* fix nf-test config for v0.8.0

* revert nf-test to 0.7.3

* prettier

* set default sex to female instead of male

* fix tests samplesheet

* Fix vep container issue (#52)

* add dynamic container fetching for vep

* linting fix

* moved the function to the process itself

* fix conda issue

* eclint

* fix params schema

* fix linting

* fix for vep issue?

* fix for missing singularity container

* Add filtering on annotations + fix linting (#55)

* Add filtering on annotations

* fix linting

* begone trailing whitespace

* add annotation filter to test

* CNV VCFs (#58)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* Family merging (#61)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* add z-score to VCF header

* small fixes to sex defining

* Add proper testing + fix some issues found (#64)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* small clarification in the header

* Fix annotsv TCL error (#69)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* split annotsv input files by svtype and merge them back

* small clarification in the header

* fix bcftools concat issue

* fix linting

* don't lint .prettierignore

* Add Svync standardization + other fixes (#72)

* add vcf creation to wisecondorx

* add qdnaseq vcfs

* update tests

* add a simple qdnaseq filter

* small fix in the wisecondorx config

* small naming fix

* add jasmine

* fix for config?

* fix tests

* prettier

* fix tests again

* fix output logic

* fix jasmine + add annotation

* eclint

* more output logic fixes

* eclint (again)

* more output logic fixes

* fix cnv outputs

* remove obsolete if statement

* fix issue with meta joining

* add family name to the samplesheet and meta

* changes to jasmine subwf + add family merging subwf

* add correct reheadering

* add family merging (does not work rn)

* some more work

* fix repeat  VCF error

* first commit

* add z-score to VCF header

* small fixes to sex defining

* add test for bam_cnv_calling

* test bam_prepare_samtools

* test bam_repeat_estimation_expansionhunter

* test bam_sv_calling

* test bam_variant_calling_delly

* test bam_variant_calling_gridss

* test bam_variant_calling_manta

* test bam_variant_calling_qdnaseq

* test bam_variant_calling_smoove

* test bam_variant_calling_wisecondorx

* uniformize config

* test vcf_annotate_vep_annotsv_vcfanno

* test vcf_concat_bcftools

* test vcf_merge_callers_jasmine

* fix header

* update vep annotations

* oops

* add gnomad to vep

* revert vep container changes

* add sift

* fix merge crams with families

* fix join duplication for replicates

* fully test jasmine family merge

* uniformize tests

* fix duplication glitch

* add pipeline tests for callers

* add families tests

* add duplicates tests

* remove old tests and add new tests to ci

* fix linting

* use nf-core dev for linting

* update minimum nextflow version

* revert nf-core version to latest

* upload artifacts

* fix artifacts

* only upload nf-test folder

* fix artifact again?

* fix artifact again?

* fix artifact again?

* prettier

* only get artifacts on failure

* only get artifacts on failure

* only get artifacts on failure

* fix qdnaseq test

* set nf-test to v0.8.1

* try to fix permission errors

* update family jasmine tests

* split annotsv input files by svtype and merge them back

* small clarification in the header

* fix bcftools concat issue

* fix linting

* don't lint .prettierignore

* deprecate gridss for now

* implement delly svync

* implement manta svync

* implement smoove svync

* add better headering to merge callers

* small fix to bcftools annotate

* fix headers for family merging

* remove unused assets

* fix some stability issues

* simplify expansionhunter handling

* reheader with bcftools/reheader again

* fix the headering once and for all

* add a quick view statement

* change view to dump

* rename dump tag

* extra dump

* fix join issue

* fix header issues with annotsv

* add missing header line for annotsv

* fix callers info field

* remove some views

* add views for hpc debugging

* a workaround for double variants in annotsv

* a workaround for double variants in annotsv try 2

* remove view statements

* fix join error?

* fix some linting issues

* don't test gridss

* drop pinned nf-test version

* disable gridss tests for real

* fix some tests

* update subwf tests

* update pipeline tests

* fix annotate tests?

* fix annotate tests again?

* add views to annotation test

* fix join error + fix test jasmine

* revert join error changes?

* another try to fix weird join error

* join error fix try 3

* join error fix try 4

* join error fix try 5

* join error fix try 6

* join error fix try 7

* Template v2.13.1 sync + update all modules (#73)

* Template update for nf-core/tools version 2.6

* Template update for nf-core/tools version 2.7.2

* update pipeline name

* update test-data repo name

* update test data config name

* update pipeline name

* update pipeline name

* update nf-core yaml

* Template update for nf-core/tools version 2.13.1

* Merge branch 'TEMPLATE' into template-v2.13.1-sync

* fix some linting issues

* update all modules

* fix subwf in modules json

* remove unused code

* remove unused code

* fix jasminesv

* fix eclint

* prettier

* have nf-test in the CI

* fix samtools convert stub

* prettier

* fix tabix outputs + tests

* update test snapshots

* remove views

* rearrange joins

* revert last commit

* another try 🤞

* another try 2 🤞

* try adding by

* fix wrong versions yml

* patch bcftools concat

* fix bcftools concat versions

* fix delly svync config

* use midnighter's approach of joining channels

* fix linting

* fix test

* Create beautiful docs with mkdocs (#74)

* create beautiful docs

* some small fixes

* set dev version to beta version

* Docs 0.1.0dev (#75)

* update docs

* fix linting

* update description of pipeline

* docs-output (#76)

* docs

* parameters.md adapted

* relative links

* comments for docs-output PR

* parameters.md correction

* Last fixes before v0.1.0 release (#77)

* fix tests

* some esthetic fixes

* move the logo in the metro map

* made the metro map logo a bit smaller

* update parameter docs

* bump to version 0.1.0

* update to dev version

* update docs building

* try to fix the doc building action

* merge master into dev

* Add keyword shortcuts to the `--callers` parameter (#85)

* add more keywords + update docs

* update tests

* prettier

* fix tests

* bump wisecondorx to v1.2.6 (#87)

* bump wisecondorx to v1.2.6

* add missing arguments to wisecondorx predict

* add seed 0

* Fix a wisecondorx arg typo

* Fix smoove output when using `--output_callers` (#89)

* fix smoove output

* fix copy paste error

* fix smoove tests

* update changelog

* Make the workflow pluggable (#91)

* make main workflow pluggable

* update svync config handling

* update bedgovcf config handling

* pluggable annotation workflow

* update bam_cnv_calling test

* update all modules

* another wave of module updates

* let the pipeline run again

* fix last modules

* make editorconfig happy

* fix tests

* Update/fix annotsv + add HPO terms (#93)

* make main workflow pluggable

* update svync config handling

* update bedgovcf config handling

* pluggable annotation workflow

* update bam_cnv_calling test

* update all modules

* another wave of module updates

* let the pipeline run again

* fix last modules

* make editorconfig happy

* fix tests

* update annotsv

* add hpo terms

* re-add samples to samplesheet

* fix the annotsv tests

* update changelog

* bump version to 0.2.0

* fix wrongly merged conflict in modules.config

* fix another weird issue

* use latest vep version

---------

Co-authored-by: mvheetve <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mvheetve mvheetve mvheetve approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nvnieuwk @mvheetve