Samplesheet Validator #238

apeltzer · 2024-08-12T07:04:17Z

HI @nschcolnicov / @edmundmiller / @atrigila - I merged your PR @nschcolnicov to this one based on dev to enable multiple people working / looking at things. Please continue here :)

I'll also try fixing the tests at least but saw that #237 was opened to track some samshee improvements by @edmundmiller which we can still address prior to 1.5.0 release :)

PR checklist

…plesheet_validator

Co-authored-by: Edmund Miller <[email protected]>

Samplesheet validator

github-actions · 2024-08-12T07:04:29Z

This PR is against the `master` branch ❌

Do not close this PR
Click Edit and change the base to dev
This CI test will remain failed until you push a new commit

Hi @apeltzer,

It looks like this pull-request is has been made against the nf-core/demultiplex master branch.
The master branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to master are only allowed if they come from the nf-core/demultiplex dev branch.

You do not need to close this PR, you can change the target branch to dev by clicking the "Edit" button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.

Thanks again for your contribution!

github-actions · 2024-08-12T07:20:31Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit ec132a3

+| ✅ 191 tests passed       |+
#| ❔   3 tests were ignored |#
!| ❗   6 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in README.md: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required

❔ Tests ignored:

files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
actions_ci - actions_ci

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-demultiplex_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-demultiplex_logo_light.png
files_exist - File found: docs/images/nf-core-demultiplex_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-demultiplex_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowDemultiplex.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.5.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.trim_fastq= true
nextflow_config - Config default value correct: params.skip_tools= []
nextflow_config - Config default value correct: params.sample_size= 100000
nextflow_config - Config default value correct: params.downstream_pipeline= default
nextflow_config - Config default value correct: params.demultiplexer= bclconvert
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.checkqc_config= []
nextflow_config - Config default value correct: params.max_cpus= 16
nextflow_config - Config default value correct: params.max_memory= 128.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.remove_adapter= true
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-demultiplex_logo_light.png matches the template
files_unchanged - docs/images/nf-core-demultiplex_logo_light.png matches the template
files_unchanged - docs/images/nf-core-demultiplex_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (252 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - UNTAR found in conf/modules.config and Nextflow scripts.
modules_config - BCLCONVERT found in conf/modules.config and Nextflow scripts.
modules_config - BCL2FASTQ found in conf/modules.config and Nextflow scripts.
modules_config - BASES2FASTQ found in conf/modules.config and Nextflow scripts.
modules_config - FASTP found in conf/modules.config and Nextflow scripts.
modules_config - FALCO found in conf/modules.config and Nextflow scripts.
modules_config - KRAKEN2 found in conf/modules.config and Nextflow scripts.
modules_config - SEQTK_SAMPLE found in conf/modules.config and Nextflow scripts.
modules_config - MD5SUM found in conf/modules.config and Nextflow scripts.
modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
modules_config - SGDEMUX found in conf/modules.config and Nextflow scripts.
modules_config - FQTK found in conf/modules.config and Nextflow scripts.
modules_config - CELLRANGER_MKFASTQ found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
modules_config - CHECKQC found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-08-12 12:48:22

apeltzer · 2024-08-12T07:39:56Z

Ok, tests at least pass again 👍🏻 No 🤠 mode here ;-)

modules/local/samshee/meta.yml

docs/usage.md

apeltzer · 2024-08-12T13:11:11Z

Guess you need to review before we can merge this one @atrigila / @nschcolnicov

zxBIB Schcolnicov and others added 18 commits August 8, 2024 17:38

Started adding samplesheet validator

094ca56

Fixed module execution

5030413

Updated docs, schema and module

0f38381

Remove testing line

be62c8f

Ran precommit, updated changelog

e502194

Merge branch 'dev' of https://github.com/nf-core/demultiplex into sam…

51e29f1

…plesheet_validator

Reverted change to default kraken_db:

f7769e9

Updated container to wave

a3153bf

Fixed tests

3b5f83e

fixing bcl2fastq test

de35498

Fixed linting

f77f1bd

build: Add environment.yml

a534f20

Apply suggestions from code review

90bdd58

Co-authored-by: Edmund Miller <[email protected]>

Added versions.yml

8700822

Updated kraken snap to work with CI

896d02b

Updated citations and readme, renamed module

8587df1

Merge branch 'dev' into samplesheet_validator

c95aa4b

Merge pull request #234 from nschcolnicov/samplesheet_validator

119e8bc

Samplesheet validator

apeltzer requested review from nh13, sam-white04, blajoie and a team as code owners August 12, 2024 07:04

apeltzer requested review from glichtenstein and removed request for a team August 12, 2024 07:04

apeltzer changed the base branch from master to dev August 12, 2024 07:04

apeltzer mentioned this pull request Aug 12, 2024

Samplesheet validator #234

Merged

11 tasks

apeltzer added 2 commits August 12, 2024 07:08

Fix lint

b50e71d

Add versions to csv2tsv module

08460b2

apeltzer added this to the 1.5.0 milestone Aug 12, 2024

Update test to include sed version

ee95c5d

atrigila reviewed Aug 12, 2024

View reviewed changes

modules/local/samshee/meta.yml Outdated Show resolved Hide resolved

docs/usage.md Outdated Show resolved Hide resolved

PR comments

ec132a3

nschcolnicov approved these changes Aug 12, 2024

View reviewed changes

apeltzer merged commit 892b9d8 into dev Aug 12, 2024
13 checks passed

apeltzer deleted the samplesheet-validator branch August 12, 2024 13:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Samplesheet Validator #238

Samplesheet Validator #238

apeltzer commented Aug 12, 2024

github-actions bot commented Aug 12, 2024

github-actions bot commented Aug 12, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

apeltzer commented Aug 12, 2024

apeltzer commented Aug 12, 2024

Samplesheet Validator #238

Samplesheet Validator #238

Conversation

apeltzer commented Aug 12, 2024

PR checklist

github-actions bot commented Aug 12, 2024

This PR is against the master branch ❌

github-actions bot commented Aug 12, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

apeltzer commented Aug 12, 2024

apeltzer commented Aug 12, 2024

This PR is against the `master` branch ❌

github-actions bot commented Aug 12, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️