Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev -> Main for 0.2.0 release #34

Merged
merged 92 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
750491f
Bump pipeline version to 0.2.0dev
drpatelh Dec 13, 2023
ead81f8
Allow underscores in workplace name (#19)
robsyme Apr 30, 2024
ac3680e
Copy in CI from fetchngs
pinin4fjords May 1, 2024
ee757cf
Add include.yaml
pinin4fjords May 1, 2024
dbcc41e
Bump tower-cli
pinin4fjords May 1, 2024
c0e93a7
Bump cli
pinin4fjords May 1, 2024
022d18c
Poke CI
pinin4fjords May 1, 2024
464bd09
Revert "Poke CI"
pinin4fjords May 1, 2024
1f902f6
Update dump snapshot for new tw
pinin4fjords May 1, 2024
ad27918
Bump a snapshot md5
pinin4fjords May 1, 2024
5586d23
Merge branch 'fusion_fixes' of github.com:seqeralabs/nf-aggregate int…
pinin4fjords May 1, 2024
0ab787e
Fix snapshot order
pinin4fjords May 1, 2024
cc87a49
Merge pull request #23 from seqeralabs/fusion_fixes
pinin4fjords May 1, 2024
24332e1
fix: remove escape from tower env var
ejseqera May 1, 2024
0d6a7d7
Remove special chars from version
pinin4fjords May 1, 2024
5c1b6b5
Add missing escapes
pinin4fjords May 1, 2024
99b0044
Revert "Add missing escapes"
pinin4fjords May 2, 2024
8efa692
Revert "Remove special chars from version"
pinin4fjords May 2, 2024
97b971b
Make sure colors turned off in versions command
pinin4fjords May 2, 2024
7b5219e
Merge pull request #28 from seqeralabs/fix_mamba_versions
pinin4fjords May 2, 2024
736e146
Bump build for singularity compatibility
pinin4fjords May 2, 2024
2157516
Poke CI
pinin4fjords May 2, 2024
e913f80
Fix versions.yml for new build
pinin4fjords May 2, 2024
64f2b7b
Merge pull request #29 from seqeralabs/bump_images_for_singularity
pinin4fjords May 2, 2024
e060d37
Merge branch 'dev' into update_ci
pinin4fjords May 2, 2024
0389e61
Older nf won't work
pinin4fjords May 3, 2024
9111f97
Fix up tests for utils_nf_aggregate
pinin4fjords May 3, 2024
946bffd
Revert "Fix up tests for utils_nf_aggregate"
pinin4fjords May 3, 2024
8de332d
Activate test profile for nf-test
pinin4fjords May 3, 2024
deebe24
Add token to env
pinin4fjords May 3, 2024
7ce3bae
Merge pull request #22 from seqeralabs/update_ci
pinin4fjords May 3, 2024
b982c61
Skip Gantt plots with non-fusion runs
pinin4fjords May 3, 2024
0cdfe49
appease eclint
pinin4fjords May 3, 2024
0ca152b
prettier
pinin4fjords May 3, 2024
b35d927
branch -> filter
pinin4fjords May 7, 2024
f01b2ef
Add a nonfusion test run
pinin4fjords May 7, 2024
d48e792
Update modules/local/seqera_runs_dump/main.nf
pinin4fjords May 7, 2024
298fdbd
Harshil displeased eclint
pinin4fjords May 7, 2024
b9fde15
Fix shapshot
pinin4fjords May 7, 2024
72167c1
Update snapshot
pinin4fjords May 7, 2024
45941f9
Update API URI
pinin4fjords May 7, 2024
a493ce4
Update nextflow_schema.json
pinin4fjords May 7, 2024
0003d75
update snapshot
pinin4fjords May 7, 2024
1ed0592
Merge pull request #31 from seqeralabs/update_api_uri
pinin4fjords May 7, 2024
093005d
Merge branch 'dev' into fix_for_fusion
pinin4fjords May 7, 2024
bb249ae
Merge pull request #30 from seqeralabs/fix_for_fusion
pinin4fjords May 8, 2024
1100472
Bump multiqc
pinin4fjords May 20, 2024
8f47be8
Bump subworkflows
pinin4fjords May 20, 2024
910b094
Remove reference to swf summary_params
pinin4fjords May 20, 2024
05723c5
Add tests dir for swf
pinin4fjords May 20, 2024
6aabfaf
Update nf-test config
pinin4fjords May 20, 2024
c515dc3
Bump service-info.json
pinin4fjords May 20, 2024
1c14099
Merge pull request #32 from seqeralabs/bump_modules
pinin4fjords May 21, 2024
4d23522
Remove docker.userEmulation
adamrtalbot May 24, 2024
e4016db
Merge pull request #33 from seqeralabs/remove-dockerUserEmulation
drpatelh May 28, 2024
3e517b2
Merge pull request #27 from seqeralabs/tower-var-fix-gcp
drpatelh May 28, 2024
f2d14b2
Update CHANGELOG
drpatelh May 28, 2024
2380dbc
Add note in README for Gantt plotting with Fusion
drpatelh May 28, 2024
829314b
Bump pipeline version to 0.2.0
drpatelh May 28, 2024
42ee5b3
Remove unused params.pipelines_testdata_base_path from nf-test nextfl…
drpatelh May 28, 2024
b006b9e
Tidy up filtering syntax before PLOT_RUN_GANTT process
drpatelh May 28, 2024
e056098
Update README.md
drpatelh May 28, 2024
f2805f4
Merge pull request #35 from seqeralabs/updates
drpatelh May 28, 2024
959b813
initial template build from nf-core/tools, version 2.14.1
maxulysse May 29, 2024
51b0472
Merge branch 'dev' into new_dev
maxulysse May 29, 2024
ac26fcc
fix name
maxulysse May 29, 2024
719217a
fix name
maxulysse May 29, 2024
fe6468c
fix name
maxulysse May 29, 2024
8868b86
more linting fix
maxulysse May 29, 2024
6c8d615
update CHANGELOG
maxulysse May 29, 2024
4b678c6
fix lint
maxulysse May 29, 2024
c478f43
Update README.md
maxulysse May 29, 2024
fe8d1f7
remove files
maxulysse May 29, 2024
f940d3a
Delete .github/.dockstore.yml
maxulysse May 29, 2024
0c7a780
remove file
maxulysse May 29, 2024
6df2577
remove file
maxulysse May 29, 2024
fa9b4ee
remove file
maxulysse May 29, 2024
6fa3056
fix script
maxulysse May 29, 2024
5c04a40
remove file
maxulysse May 29, 2024
ba0349e
Delete subworkflows/local/utils_nfcore_aggregate_pipeline/main.nf
maxulysse May 29, 2024
8868f14
disable that
maxulysse May 29, 2024
6ac4c9b
remove more files
maxulysse May 29, 2024
ce2e0c2
disable that
maxulysse May 29, 2024
e283663
add nf-core lint
maxulysse May 29, 2024
3483a3b
wave profile should be there
maxulysse May 29, 2024
7202399
Apply suggestions from code review
drpatelh May 29, 2024
53eeb23
Delete modules/nf-core/multiqc/tests/main.nf.test.snap
maxulysse May 29, 2024
a6b3439
include test modules data path
maxulysse May 29, 2024
a7092f9
add nf-test files
maxulysse May 29, 2024
5790aec
restore snap file
maxulysse May 29, 2024
e08937b
Update CHANGELOG.md
maxulysse May 29, 2024
c3f4f83
Merge pull request #40 from seqeralabs/nf-core-template-merge-2.14.1
drpatelh May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,7 @@
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/opt/conda/bin/autopep8",
"python.formatting.yapfPath": "/opt/conda/bin/yapf",
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.defaultInterpreterPath": "/opt/conda/bin/python"
},

// Add the IDs of extensions you want installed when the container is created.
Expand Down
11 changes: 10 additions & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,16 @@ end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset
indent_size = unset
[/subworkflows/nf-core/**]
charset = unset
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset

[/assets/email*]
indent_size = unset

# ignore python and markdown
[*.{py,md}]
indent_style = unset
118 changes: 118 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# seqeralabs/nf-aggregate: Contributing Guidelines

Hi there!
Many thanks for taking an interest in improving seqeralabs/nf-aggregate.

We try to manage the required tasks for seqeralabs/nf-aggregate using GitHub issues, you probably came to this page when creating one.
Please use the pre-filled template to save time.

However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;)

## Contribution workflow

If you'd like to write some code for seqeralabs/nf-aggregate, the standard workflow is as follows:

1. Check that there isn't already an issue about your idea in the [seqeralabs/nf-aggregate issues](https://github.com/seqeralabs/nf-aggregate/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [seqeralabs/nf-aggregate repository](https://github.com/seqeralabs/nf-aggregate) to your GitHub account
3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged

If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).

## Tests

You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:

```bash
nf-test test --profile debug,test,docker --verbose
```

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

There are typically two types of tests that run:

### Lint tests

`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.

If any failures or warnings are encountered, please follow the listed URL for more documentation.

### Pipeline tests

Each `nf-core` pipeline should be set up with a minimal set of test-data.
`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
If there are any failures then the automated tests fail.
These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.

## Patch

:warning: Only in the unlikely and regretful event of a release happening with a bug.

- On your own fork, make a new branch `patch` based on `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- A PR should be made on `master` from patch to directly this particular bug.

## Pipeline contribution conventions

To make the seqeralabs/nf-aggregate code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.

### Adding a new step

If you wish to contribute a new step, please use the following coding standards:

1. Define the corresponding input channel into your new process from the expected previous process channel
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new parameters to `nextflow.config` with a default (see below).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
6. Add sanity checks and validation for all relevant parameters.
7. Perform local tests to validate that the new code works as expected.
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.

### Default values

Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.

Once there, use `nf-core schema build` to add to `nextflow_schema.json`.

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

### Naming schemes

Please use the following naming schemes, to make it easy to understand what is going where.

- initial process channel: `ch_output_from_<process>`
- intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`

### Nextflow version bumping

If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`

### Images and figures

For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).

## GitHub Codespaces

This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal.

To get started:

- Open the repo in [Codespaces](https://github.com/seqeralabs/nf-aggregate/codespaces)
- Tools installed
- nf-core
- Nextflow

Devcontainer specs:

- [DevContainer config](.devcontainer/devcontainer.json)
54 changes: 54 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Bug report
description: Report something that is broken or incorrect
labels: bug
body:
- type: textarea
id: description
attributes:
label: Description of the bug
description: A clear and concise description of what the bug is.
validations:
required: true
- type: textarea
id: command_used
attributes:
label: Command used and terminal output
description: Steps to reproduce the behaviour. Please paste the command you used
to launch the pipeline and the output from your terminal.
render: console
placeholder: "$ nextflow run ...


Some output where something broke

"
- type: textarea
id: files
attributes:
label: Relevant files
description: "Please drag and drop the relevant files here. Create a `.zip` archive
if the extension is not allowed.

Your verbose log file `.nextflow.log` is often useful _(this is a hidden file
in the directory where you launched the pipeline)_ as well as custom Nextflow
configuration files.

"
- type: textarea
id: system
attributes:
label: System information
description: "* Nextflow version _(eg. 23.04.0)_

* Hardware _(eg. HPC, Desktop, Cloud)_

* Executor _(eg. slurm, local, awsbatch)_

* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud,
or Apptainer)_

* OS _(eg. CentOS Linux, macOS, Linux Mint)_

* Version of seqeralabs/nf-aggregate _(eg. 1.1, 1.5, 1.8.2)_

"
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Feature request
description: Suggest an idea for the seqeralabs/nf-aggregate pipeline
labels: enhancement
body:
- type: textarea
id: description
attributes:
label: Description of feature
description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
validations:
required: true
25 changes: 25 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!--
# seqeralabs/nf-aggregate pull request

Many thanks for contributing to seqeralabs/nf-aggregate!

Please fill in the appropriate checklist below (delete whatever is not relevant).
These are the most common things requested on pull requests (PRs).

Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.

Learn more about contributing: [CONTRIBUTING.md](https://github.com/seqeralabs/nf-aggregate/tree/master/.github/CONTRIBUTING.md)
-->

## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/seqeralabs/nf-aggregate/tree/master/.github/CONTRIBUTING.md)
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
- [ ] `README.md` is updated (including new tool citations and authors/contributors).
10 changes: 10 additions & 0 deletions .github/include.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
".":
- ./.github/workflows/**
- ./nf-test.config
- ./nextflow.config
tests:
- ./assets/*
- ./bin/*
- ./conf/*
- ./main.nf
- ./nextflow_schema.json
Loading
Loading