Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Download pipelines for Tower #2247

Merged
merged 43 commits into from
Jun 1, 2023
Merged

Download pipelines for Tower #2247

merged 43 commits into from
Jun 1, 2023

Commits on May 25, 2023

  1. Add -t / --tower option to 'nf-core download'.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f9d07ef View commit details
    Browse the repository at this point in the history

  2. Intermediate commit

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    e94dce0 View commit details
    Browse the repository at this point in the history

  3. Implement logic for the Tower download in DownloadWorkflow:download_w… 

    …orkflow()
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    cba336b View commit details
    Browse the repository at this point in the history

  4. Extend ModulesRepo:setup_local_repo() with a cache_only bool, so we c… 

    …an use ModuleRepo as superclass to the new WorkflowRepo.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    15c0588 View commit details
    Browse the repository at this point in the history

  5. Create WorkflowRepo subclass of ModuleRepo and initialise local clone.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    2be88c4 View commit details
    Browse the repository at this point in the history

  6. TypeError: HEAD is a detached symbolic reference as it points to ...

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    d72169d View commit details
    Browse the repository at this point in the history

  7. Split history ./modules/modules_repo.py to synced_repo.py

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    a6b3492 View commit details
    Browse the repository at this point in the history

  8. Split history ./modules/modules_repo.py to synced_repo.py

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    8984a63 View commit details
    Browse the repository at this point in the history

  9. Split history ./modules/modules_repo.py to synced_repo.py

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    33d0381 View commit details
    Browse the repository at this point in the history

  10. Duplication of ModulesRepo to SyncedRepo done.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    caef187 View commit details
    Browse the repository at this point in the history

  11. Strip ModulesRepo class of the methods moved to new superclass.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    2367ae7 View commit details
    Browse the repository at this point in the history

  12. Rebase to current dev.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f5f0df2 View commit details
    Browse the repository at this point in the history

  13. Local caching of the repo works now.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f852159 View commit details
    Browse the repository at this point in the history

  14. Started implementing the config download.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    af4754e View commit details
    Browse the repository at this point in the history

  15. Started to implement the multiple revision selection for the Tower do… 

    …wnload.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    3bc97c5 View commit details
    Browse the repository at this point in the history

  16. Rewrite get_revision_hash() function to accomodate multiple revisions.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    e17a8e9 View commit details
    Browse the repository at this point in the history

  17. The 2nd revivial of the config choice. Now available for archives wit… 

    …h only one revision.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    ecaabf8 View commit details
    Browse the repository at this point in the history

  18. Inclusion of the revision in the output file name is problematic with… 

    … the new ability to download multiple revisions at once. This resulted in loooooooong filenames.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    6d04ec8 View commit details
    Browse the repository at this point in the history

  19. Allow multiple instances of the -r argument. Needed for scripted down… 

    …load. Ultimately, this now means that I also have to implement multiple version downloads for the classic download. Just downloading the first doesn't seem to make sense from a UX perspective.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    7642e4f View commit details
    Browse the repository at this point in the history

  20. Finished updating the prompts for the dialogues.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    7f93edb View commit details
    Browse the repository at this point in the history

  21. Converted the self.wf_download_url into a dict.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    12bf942 View commit details
    Browse the repository at this point in the history

  22. Enable multi-revision classic download.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    2ff62f3 View commit details
    Browse the repository at this point in the history

  23. Small tweaks to ensure that tools doesn't bail out if there is no sym… 

    …link from singularity to apptainer on the system.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    986f791 View commit details
    Browse the repository at this point in the history

  24. Initialise the Git repo clone of the workflow.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    6f95829 View commit details
    Browse the repository at this point in the history

  25. WorkflowRepo attributes and functions.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    760fcaa View commit details
    Browse the repository at this point in the history

  26. Finished the Tower download branch.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    c381776 View commit details
    Browse the repository at this point in the history

  27. Minor tweaks to the container download functionality.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    526a26e View commit details
    Browse the repository at this point in the history

  28. Updating docs and changelog, fixing linting errors.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f4b9e67 View commit details
    Browse the repository at this point in the history

  29. Hopefully fixed the existing tests. New ones still need to be written.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    2bf14bd View commit details
    Browse the repository at this point in the history

  30. Refactor the CLI commands for the Singularity Cache Dir

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    8de588e View commit details
    Browse the repository at this point in the history

  31. Readme updates for the new remote Singularity cache feature.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    d729bde View commit details
    Browse the repository at this point in the history

  32. Add interactive check in retry for parsing the index.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    0f58c29 View commit details
    Browse the repository at this point in the history

  33. Incorporating some suggestions by @mashehu.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    6294d74 View commit details
    Browse the repository at this point in the history

  34. Apply suggestions from code review @mashehu 

    Co-authored-by: Matthias Hörtenhuber <[email protected]>
    @MatthiasZepper @mashehu
    MatthiasZepper and mashehu committed May 25, 2023
    Configuration menu
    Copy the full SHA
    8d327a4 View commit details
    Browse the repository at this point in the history

  35. Writing additional tests for the --tower download functionality.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    340c519 View commit details
    Browse the repository at this point in the history

  36. Move alterations from Version 2.8 (which this PR didn't make anymore)… 

    … to Version 2.9dev.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f599237 View commit details
    Browse the repository at this point in the history

  37. Adding the info about remote containers to the summary log rather tha… 

    …n showing it separately.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    2518a4b View commit details
    Browse the repository at this point in the history

  38. Moved the notification about remote containers to summary_log.

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    4f390be View commit details
    Browse the repository at this point in the history

  39. Apply suggestions from code review 

    Co-authored-by: Matthias Hörtenhuber <[email protected]>
    @MatthiasZepper @mashehu
    MatthiasZepper and mashehu committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f8e5068 View commit details
    Browse the repository at this point in the history

  40. Fixes suggested by @mirpedrol during review. Thanks!

    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    e512878 View commit details
    Browse the repository at this point in the history

  41. @mashehu suggested that downloading the containers should not be opti… 

    …onal for Tower downloads. Given that there is the option to provide the list of remote containers to skip their download, I agree that this is reasonable.
    @MatthiasZepper
    MatthiasZepper committed May 25, 2023
    Configuration menu
    Copy the full SHA
    315b9a3 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. Bugfix: WorkflowRepo.tidy_tags() did indeed only tidy tags. However, … 

    …revisions may also be branches. Therefore, I rewrote this function to account for revisions that are not releases.
    @MatthiasZepper
    MatthiasZepper committed May 26, 2023
    Configuration menu
    Copy the full SHA
    6a806ee View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. Merge branch 'dev' into DownloadForTower

    @MatthiasZepper
    MatthiasZepper authored May 31, 2023
    Configuration menu
    Copy the full SHA
    259afa6 View commit details
    Browse the repository at this point in the history