Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize classes/re-architecture code #13

Open
jwodder opened this issue Jan 2, 2024 · 0 comments
Open

Reorganize classes/re-architecture code #13

jwodder opened this issue Jan 2, 2024 · 0 comments
Labels
code quality Nonfunctional improvements to the codebase under consideration Dev has not yet decided whether or how to implement

Comments

@jwodder
Copy link
Member

jwodder commented Jan 2, 2024

WIP design document for a new code architecture

  • Add a DandisetBackup class that takes care of all repository-wide operations for backing up a Dandiset

    • Primary attributes:
      • dandiset: RemoteDandiset
      • ds: AsyncDataset
      • manager: Manager
    • Merge all Syncer methods into a single method of this class
    • Move the following Datasetter methods to this class:
      • update_dandiset()
      • ensure_github_remote()
      • sync_dandiset()
      • part of update_github_metadata()?
      • tag_releases()
      • mkrelease()
      • backup_zarrs()
    • Make update_dandiset_metadata() a method of this class
    • Make async_assets() a method of this class
    • Make register_s3urls() a method of this class
  • Add a ZarrBackup class for subsuming ZarrSyncer?

  • Wrap this block in a function that takes a DandisetBackup, aiterassets() async iterator, tracker, & error_on_change and returns a structure containing Report, dm.last_timestamp, dm.need_add, & dm.zarrs

  • Move the addurl methods & attributes of Downloader to an Addurl(?) class, an instance of which will be passed to Downloader on instantiation

  • Add a Superdataset class that takes care of operations on the superdataset

@jwodder jwodder added code quality Nonfunctional improvements to the codebase under consideration Dev has not yet decided whether or how to implement labels Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code quality Nonfunctional improvements to the codebase under consideration Dev has not yet decided whether or how to implement
Projects
None yet
Development

No branches or pull requests

1 participant