Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make cap a submodule to SCHISM #23

Open
platipodium opened this issue Mar 2, 2023 · 21 comments
Open

Make cap a submodule to SCHISM #23

platipodium opened this issue Mar 2, 2023 · 21 comments
Assignees
@platipodium
Labels
enhancement New feature or request

Comments

@platipodium
Copy link
Member

Dan Rosen and Arun Chawla recommended to move the cap to within SCHISM, as a submodule
@platipodium
Copy link
Member Author

Joseph does not really favour git submodules in SCHISM, and Carsten does not favour including the code hard into SCHISM as subdirectory ... so maybe just leave as is?

@platipodium platipodium self-assigned this Mar 2, 2023
@platipodium platipodium added the enhancement New feature or request label Mar 2, 2023
@jreniel
Copy link
Member

jreniel commented Mar 2, 2023 •
edited
Loading

Maybe what you want is a git subrepo?

Recap:

  • git submodules are used when you don't want to explicitly include the code (for example, optional helper functionality).
  • git subtrees can be used when you want explicitly add another repo's code to your code, but only works well if you don't plan to push back to the parent (this could be your case).
  • git subrepo can be used for 2-way push pull, and clones the code explicitly.

Git subrepo is the "newest" of the three. It definitely replaces git subtrees (which doesn't cleanly allow for 2 way comm between the repos) but submodules still have their purpose, when you want to include links to another code base, rather than the codebase itself.

Hope this helps.

[EDIT]: I strongly advice against subtrees, because your commit history will get messed up if you push from the subtree to the parent. If others have subtree cloned the repo, they'll get your history too. It's a mess if it's not going to be used for one-way only (e.g. schism repo pulling from cap repo is OK, but schism repo pushig to cap repo is bad). git subrepo fixes this problem.

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023

I've been using git subrepo and I highly recommend it. Git subtree was working well, until I tried 2-way push/pull. That's what took me to git subrepo, which finally solved the problem of 2-way comm between repos.

@platipodium
Copy link
Member Author

Thanks @jreniel. I have not looked into git subrepo but would go with your recommendation. If SCHISM is s subrepo of schism-esmf (which is more logical), we can just go ahead, but if schism-esmf is a submodule of schism (which is what the community usually does), @josephzhang8 has to decide in the end.

@platipodium
Copy link
Member Author

git-subrepo is a separate piece of software, so not many users will have it installed...

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023 •
edited
Loading

You can make it either ways or even technically (maybe nonsensical), both.
Each repo has their own build system, therefore, their own system tree to look for.

git-subrepo works without the end users having it installed, this is explicitly stated in the readme.md.

You can always test these things in your own workspace and explore the result without necessarily pushing.
After installing subrepo, simply do git subrepo clone [email protected]:schism-dev/schism in your schism-esmf repo.
You can do this on another branch or you can revert: git reset HEAD~1 ... Or you can even copy your workdir and try on a different dir. Either way, it will result in an opening of your options...

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023

To give you an example, git subtree also works without the users having it installed. One example is jigsaw-python which includes ext/jigsaw as a subtree. The users don't know this is a subtree, and don't need git-subtree installed. The same happens with git subrepo, (but not so much with git submodules, which kind-of requires to have submodules in PATH).

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023 •
edited
Loading

Having given it some thought, I think in your case it makes more sense that schism-esmf integrates schism as a subrepo, rather than the other way around. The "child" module integrating the parent module will make more sense in the long-run. Otherwise, in a sense, you'd be handing off the maintenance of your package to the parent, which doesn't make a lot of sense.

This is similar to how jigsaw-python integrates jigsaw, rather than the other way around.

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 2, 2023 via email

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 2, 2023 via email

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023

The projects maintainer (the person doing the subrepo adding) needs to install it. But once the surepo has be initialized, anyone that clones the repo does not need it installed. Plz read carefully the subrepo docs. It really does solve your problem.

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023 •
edited
Loading

(It is also trivial to install it)
https://github.com/ingydotnet/git-subrepo#installation

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023 •
edited
Loading

In this case, Carsten would be the only one needing to do the subrepo install, no one else would need to install it, not even the SCHISM repo. Subrepo does not interfere with the parent, which is the problem with subtree, that can interfere with the parent/siblings if more there's 3-way push/pull. Subtree only work well in 1-way mode, which would still be valid for Carsten's case, however I'm sure he'd prefer a "clean 2-way mode", which is what subrepo provides and subtree lacks.

Anyone cloning schism-esmf also would not need it installed, just the people doing the direct maintenance on the subrepo itself would need to install it.

@jreniel
Copy link
Member

jreniel commented Mar 2, 2023

Pretty much all the common arguments against using it are addressed here:
https://github.com/ingydotnet/git-subrepo#benefits

@platipodium
Copy link
Member Author

@jreniel what a fervent discussion :=). Although it sounds nonsensical, I could see in the long run that they include each other (I can only hope that won't be recursive). I can try on the schism-esmf side first and we see how it works for CoastalApp folks (and for ourselves).

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 3, 2023 via email

@platipodium
Copy link
Member Author

For compatibility with CMEPS, Marina Verteinstein suggested to make the cap a submodule to schism .... Just noting down this. There's her personal fork that did that.

My conclusions so far

  • I prefer to keep separate repos: less spam for developers, easier maintenance. Not too many interdependencies between the codes so not much likelihood of breaking; both schism and schism-esmf are usually used at bleeding edge for those who use the coupled model anyway.
  • The community seems conservative in wanting only one repo per science model, but I really don't see a good reason for that, other than convenience. CoastalApp seems fine though. CEMS is not.
  • If we find that the next CoastalApp (possibly replaced by ESMX) requires a single repo, then we use git subrepo to satisfy their needs. I am convinced that git subrepo is the best next to keeping repos separate.

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 4, 2023 via email

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 6, 2023 via email

@platipodium
Copy link
Member Author

I don't see urgency to act on this.

  • As long as we're developing the ESMF-SCHISM interface, we should keep the development separate to avoid spamming too many people.
  • There is CI/CD and other features in the ESMF-SCHISM repo that are not available in SCHISM (which in the mid-term should also be adopted)
  • In the long run. we should git subrepo (I am still against hard inclusion) the cap into SCHISM. But there may be even preferable options to this that we should carefully consider. Like having one SCHISM repo with supremos "Core/Hydro", and all the other stuff that is now a subdirectory. That would be much cleaner (but is beyond the point here).

@josephzhang8
Copy link
Member

josephzhang8 commented Mar 21, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@platipodium platipodium

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@platipodium @jreniel @josephzhang8