Consider repos which already have build recipes

[pelson]

Some repos ship with a conda recipe as part of the source - it would be nice to avoid duplicating the recipe, and just use that directly - perhaps using git submodules (which are horrible, but would work for this process). Updating a recipe would then be a matter of updating the submodule location.

I did something similar in https://github.com/pelson/package_with_continuous_delivery/tree/_build.

This might also give us a way to package packages for which no recipe is needed (because the conda skeleton approach works).

[ChrisBarker-NOAA]

This is a great idea -- if nothing else, I have a couple packages that I maintain, and I"ve been struggling with where to maintain the recipe -- in the repo of the package itself is the obvious way to do it.

I agree that git submodules are horrible -- is there really no other way?

Maybe a meta-recipe that bootstraps conda-build, and then pulls everything from the downloaded repo? I'd have to poke in more to see if that's possible. And it would still require SOMETHING outside the primary repo that would need maintaining...

[pelson]

And it would still require SOMETHING outside the primary repo that would need maintaining...

Yes. The only problem with recipes inside the repo is that they with by necessity be getting their source from a relative location, whereas a recipe from outside the repo will be pointing to an absolute location. We can mitigate this by requesting the recipe has the ability to switch source based on either some environment variable, or by a tool which automatically adjusts the recipe for us.

The conda skeleton concept is definitely one I'd like to explore though. The big downside of that, and the "fetch recipe from canonical repo" approach, is that it adds run-time to the build process as you end up having to fetch more and more external sources to determine if a version needs building or not. Clearly that needs careful consideration if we want this to scale. The other concern with having the canonical source outside of a feedstock is how one triggers the build process - if the build process is normally triggered by a merge to the feedstock repo, then we will still need to be adding commits to a repo in order to get anything built - why not just have a script in the feedstock that does that recipe fetching, and then we commit the results.

Its a bit early in the morning, so hopefully that made some sense. 😄

[danielballan]

Another detail worth keeping in mind: recipes in the repo can't include the checksum.

I have been suing this approach on my projects: the recipe in the repo tagets master with no checksum, and a collection of recipes outside the repo target tagged versions and include checksums.

[pelson]

Another detail worth keeping in mind: recipes in the repo can't include the checksum.

Much more eloquently put @danielballan - thank you!

[jakirkham]

Is this still a problem worth solving? It seems we have moved away from using recipes in repos for quite sensible reasons well outlined by @danielballan. Namely recipes in repos are there to be useful for development primarily and possibly releases. Whereas recipes here are for releases only.