Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build.jl files should include dependencies #194

Closed
staticfloat opened this issue Mar 20, 2018 · 2 comments
Closed

build.jl files should include dependencies #194

staticfloat opened this issue Mar 20, 2018 · 2 comments
Assignees

Comments

@staticfloat
Copy link
Member

Now that building dependent packages is working pretty well (see, for instance, GlibBuilder) we should make sure that the generated build.jl files chain up with the other build.jl files automatically, so that if the user downloads the auto-generated build.jl file, it will automatically fetch and install all necessary dependencies.

@staticfloat staticfloat self-assigned this Mar 20, 2018
staticfloat added a commit to staticfloat/QemuBuilder that referenced this issue Mar 20, 2018
Until JuliaPackaging/BinaryBuilder.jl#194 is
fixed, we have a slightly untidy `dependencies` section.  That will be
fixed eventually, so we only have to depend on the top-level
dependencies, and the rest will get transparently added/removed.
staticfloat added a commit to staticfloat/QemuBuilder that referenced this issue Mar 20, 2018
Until JuliaPackaging/BinaryBuilder.jl#194 is
fixed, we have a slightly untidy `dependencies` section.  That will be
fixed eventually, so we only have to depend on the top-level
dependencies, and the rest will get transparently added/removed.
staticfloat added a commit to staticfloat/QemuBuilder that referenced this issue Mar 20, 2018
Until JuliaPackaging/BinaryBuilder.jl#194 is
fixed, we have a slightly untidy `dependencies` section.  That will be
fixed eventually, so we only have to depend on the top-level
dependencies, and the rest will get transparently added/removed.
@staticfloat
Copy link
Member Author

So far we have the unofficial agreement that, if binary package FLAC requires binary package Ogg, then this implies that FLAC.jl also requires Ogg.jl, meaning that within FLAC.jl there is a using Ogg. This causes Ogg.jl to dlopen("libogg"), which then causes a dlopen("libFLAC") to not fail when it tries to lookup libogg as libogg is already loaded into the julia process.

However, we now have situations where this isn't the case; we need a way for a build.jl to signify that it might need to install another binary package. I'm comfortable with having FLAC.jl cause the installation of Ogg.jl, but we shouldn't require FLAC.jl to run using Ogg within itself. What would be ideal is for FLAC.jl/deps/build.jl to express a dependence on Ogg.jl/deps/build.jl, and automagically extract the products from that build.jl, and find them at Pkg.build() time, so that when the user runs using FLAC, which then runs include("deps/deps.jl"), the auto-generated deps.jl can dlopen(libogg_path) and then dlopen(libflac_path).

In order for this to work, we need a few things:

  • We need to be able to extract the products from another package. I'm thinking this could be as simple as just include(depsjl_path) within our build.jl.

  • We need for build.jl files to express a concept of "dependencies". At build_tarball.jl time, it is appropriate to encode this as an URL to a build.jl file. At Pkg.build() time, I argue this is not appropriate and we should assume that Pkg has done its job and everything we need is already installed. Ideally, I'd like an encoding that looks like:

dependencies = [
    Pkg.dir("Ogg", "deps", "deps.jl") => [:libogg],
    Pkg.dir("CodecZlib", "deps", "deps.jl") => [:libz],
]

This would express the idea that we need libogg.so from the Ogg package and libz from the CodecZlib package to be loaded before we can load our own libraries. This dependencies would be passed in to install(), and the generated deps.jl file would read in the dependent deps.jl files, locate the paths proffered by the libogg and libz variables defined within those deps.jl files, and finally dlopen() them before we attempted to dlopen() our own libraries.

It is unfortunate that there's no way for us to magically go from OggBuilder URL to the package Ogg, so this is a case where I think we're going to have to ask the user to provide more information manually. :(

@staticfloat
Copy link
Member Author

This is fixed by the implicit dependency web that JLL packages create. Huzzah!

fingolfin added a commit to fingolfin/BinaryBuilder.jl that referenced this issue May 22, 2020
In `locate`, when launching an external Julia to test whether dlopen work
on a library, pass `--startup-file=no` to the subprocess to avoid any issues
with commands in a user's startup.jl file.

Fixes JuliaPackaging#194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant