Use BinaryBuilder/BinaryProvider

[davidanthoff]

Has anyone started thinking about using those for the binary dependencies? That might add some welcome stability to the package?

[lobingera]

Maybe it's just me, but https://github.com/JuliaPackaging/BinaryProvider.jl/ itself announces it with [WIP] and build failing in most cases. How do you come to the idea that this introduces more stability here?
(read as: i would be pleased to replace BinDeps / Homebrew / WinRPM, but i want to see this working first...)

[davidanthoff]

Oh yeah, I'm not saying that it is ready. But it is looking quite promising, and I was just wondering whether folks have started thinking about it in relation to this package here. I've successfully used it for a much, much simpler project, and so far so good. Main benefit seems to be that there is no reliance on things like the opensuse servers etc. anymore.

My guess is that Cairo.jl would be one of the more challenging packages for this new BinDeps2 combo, so it might also be a really good stress test for the design of the new binary delivery stuff?

[lobingera]

I'll see your PR (as WIP).

[davidanthoff]

I think the folks that are building BinaryProvider/BinaryBuilder might actually take a crack at it :) You won't see a PR from me, these build things drive me absolutely crazy and I don't understand enough of that kind of thing to stand even a slight chance of being successful with that kind of project.

[SimonDanisch]

Gave it a try here:
JuliaGraphics/CairoBuilder#1
Of course it didn't work on first try ;)

[SimonDanisch]

First build of Cairo for most platforms went through :) Right now I'm building the remaining (more obscure) platforms and I see no reason why they should fail ;)

But we might be in a problematic situation now.

This is a minimal build of Cairo, without any of the more advanced backends like GTK or xserver.
I'm not sure if one can link those in dynamically - but as it stands I'm perfectly happy with this minimal Cairo binary for my purposes. On the other hand, GTK will take quite a while untill we can offer it via BinaryProvider and to be honest I gave up on GTK - every time I try it out it gives me a new segfault, heisenbug or obscure behaviour and it's such a big library that I feel a bit overwhelmed at the thought of fixing it.
So, since I'm happy with this binary and others may not, I'm not really sure how we can cater different binaries to different audiences of Cairo! Maybe we'll even need a fork?
I hope this ends up as a non issue, since GTK possibly just ships Cairo, so people who use it with GTK could just rely on that?!

In any case, I'll prepare a PR to Cairo and see if the test pass with the current binaries to get things started!

[tknopp]

Gtk.jl has to be compatible with Cairo.jl. This won't work if they use different Cairo binaries. I would strongly prefer not doing 0.6 -> 0.7 transition simultaneously with the transition to BinaryBuilder. This BinaryBuilder thing looks like a massive effort to get fully done and this should happen in a coordinated fashion so that the user of the packages gets a smooth transition.

[SimonDanisch]

let's move the discussion to #229

[giordano]

An update of what I'm doing. First of all, I moved development of the builders to https://github.com/JuliaPackaging/Yggdrasil, as this is the place where all builders should move at some point for convenience.

• I managed to build Cairo with a native backend for macOS and Windows: Add builder for Cairo JuliaPackaging/Yggdrasil#39
• For macOS and Windows, I managed to build Pango with support for Cairo: Add builder for Pango JuliaPackaging/Yggdrasil#41

I'd like to stress that I didn't actually test these builds, so I'm no sure they're working at all, but at least I compiled successfully 🙂.

What remains to do:

• add a native Cairo backend for Linux and FreeBSD. This requires building libX11 (at least for Linux, I hope it'll be good also for FreeBSD), I have no idea how easy it'll be.

[davidanthoff]

This is awesome, thanks so much!!

Just in terms of high level strategy: is the plan then to also add a Gtk builder to Yggdrasil? Because my understanding had been that we can only move Cairo.jl here over to a new binary if we also have the Gtk.jl binary story sorted out, right?

[SimonDanisch]

Would a Xorbuilder help with this?
https://github.com/JuliaGraphics/XorgBuilder/releases

[giordano]

Just in terms of high level strategy: is the plan then to also add a Gtk builder to Yggdrasil? Because my understanding had been that we can only move Cairo.jl here over to a new binary if we also have the Gtk.jl binary story sorted out, right?

I don't have a plan, I just build random packages 😂 Cairo doesn't need to build whole Gtk to build, you can see here Cairo's dependencies:
https://github.com/JuliaPackaging/Yggdrasil/blob/eefd9238db11fe58849422d5ee3e5adce3eb3db9/C/Cairo/build_tarballs.jl#L53-L59.

Would a Xorbuilder help with this?

Wow, I didn't know there was one! I don't think Cairo needs whole Xorg, only libX11 (and its dependencies, like xproto and maybe something else) should be required, but it's encouraging to see that BinaryBuilder worked at least on one platform for whole Xorg!

[davidanthoff]

Cairo doesn't need to build whole Gtk to build

I think the concern of the folks maintaining Cairo.jl is that if they switch over to binaries for cairo that don't work with Gtk.jl, then Gtk.jl would be broken, which wouldn't be acceptable.

I'm actually not pushing that position, because to me everything seems better than the status quo where essentially nothing is working on Mac, but it was a constraint that was mentioned before.

[giordano]

I think the concern of the folks maintaining Cairo.jl is that if they switch over to binaries for cairo that don't work with Gtk.jl, then Gtk.jl would be broken, which wouldn't be acceptable.

Do the two packages need to talk to each other? I'm asking because I really don't know anything about this.

Anyway, I think that if we succeed in building Cairo, someone will undertake the Gtk adventure, too.

[davidanthoff]

I'm no expert on this, but I believe Gtk.jl has a dependency on Cairo.jl, so Cairo.jl needs to use binaries that the gtk binaries in Gtk.jl are compatible with.

[giordano]

Right, Gtk.jl depends on Cairo.jl, but if they only talk to each other through julia and there is no direct communication between the underlying C libraries I don't expect that there could be problems.

[giordano]

By the way, I got an almost working libX11 build (it's not building only on Windows x86_64, but this is not worrying as far as Cairo is concerned): JuliaPackaging/Yggdrasil#42.

Now we only need to put all the builders together. This means building Cairo for Linux and FreeBSD with x11 support and use it to build Pango, then test that all these libraries we've built can be finally used in Cairo.jl

[davidanthoff]

there is no direct communication between the underlying C libraries I don't expect that there could be problems

My understanding is that the gtk binaries use the cairo binaries directly. I might well be wrong, though :)

[tknopp]

Gtk uses Cairo directly. In fact, the entire rendering is done through Cairo.

The dependencies of Gtk are listed here:

https://developer.gnome.org/gtk3/stable/gtk-building.html

I think it makes absolutely sense to make this step by step. As long as Cairo does not make a release, everything is fine. Even if you do, we could pin the version of Cairo in Gtk. Just to highlight, why it is important to get both working: Winston.jl requires Cairo and Gtk and it will break if only Cairo is working.