[RFC 0023] Support musl libc #23
Conversation
Move links to later.
|
👍 from me. |
|
This almost perfectly matches my own thoughts. I’m all for it. 👍 |
|
Given that the only hardware platform that >90% have with root access is And with the current amount of effort towards cross-compilation, don't we have any cross-targets in use by Nixpkgs users where musl is the preferred libc? |
|
IMHO it's best to keep musl support in a branch or overlay so that the effort of maintaining musl-specific patches is not inflicted on package maintainers who don't care about musl. (The same applies to RISC-V support BTW, which really should also be the subject of a RFC.) |
|
I am in favor of a design where everything is in one repo where a broken |
|
Overlays work best when the package set is different. In the case of musl it seems like it would be hard to reach into all the packages with overrides. Nix is quite composable but some times it's not possible to pry open a derivation enough to replace the inner parts (like when a function is hiding the scope). How about accepting the musl patches but not making it a CI failure? It's not official but it's already the case that we have supported platforms and unsupported ones in nixpkgs. |
|
I mean as part of an unsupported platform we would set the expectations: it's okay to break the musl patches as part of the maintenance cycle. The musl team can have their own build and test infrastructure to sweep after breaking changes. |
|
I'm super for it. And I'd like some cross platform to in fact be required by CI too. I think it's good that different platforms make packaging harder, it keeps us and upstream honest. Lastly, Musl, with it's minimal set of features, is a great test because if a package is built with musl, it can probably be built with anything. Just to reiterate, the key to writing portable software is parametricity: don't add endless |
|
@edolstra are you also against the cross-compilation support in the main repository? Making a call on macOS failures is already needed, and with musl I can at least reproduce the problem locally and see if it is easy to fix; so it is just the question about the level of expected stability. |
|
@Ericson2314 it's like every platform. First it should go through a trial phase where compilation success is not required. Once it has been demonstrated as a viable target it can be promoted to the supported platforms. The question is, who is responsible for deciding which platforms are supported or not :) |
|
@zimbatm Agreed, I think a |
|
Alright let's do this. Any objections @edolstra ? |
|
I should add that @dtzWill was a bit more to the point in the original RFC than me pontificating on parametricity above. I think prior portability efforts were a bit more "conditional-explosion-y" than the current ones: the patches we're capable of landing today really do improve the software in code quality too, and I'd love to upstream them. Given the necessary ugliness of earlier efforts, I understand and sympathize @edolstra's reservations. Given the elegance we can get away with now, I don't think those concerns still apply. |
|
I've noticed the RFC as-is doesn't go much in to how maintenance will work:
We should also take care to remember this adds a bit more complexity to many derivations. Does musl work on macOS? What happens if someone tries to use musl on MacOS in Nixpkgs? |
rfcs/0023-musl-libc.md
Outdated
| feature: musl-libc | ||
| start-date: 2018-02-19 | ||
| author: Will Dietz | ||
| co-authors: (find a buddy later to help our with the RFC) |
There was a problem hiding this comment.
I'm concerned that a whole new libc only has one "sponsor" here.
There was a problem hiding this comment.
The more the merrier, I think! This writes down who can be pinged for future musl problems.
There was a problem hiding this comment.
Would it be a hard task?
|
@graham I think the idea is that initially, nothing is supported, but merely allowed to exist.
The only way to is to build, but the package maintainer isn't obligated to do this.
Depends how the experiment goes
Yes. Packages that don't will probably be eventually broken on glibc as it becomes more standards-enforcing. (I didn't know this until I read the RFC. This is great news as too much of GNU doesn't require opt-in to extra GNU-specific features.)
Nothing. Well almost nothing; there are a few odd jobs in release-cross.nix.
We don't need to for an unsupported experiment. |
I don't love the idea of having unsupported, unexercised code sitting in Nixpkgs. What is the point of having it in nixpkgs, if it is allowed to be broken by people who don't even know it is being broken? At that level of "unsupported", there is very little reason to have it in the repo. Nixpkgs receives almost 500 commits a week, and a lot of rot can set in over that time. I think if we're going to add major new support, part of the experiment should be at least diagnostics-level checking of the code per PR. If you disagree with this, please pretend we're a few months down the line and musl is a resounding success and I re-asked all those questions. P.S. I feel positive about this project. |
rfcs/0023-musl-libc.md
Outdated
| I believe the linked implementation from current nixpkgs master is perhaps | ||
| the best embodiment of "Detailed Design"? | ||
|
|
||
| Linky: [bd7d5b365799a145717d31122ebfd24b51fea117](https://github.com/NixOS/nixpkgs/commit/bd7d5b365799a145717d31122ebfd24b51fea117) |
There was a problem hiding this comment.
I suspect this link is wrong :)
There was a problem hiding this comment.
I think it's right, it's just revision of master at time of writing.
There was a problem hiding this comment.
Maybe a diff or merge commit would be better?
rfcs/0023-musl-libc.md
Outdated
| author: Will Dietz | ||
| co-authors: (find a buddy later to help our with the RFC) | ||
| author: Will Dietz (@dtzWill) | ||
| co-authors: Shea Levy (@shlevy) |
|
A new libc or port is simply not something that can realistically live out of tree, overlays etc. aren't expressive enough. Saying that we need a full support story before we can start work proving viability, attracting maintainers, and building userbase is effectively saying "go fork nixpkgs, this is never going to happen upstream." Or if you disagree, how exactly do you propose we go about this kind of thing such that we have a realistic chance of ever getting it merged, given the above constraints? |
|
The point of having it in nixpkgs is so that people can use it, and iterate on it. People undertaking these efforts understand there is burden on them to do whatever form of CI they need on their own resources until there is enough community support for more. If things bitrot, that's what |
|
I hesitate to post this because I think it's a worse solution than allowing it upstream, but: if the ultimate answer here is no (and, setting this particular RFC aside, we don't seem to have any way to get to an ultimate answer in these...), then I'll (on my own fork) start To reiterate, this is a worse solution than finding a way to let this kind of work happen upstream. We're not the kernel community, and the last thing we need is further impaired discoverability, further divergence in possible user configurations, and more third-party infrastructure. But I'm tired of ideas being shot down because of potential maintenance burden without any viable alternative existing, so there's a viable alternative. |
|
You can also count on IOHK helping maintain musl (as a consumer of the work). About the integration: I'd start with one codebase (nixpkgs master) and decide otherwise if it's too much conditionalization once the prototype is ready for review. |
|
OK, I think this is a go then. @dtzWill any modifications before we merge? |
|
I'd say double check with @grahamc that his questions are addressed. That should be quick back and forth. |
Following these, as far as I'm concerned, |
|
I would say that if a musl patch is changing just the assumption «Linux = Android xor glibc», it could be OK to delay its application on Darwin to reduce the number of rebuilds, but eventually applying it could indeed be a good idea. |
|
As a non-contributor, I don't know if my opinion carries any weight at all, but here it is anyway:
tl;dr I can't speak to whether this approach to supporting musl-in-nixpkgs is the right one, but I can say I'd definitely be a user of musl-in-nixpkgs and so the need addressed by this RFC is (for me, anyway) real. |
|
Alright, I suppose it's ready? 😇 @grahamc I liked the team idea so I put it in the RFC, hope that's alright 👍. |
|
I'm interested in being the part of the musl team, too. |
Hear, hear! |
Add myself to the "musl team"
|
(... I just switched my NixOS profile to use musl-built Nix 2 😁 ...) Note that this currently requires a small patch to Nix to avoid stack-allocating large buffers: dtzWill/nix@b538660 |
@Ericson2314: do you mean that you want the musl-support patches to keep getting cherry-picked to the stable 18.03 branch already? EDIT: 18.03 is very close to (belated) branching ATM. |
|
@vcunat I guess I didn't think that far; correct me if I'm wrong @dtzWill, but I thought stream of musl fixes and slowed to a trickle :). All I was thinking is if we're going to go through with this, it would be a cool thing to publicize (somewhat, is it is technically an experiment). If we end up going ahead but only decide after after 18.03, we miss a good window to give the experiment more visibility. |
|
Merged this because it reflects the current situation. |
|
It seems the 1.1.19 -> 1.1.20 upgrade broke GCC for me: NixOS/nixpkgs#46064 (comment) |
Discussed during NAT meeting #23 Co-authored-by: John Ericson <[email protected]>
* copy rfc template, unnumbered as of yet * musl-libc RFC: first draft * musl-libc RFC: rename to patch pull req number * musl-libc: whoops, no links in top bit, that's metadata. Move links to later. * record the name of a brave co-author * musl rfc: squash second batch of changes * rfc: specify initial team \o/, add a small clarification. * Add myself to the "musl team"
WIP, but comments and discussion welcome throughout the process :).
RFC 23 - support musl libc (rendered)