-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set project and meson options in cross/native files #6597
Conversation
@marc-h38, I think this was what you were wanting. |
I haven't read the whole thing yet, but one thing I'm wondering is why do you split options in different sections? I was expecting a simpler format, like the one we already use in [options]
prefix = '/opt/gnome'
default_library = 'both'
ffmpeg:default_library = 'static'
glib:gtk_doc = 'disabled' That way you can load that into a single dict that would match exactly as if the user passed them with I think one common use-case is when you have only a cross file, and I think you should be able to set all options from there. But then what happens if you have both a cross file and a native file with options defined in both? I guess we should abort with an error if the same option is set in both files. That would be similar to what we currently do when passing both |
f7e39f8
to
6f4e15b
Compare
@xclaesse I really want to get rid of the one off format in cmd_line.txt and replace it with a native/cross file.
Composition, for one. cross and native files can be composed by passing the argument more than once. Since each component is it's own section you can re-use a cross file in more than one build. Say you need zlib in two different projects, and you want to statically link it in both cases, now you can have a single zlib file. For meson builtin versus project arguments it makes sense to split them as well, because they're conceptually different. Project arguments by definition apply only to one project, meson builtins (at least generally) apply to all projects, so being able to specify them separately makes sense to me.
This is clearly defined. If you have both a cross and native file |
I'm so glad you posted this, because I was going to start working on it myself today. :) It would be great to get all functionality needed for configuration so we can merge #6595 to get a general solution for all options rather than a one-off feature for one of them. One possible syntax change comes to mind. Rather than doing this: [meson options]
c_std = 'c99'
[zlib:meson options]
default_library = 'static' could we instead do this: [meson options]
c_std = 'c99'
zlib:default_library = 'static' |
I thought about that, but I really dislike it because it means you have to repeat yourself a lot if you have more than one options: [meson options]
zlib:foo = 'bar'
zlib:bar = 'foo'
zlib:thing = false vs [zlib:meson options]
foo = 'bar'
bar = 'foo'
thing = false And on the implementation side this makes things easier because we know what subproject we're in when we're applying defaults, so we load the right options from the start and don't have to loop over them asking "are you a zlib option?"
I've based this on @xclaesse's series, I'm not sure exactly what parts you don't like, but all of the machinery he put in place except in the interpreter changes is needed to make per-subproject builtins work. The language options are more complicated because they're generated dynamically. I have a branch for that, but there are still bugs and no tests. It's more complicated because, again, language options are added when a language is added to meson, we don't by default have |
Per-project values of options should only be settable via the native/cross file. There are usually more than one of these, so concentrating them all in one place makes it easier to review. |
I think we should be able to set them via the command line. We allow setting project options on the command line, and we're going to get no end of complaints about |
It's also going to be very inconsistent to not be able to set arguments on the command-line like that. We already have a bunch of unavoidable inconsistency in our argument handling, let's not increase it. Setting arguments on the command-line vs a file is also much clearer for CI purposes since it's right there in the yaml file, and it shows up in the CI logs for debugging. |
Would this make it possible to set the Ninja binary location as well in a native file, to avoid use of NINJA or PATH environment variable? |
@scivision That would be a separate change, because I don't think we need any infrastructure changes for that, we just need to find ninja using the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought the only conceptual difference between "meson build options" and "project options" was: meson options apply globally to all (sub)projects whereas project options are (sub)project specific. No conceptual difference when only one project. Simple, but now I'm confused by these:
For meson builtin versus project arguments it makes sense to split them as well, because they're conceptually different. Project arguments by definition apply only to one project, meson builtins (at least generally) apply to all projects
...
You can set some meson level options on a per-subproject basis (!)
Emphasis mine.
I've read https://mesonbuild.com/Build-options.html (please rename to "meson options") and https://mesonbuild.com/Reference-manual.html#project a good few times and I'm still confused.
While we're very far from CMake yet, I'm starting to have this sinking feeling: #5859 (comment).
While this, #3001, #4637, #5061, etc. all seem very useful, I'm worried they will increase the configuration possibilities and complexity before clearly and briefly documenting the high-level and already existing concepts behind them. BTW "whatever is well conceived is clearly said, and the words to say it flow with ease." So either the difference between meson level options and project level options can be stated in one or two completely unambiguous sentences, or there's some design issue with them.
I understand life is not simple and build systems much less, however it should at least be possible to describe the top-level concepts and features clearly and briefly. Maybe I only found low-level / reference documentation and missed these high-level descriptions? In that case apologies and thanks for the pointer.
Feeling of sinking much deeper with c_args #6362
docs/markdown/Machine-files.md
Outdated
Native files allow layering (cross files can be layered since meson 0.52.0). | ||
More than one native file can be loaded, with values from a previous file being | ||
overridden by the next. The intention of this is not overriding, but to allow | ||
composing native files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While overriding is not the intention, it's still allowed correct? For instance you could have a clang-common.ini
with -foptimization-fubar=true
but -foptimization-fubar=false
in clang5.ini
and only in clang5.ini
because clang 5 is broken for that feature. This already works BTW.
I still strongly prefer that syntax over splitting into different sections. We use the I also do not agree this supersede cmd_line.txt, we still have to remember options set from command line and from Ultimately what I would like to see happening is projects shipping a set of cross files with their source code, that contains default values to build for specific targets (e.g. Android NDK, Linux distro mingw, etc). That often means specific values for a set of options, but still allowing them to be overriden from command line by the user. |
Also, nitpicking, but I don't think we need "meson" in |
While pretty much everything around here is meson-related and the term "meson" is almost as vague and overloaded as "build", I suspect |
meson_options.txt only contains project options, so that would be a misleading reference since in the current proposal |
Mmmm... so this PR wants to add
... yet Am I really the only one worried about the concepts and terminology? As a sanity check, do you guys try from time to time to discuss these terms and concepts with developers not familiar with meson? |
I'm glad you volunteer to improve our documentation, there is certainly a lot to do :D |
@xclaesse to a reasonable extent I do volunteer to improve the documentation, in fact I already started that a little bit in other places. However no one can improve the documentation if developers use the terms "meson options" and "project options" in one place but vice-versa in another (really?), in that case it's a lost cause. |
I never checked what the doc says exactly. From meson_options.txt are project options because they are the options defined by the project, and not defined by meson itself. |
And I'm strongly in favor of having separate sections for each project. I think it makes sense because of composition (ie, mesa will probably ship a config file for just zlib for windows, but unless you're doing a windows->windows build you'll need to compose it with another cross file) and it makes it clear, at a glance, exactly what projects are being modified by the file.
How about "builtin options" then? I'd like to make it clear that one section is for meson level arguments, and the other is for options defined in your meson_options.txt, which I guess should be called "build options" as it is in the docs? |
Composition works just fine with a single section, they will be merged.
The simple fact that it's hard to name is already a proof we should not split options, because whatever name we pick it will not be understood by the user without reading long documentation. If we insist on splitting options, it should at least follow the naming we have in |
Certainly, but if you have three files [zlib:builtin options]
c_std = 'c99'
c_args = ['-someopt'] [expat:builtin options]
c_std = 'gnu99' [meson options]
c_std = 'c11'
cpp_std = 'c++11' It's super clear what is being done in each one, the first only only affects zlib, I can see that just by reading the section header, the second one only affects expat, and the thrid only affects the primary project. compare that to: [builtin options]
zlib:c_std = 'c99'
zlib:c_args = ['-someopt'] [builtin options]
expat:c_std = 'gnu99' Where I can't tell at a glance that a file affects more than one project, but lets have a bigger more complicated example: [zlib:builtin options]
c_std = 'c99'
c_args = ['-foo', '-bar', 'something else']
b_ndebug = true
b_lto = false
[expat:builtin options]
c_std = 'gnu99'
c_args = ['-foo']
c_link_args = ['-Wl,--arg']
b_ndebug = true
[builtin options]
b_ndebug = true
c_std = 'c11' vs ```ini
zlib:c_std = 'c99'
zlib:c_args = ['-foo', '-bar', 'something else']
zlib:b_ndebug = true
zlib:b_lto = false
expat:c_std = 'gnu99'
expat:c_args = ['-foo']
expat:c_link_args = ['-Wl,--arg']
b_ndebug = true
c_std = 'c11' The first way is much more readable to me, and it forces you to keep per-project options sorted, you would end up with [builtin options]
c_std = 'c99'
zlib:b_ndeug = false
c_args = ['foo', 'bar']
c_link_args = ['-Wl,-something']
expat:c_std = 'c11'
zlib:c_args = ['-foo']
expat:b_ndebug=false
That's a terrible argument. Naming things is one of the hardest things in computer science, by that logic no software should be written at all! But I don't think we're going to solve this by arguing about it anymore, clearly I like blue bikesheds and you like yellow ones. Maybe some other people have opinions? |
Exactly what I have in mind, pretty readable to me, it's a matter of coding style, like in any language. It's also exactly what I already have in script that calls meson with all options set.
Indeed, I think it's pretty clear we'll never agree, so we need others to vote :-) |
Monday morning, fresh mind let's give it a go.
I found the name "User" in the first sentence of that page, didn't take me much effort. "User" is IMHO a relatively good negation of "built-in" but please suggest better ones. Nothing gets called "build options" because in a meson context, it's hard to come with a name more vague, less "searchable" and generally less useful than "build". In the same meson context, the name... "meson" (!) options is slightly less vague but barely. The name "meson BUILTINS" is tolerated in the rare case of potential confusion with some other sort of builtins. BTW "build" is already used in the build/host/target autoconf triplet which is very relevant to... cross files! Another, huge reason not to overload it with anything else in the same area. It's a bit too late to change autoconf names, like 30 years late. There's also a "build step", a "build directory" and I bet we could find other "build"/nameless things (most of them also much older than meson). In case anyone doubts "build" is an empty placeholder, see how "build options" sometimes refers to not-builtins-options, example here: https://mesonbuild.com/Builtin-options.html
In some other places "build options" refers to ALL option kinds! Example: https://mesonbuild.com/Configuring-a-build-directory.html
https://mesonbuild.com/Subprojects.html#build-options-in-subproject The names "Project and "Global" are forbidden outside subproject-related topics to avoid overloading and intolerable confusion. I really, really hope I didn't misunderstand the concepts here. If I have please correct the concepts and do not discuss the names I just suggested as gold standards, that would be pointless. I may have misunderstood the concepts because... they don't have good, standard names yet :-) |
Once we have the big bikesheding debate worked out I'll also rename the sections to match our documentation, |
Which is super helpful in debuggers
This puts all of them together, in the next patch they'll be pulled back out, but it's convenient to start that refactor by moving them all there, then moving them into env as a whole.
This creates a full set of option in environment that mirror those in coredata, this mirroring of the coredata structure is convenient because lookups int env (such as when initializing compilers) becomes a straight dict lookup, with no list iteration. It also means that all of the command line and machine files are read and stored in the correct order before they're ever accessed, simplifying the logic of using them.
0622bf9
to
4d2a170
Compare
Okay, I think I have them all resolved now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, LGTM.
Test failure is unrelated. |
I'm going to merge then. \o/ |
@dcbaker I'm sorry I've been too arch last week. I know you worked a lot on this PR but to be honest I've said many times I don't like neither the syntax nor the implementation. I have suggested multiple times a much simpler implementation that could have been merged months ago. I personally need this feature and like it a lot, but I'm not the one to give the +1 on it given that I'm not happy with how it's done. That being said, cool that it got merged, I'll read that code while rebasing my branch. |
You got PKG_CONFIG_PATH wrong, the env override value from native file, and it warn uselessly on reconfigure if you have the value in native file. The warning should be removed, it never worked (was dead code before this PR). It requires more work to actually do it properly. I'm preparing a fix. |
Wait, do I understand this correctly: Env variables override stuff in native/cross files? If yes, why? |
It's a bug in this PR, I'll fix it. |
OK, thanks for clarifying. I thought this was a general thing in meson for a second :) |
Actually, a comment suggests it's intentional: https://github.com/mesonbuild/meson/pull/6597/files#diff-638d32e2abc720f45ee2279d5befe1e5R690 @dcbaker what's the rational to make that change? Seems weird that -Dpkg_config_path=foo overrides PKG_CONFIG_PATH but not setting pkg_config_path in machine file. |
I don't think Before this PR if you run |
I did also think the precedence we CLI > config files > env vars. |
|
It also print that warning when the main project sets options in project(): |WARNING: In subproject sub1: Unknown options: "sub1:buildtype, sub1:cpp_eh, sub1:cpp_std, sub1:prefix, sub1:sub1:test_option" |
@nirbheek @Ericson2314 Since you gave approval to this PR, did you make a in depth review of it and approve this change? Do you agree with @dcbaker that his approach is better and more maintainable than #7293? Since I get no support from anyone when pushing for a simpler solution, and if it is a well thought decision, then I'll have to accept the decision of the majority. |
This is a partial implementation of #3001, but not everything.
This series allows meson builtins and project options to be specified in the cross and/or native files. This should make it easier to set up subprojects in a reliable way, as all of the options can be kept in a native/cross file. With file layering a projects can even ship preconfigured options, that the end user can overlay with their own cross/native files.
For mesa we'd like to couple this with the ability to set some builtin options (namely default_library) on a per-subproject basis to control wraps in a predictable way.