Make it possible to override find_program [skip ci] #3218

jpakkane · 2018-03-11T15:29:43Z

This is not yet finished, hence the CI skipping.

The point of this PR is that you can do meson.override_find_program('progname', [program object]) which causes all find_programs for programe to use the given program object instead.

trying to override the name after it has been used is a hard error
trying to override an overridden name is a hard error
works only for ExternalProgram objects but could in theory be expanded to do Executable as well but that has more corner cases
documentation updates are missing

The point of this is to gather comments to see whether this is a usable approach. With this it should be possible to both bootstrap glib by itself and also to make it usable as a subproject on platforms where it is not available.

nirbheek · 2018-03-11T15:52:44Z

I'm really glad you posted this! Just yesterday I was building gstreamer on Windows to prepare for the 1.14 release and it was too difficult to do that without this feature.

trying to override the name after it has been used is a hard error
trying to override an overridden name is a hard error

These are strict requirements, but I can see why they exist. I think it's good to start with these and relax them if/when we find reliable ways to do that.

works only for ExternalProgram objects but could in theory be expanded to do Executable as well but that has more corner cases

Build-time targets do need a bit of extra work, but I think we can have configure_file() working in this PR since they will be available on-disk when meson.override_find_program() is called with the appropriate object.

Without that, it's actually really hard to build anything that uses the GNOME module using subprojects. For instance, if you want to build gstreamer on Windows, you need the glib subproject which provides glib-tools. The most common ones are glib-mkenums, glib-genmarshal and gdbus-codegen all of which are configure_file().

It's the same if you want to build anything that uses introspection on Windows, since the gobject-introspection project provides g-ir-scanner, g-ir-compiler, etc, which are configure_file().

PS: [skip ci] has to be in the commit title, not the PR title 😄

nirbheek · 2018-03-11T15:57:53Z

Actually, supporting the File object returned by configure_file() is almost trivial: just run ExternalProgram on it inside override_find_program_method().

sarum9in · 2018-03-11T16:05:16Z

For distro overrides it would be super convenient to have some flag or option to override specific programs without meson.build modifications. Probably this would be better as separate feature though.

jpakkane · 2018-03-11T16:07:16Z

That would be #3001 again.

textshell · 2018-03-11T16:19:42Z

Looking forward a bit i would like to have this flexible enough to be able to use this for say qt or other situation where the tools are actual compiled programs as well.

From irc, it seems that this could be extended to exectuables, but that using them at meson.build evaluation time would have to be a hard error. That seems like a strange special case for find_program, but maybe that is ok? Or maybe we would want to make this more explicit with some kwarg or an different method than find_program?

ignatenkobrain · 2018-03-17T19:10:39Z

mesonbuild/interpreter.py

+            exe = exe.held_object
+        if isinstance(exe, mesonlib.File):
+            abspath = exe.absolute_path(self.interpreter.environment.source_dir,
+                                                                 self.interpreter.environment.build_dir)


[Flake8]

[E127] continuation line over-indented for visual indent

jpakkane · 2018-03-17T19:10:58Z

Now works with configure_file, please try it out. All tests should pass, documentation still missing.

nirbheek · 2018-04-05T11:26:25Z

So, I found that one particular workflow didn't work: checking if a tool is provided by the system, and taking some action to 'provide' (override) it if it isn't.

This has two use-cases:

Preferring system versions of tools if available (in glib, for example)
Taking some expensive action (such as building a subproject) to provide a tool only when absolutely necessary

I added a WIP commit for that, but this does change the semantics of the system, since now find_program() calls can now pass later even if previous (optional) calls failed. We should think of a better way to expose this, or a better way to cover those use-cases.

Alternatively, we can go with this and say that people should override their programs as soon as possible.

nirbheek · 2018-04-05T11:31:33Z

FWIW, with the commit I pushed, I have gst-build building with just a compiler + meson + ninja on Windows \o/

jpakkane · 2018-04-08T11:37:32Z

Documentation updated. This should be ready for final review.

nirbheek

Also, is my WIP commit still in the list on purpose?

nirbheek · 2018-04-09T12:53:48Z

docs/markdown/snippets/find-override.md

+
+```meson
+prog = find_program('my_custom_script')
+meson.override_find_program('mycodegen', prog)


This needs an example for configure_file() too.

It should probably go to actual documentation, people on average don't read old release notes for stuff. That would require writing a proper page for that, though...

Maybe we should start linking to the release notes for each feature; the (added in v0.xx.y) text is a good candidate for that.

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

nirbheek · 2018-04-11T16:18:23Z

I made all the changes I mentioned, +1 from me now!

nirbheek · 2018-04-15T06:18:37Z

Would be great to have this in 0.46!

nirbheek · 2018-04-15T07:22:26Z

I just realized that a nicer name for this is meson.provide_program(), in case we want to rename it 😄

Several of our tools are installed and are used by other projects to generate code. However, there is no 'install' when projects use glib as a subproject, so we needed some way for glib to 'provide' these tools. Starting from Meson 0.46, this can be done with the meson.override_find_program() function. This also means that we can use these tools ourselves via the GNOME module and don't need to manually create command-lines. WIP because it requires mesonbuild/meson#3218

Closes #2005

Otherwise we can't do the following workflow: if not find_program('foo', required : false).found() subproject('provides-foo') endif Where 'provides-foo' has a meson.override_find_program() on a configure_file() or similar.

Also link to the release notes snippet from the Reference manual

Several of our tools are installed and are used by other projects to generate code. However, there is no 'install' when projects use glib as a subproject, so we needed some way for glib to 'provide' these tools. Starting from Meson 0.46, this can be done with the meson.override_find_program() function. This also means that we can use these tools ourselves via the GNOME module and don't need to manually create command-lines. WIP because it requires mesonbuild/meson#3218

The user doesn't need to know whether or not the program was found, especially not when it's spammed for every gnome.foo() function

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

jpakkane · 2018-04-15T18:25:50Z

I prefer override_find_program because the name is very explicit in what it does, moreso than provides_program. Also if we get the generic overrider functionality it would be nice to consistently use "override" to mean "change the default behaviour of some feature".

nirbheek · 2018-04-16T02:06:08Z

👍

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

ignatenkobrain reviewed Mar 17, 2018

View reviewed changes

jpakkane force-pushed the findoverrider branch from 57a663a to 5dd136b Compare March 17, 2018 19:13

nirbheek force-pushed the findoverrider branch from 5dd136b to fcf2562 Compare April 5, 2018 08:01

jpakkane force-pushed the findoverrider branch from 54e82f9 to 59a361c Compare April 8, 2018 11:37

nirbheek reviewed Apr 9, 2018

View reviewed changes

tp-m added this to the meson-next milestone Apr 10, 2018

tp-m added enhancement gnome gstreamer labels Apr 10, 2018

nirbheek added a commit to centricular/gst-build that referenced this pull request Apr 11, 2018

win-flex-binaries subproject requires newer meson

276471c

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

nirbheek force-pushed the findoverrider branch from 59a361c to f046e4d Compare April 11, 2018 16:17

jpakkane force-pushed the findoverrider branch from f046e4d to 5240533 Compare April 14, 2018 19:35

nirbheek mentioned this pull request Apr 15, 2018

New kwargs for gdbus-codegen: sources #3412

Merged

jpakkane added 4 commits April 15, 2018 13:32

Made it possible to override find_program to return a different program.

de65adb

Closes #2005

Convert Gnome module to use find_program from interpreter.

bdb57cf

Can override programs with scripts generated with configure_file.

4256c0d

Updated all modules to work with the new API.

998892e

nirbheek and others added 4 commits April 15, 2018 13:32

find_program: Only store successful lookups

87c166d

Otherwise we can't do the following workflow: if not find_program('foo', required : false).found() subproject('provides-foo') endif Where 'provides-foo' has a meson.override_find_program() on a configure_file() or similar.

Renamed test dirs to avoid duplicate numbers [skip ci]

648e130

Added documentation [skip ci]

f489aa7

Document that override_find_program works on configure_file [skip ci]

0e6a332

Also link to the release notes snippet from the Reference manual

nirbheek force-pushed the findoverrider branch from 5240533 to fad1540 Compare April 15, 2018 08:28

find_program: Don't spam when called from a module

6cdd14f

The user doesn't need to know whether or not the program was found, especially not when it's spammed for every gnome.foo() function

nirbheek force-pushed the findoverrider branch from fad1540 to 6cdd14f Compare April 15, 2018 08:43

nirbheek added a commit to centricular/gst-build that referenced this pull request Apr 15, 2018

win-flex-binaries subproject requires newer meson

ce8abf9

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

jpakkane merged commit 0e00470 into master Apr 16, 2018

jpakkane deleted the findoverrider branch April 16, 2018 19:22

nirbheek added a commit to centricular/gst-build that referenced this pull request Apr 21, 2018

win-flex-binaries subproject requires newer meson

4652ea3

The patches from the following PR (or 0.46.0, not released yet) are needed: mesonbuild/meson#3218

textshell mentioned this pull request Aug 19, 2018

Allow override_find_program to use an executable. #4055

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make it possible to override find_program [skip ci] #3218

Make it possible to override find_program [skip ci] #3218

jpakkane commented Mar 11, 2018

nirbheek commented Mar 11, 2018

nirbheek commented Mar 11, 2018

sarum9in commented Mar 11, 2018

jpakkane commented Mar 11, 2018

textshell commented Mar 11, 2018

ignatenkobrain Mar 17, 2018

jpakkane commented Mar 17, 2018

nirbheek commented Apr 5, 2018 •

edited

Loading

nirbheek commented Apr 5, 2018

jpakkane commented Apr 8, 2018

nirbheek left a comment

nirbheek Apr 9, 2018

jpakkane Apr 9, 2018

nirbheek Apr 9, 2018

nirbheek commented Apr 11, 2018

nirbheek commented Apr 15, 2018

nirbheek commented Apr 15, 2018

jpakkane commented Apr 15, 2018

nirbheek commented Apr 16, 2018

Make it possible to override find_program [skip ci] #3218

Make it possible to override find_program [skip ci] #3218

Conversation

jpakkane commented Mar 11, 2018

nirbheek commented Mar 11, 2018

nirbheek commented Mar 11, 2018

sarum9in commented Mar 11, 2018

jpakkane commented Mar 11, 2018

textshell commented Mar 11, 2018

ignatenkobrain Mar 17, 2018

Choose a reason for hiding this comment

jpakkane commented Mar 17, 2018

nirbheek commented Apr 5, 2018 • edited Loading

nirbheek commented Apr 5, 2018

jpakkane commented Apr 8, 2018

nirbheek left a comment

Choose a reason for hiding this comment

nirbheek Apr 9, 2018

Choose a reason for hiding this comment

jpakkane Apr 9, 2018

Choose a reason for hiding this comment

nirbheek Apr 9, 2018

Choose a reason for hiding this comment

nirbheek commented Apr 11, 2018

nirbheek commented Apr 15, 2018

nirbheek commented Apr 15, 2018

jpakkane commented Apr 15, 2018

nirbheek commented Apr 16, 2018

nirbheek commented Apr 5, 2018 •

edited

Loading