Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kconfig Module #5031

Merged
merged 3 commits into from
Mar 20, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions docs/markdown/Kconfig-module.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
short-description: Unstable kconfig module
authors:
- name: Mark Schulte, Paolo Bonzini
years: [2017, 2019]
has-copyright: false
...

# Unstable kconfig module

This module parses Kconfig output files to allow use of kconfig
configurations in meson projects.

**Note**: this does not provide kconfig frontend tooling to generate a
configuration. You still need something such as kconfig frontends (see
link below) to parse your Kconfig files, and then (after you've
choosen the configuration options), output a ".config" file.

[kconfig-frontends]: http://ymorin.is-a-geek.org/projects/kconfig-frontends

## Usage

The module may be imported as follows:

``` meson
kconfig = import('unstable-kconfig')
```

The following functions will then be available as methods on the object
with the name `kconfig`. You can, of course, replace the name
`kconfig` with anything else.

### kconfig.load()

This function loads a kconfig output file and returns a dictionary object.

`kconfig.load()` makes no attempt at parsing the values in the
file. Therefore, true boolean values will be represented as the string "y"
and integer values will have to be converted with `.to_int()`.

Kconfig frontends usually have ".config" as the default name for the
configuration file. However, placing the configuration file in the source
directory limits the user to one configuration per source directory.
In order to allow separate configurations for each build directory, as is
the Meson standard, `meson.build` should not hardcode ".config" as the
argument to `kconfig.load()`, and should instead make the argument to
`kconfig.load()` a [project build option](Build-options.md).
bonzini marked this conversation as resolved.
Show resolved Hide resolved

* The first (and only) argument is the path to the configuration file to
load (usually ".config").

**Returns**: a [dictionary object](Reference-manual.md#dictionary-object).
5 changes: 5 additions & 0 deletions docs/markdown/snippets/kconfig.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## New module to parse kconfig output files

The new module `unstable-kconfig` adds the ability to parse and use kconfig output
files from `meson.build`.

1 change: 1 addition & 0 deletions docs/sitemap.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ index.md
Simd-module.md
Windows-module.md
Cuda-module.md
Kconfig-module.md
Java.md
Vala.md
D.md
Expand Down
3 changes: 3 additions & 0 deletions mesonbuild/interpreter.py
Original file line number Diff line number Diff line change
Expand Up @@ -2119,6 +2119,9 @@ def build_func_dict(self):
def holderify(self, item):
if isinstance(item, list):
return [self.holderify(x) for x in item]
if isinstance(item, dict):
return {k: self.holderify(v) for k, v in item.items()}

if isinstance(item, build.CustomTarget):
return CustomTargetHolder(item, self)
elif isinstance(item, (int, str, bool)) or item is None:
Expand Down
72 changes: 72 additions & 0 deletions mesonbuild/modules/unstable_kconfig.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Copyright 2017, 2019 The Meson development team

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from . import ExtensionModule

from .. import mesonlib
from ..mesonlib import typeslistify
from ..interpreterbase import FeatureNew, noKwargs
from ..interpreter import InvalidCode

import os

class KconfigModule(ExtensionModule):

@FeatureNew('Kconfig Module', '0.51.0')
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.snippets.add('load')

def _load_file(self, path_to_config):
result = dict()
try:
with open(path_to_config) as f:
for line in f:
if '#' in line:
comment_idx = line.index('#')
line = line[:comment_idx]
line = line.strip()
try:
name, val = line.split('=', 1)
except ValueError:
continue
result[name.strip()] = val.strip()
except IOError as e:
raise mesonlib.MesonException('Failed to load {}: {}'.format(path_to_config, e))

return result

@noKwargs
def load(self, interpreter, state, args, kwargs):
sources = typeslistify(args, (str, mesonlib.File))
if len(sources) != 1:
raise InvalidCode('load takes only one file input.')

s = sources[0]
if isinstance(s, mesonlib.File):
# kconfig input is processed at "meson setup" time, not during
# the build, so it cannot reside in the build directory.
if s.is_built:
raise InvalidCode('kconfig input must be a source file.')
s = s.relative_name()

s = os.path.join(interpreter.environment.source_dir, s)
if s not in interpreter.build_def_files:
interpreter.build_def_files.append(s)

return self._load_file(s)


def initialize(*args, **kwargs):
return KconfigModule(*args, **kwargs)
1 change: 1 addition & 0 deletions run_project_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -550,6 +550,7 @@ def detect_tests_to_run():
('failing-meson', 'failing', False),
('failing-build', 'failing build', False),
('failing-test', 'failing test', False),
('kconfig', 'kconfig', False),

('platform-osx', 'osx', not mesonlib.is_osx()),
('platform-windows', 'windows', not mesonlib.is_windows() and not mesonlib.is_cygwin()),
Expand Down
3 changes: 3 additions & 0 deletions test cases/kconfig/1 basic/.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
CONFIG_VAL1=y
# CONFIG_VAL2 is not set
CONFIG_VAL_VAL=4
16 changes: 16 additions & 0 deletions test cases/kconfig/1 basic/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
project('kconfig basic test')

k = import('unstable-kconfig')
conf = k.load('.config')

if not conf.has_key('CONFIG_VAL1')
error('Expected CONFIG_VAL1 to be set, but it wasn\'t')
endif

if conf.has_key('CONFIG_VAL2')
error('Expected CONFIG_VAL2 not be set, but it was')
endif

if conf.get('CONFIG_VAL_VAL').to_int() != 4
error('Expected CONFIG_VAL_VAL to be 4')
endif
2 changes: 2 additions & 0 deletions test cases/kconfig/2 subdir/.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CONFIG_IS_SET=y
# CONFIG_NOT_IS_SET is not set
13 changes: 13 additions & 0 deletions test cases/kconfig/2 subdir/dir/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@

k = import('unstable-kconfig')

conf = k.load(meson.source_root() / '.config')
bonzini marked this conversation as resolved.
Show resolved Hide resolved

if not conf.has_key('CONFIG_IS_SET')
error('Expected CONFIG_IS_SET to be set, but it wasn\'t')
endif

if conf.has_key('CONFIG_NOT_IS_SET')
error('Expected CONFIG_NOT_IS_SET not be set, but it was')
endif

4 changes: 4 additions & 0 deletions test cases/kconfig/2 subdir/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
project('kconfig subdir test')

# Test into sub directory
subdir('dir')
2 changes: 2 additions & 0 deletions test cases/kconfig/3 load_config files/dir/config
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CONFIG_IS_SET=y
# CONFIG_NOT_IS_SET is not set
13 changes: 13 additions & 0 deletions test cases/kconfig/3 load_config files/dir/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@

k = import('unstable-kconfig')

conf = k.load(files('config'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not used kconfig all that much, but shouldn't we load the output file? Specifically something that will get generated in the build dir rather than reading stuff from the source dir? Loading stuff from source dir means that you can have only one configuration per source dir, not one per build dir as is the Meson standard.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not used kconfig all that much, but shouldn't we load the output file? Specifically something that will get generated in the build dir rather than reading stuff from the source dir? Loading stuff from source dir means that you can have only one configuration per source dir, not one per build dir as is the Meson standard.

Indeed, since Meson only supports out-of-tree build, it is somewhat unfortunate that the default for kconfig output is .config in the toplevel source directory. However, the output file can be overridden using an environment variable, usually specified like

$ make menuconfig KCONFIG_CONFIG=foo.config

I expect that users of this module will make the location of the configuration file a Meson option too, so that the same file can be specified both as KCONFIG_CONFIG and when invoking Meson. I will add a note to the documentation.

Of course, having to specify the directory twice is ugly; unfortunately on one hand the build directory is not known when you invoke the kconfig frontend, and on the other hand the kconfig frontend must be invoked before Meson. One possible fix is to provide some kind of script that wraps meson setup and invokes kconfig before Meson (removing the need to ship a Makefile), not unlike those Meson users that are providing a simple-minded configure script. That script can then pass a file in the build directory and pass it to both kconfig and Meson.

That said, the placement of the configuration also depends on the usecase. For example, QEMU only uses kconfig as a common format to store pre-defined configuration_data for each binary; it does not employ any front-end, instead all processing of the configuration happens in meson.build. In that case, the configuration files are fixed and committed, and they definitely come from the source directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the module is unstable, we can change it later. We can merge this and change the behaviour as people test it on real world code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like:

k.generate_and_load(init_command: ['kconfig', 'source_file', 'output_file'])

That runs the given command if the output .config file does not exist in the build dir and otherwise just loads it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems sensible but I would first try and see how people use this. The config frontends usually have their own make-based build system, and invoking make from Meson would be a bit weird...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like:
k.generate_and_load(init_command: ['kconfig', 'source_file', 'output_file'])

If you were referring to: genkconfig .config -> config.h, then yes, I wrote a custom_target that does just that.

If you were referring to generating the default .config automatically with olddefconfig then no, it's very easy but apparently considered bad practice in the (interactive) Kconfig world to automagically build the default configuration. This must be an explicit choice from the user as seen in the Linux kernel example.


if not conf.has_key('CONFIG_IS_SET')
error('Expected CONFIG_IS_SET to be set, but it wasn\'t')
endif

if conf.has_key('CONFIG_NOT_IS_SET')
error('Expected CONFIG_NOT_IS_SET not be set, but it was')
endif

4 changes: 4 additions & 0 deletions test cases/kconfig/3 load_config files/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
project('kconfig subdir test')

# Test into sub directory
subdir('dir')
2 changes: 2 additions & 0 deletions test cases/kconfig/4 load_config builddir/config
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CONFIG_IS_SET=y
# CONFIG_NOT_IS_SET is not set
14 changes: 14 additions & 0 deletions test cases/kconfig/4 load_config builddir/meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
project('kconfig builddir test')

k = import('unstable-kconfig')

configure_file(input: 'config', output: 'out-config', copy: true)
conf = k.load(meson.build_root() / 'out-config')

if not conf.has_key('CONFIG_IS_SET')
error('Expected CONFIG_IS_SET to be set, but it wasn\'t')
endif

if conf.has_key('CONFIG_NOT_IS_SET')
error('Expected CONFIG_NOT_IS_SET not be set, but it was')
endif