References a partition, disk, volume or whatever you like instead of
using a device path or label. Creating those storage devices is done by
nixpart and we can infer the right labels and/or poths from the device
tree.

I've added those hooks here, because duplicating things such as fsType,
label, options or mountPoint in the storage configuration look kinda
pointless to me.

Signed-off-by: aszlig <[email protected]>

This is not the final version, because I'm not yet sure whether we want
BTRFS as a special option here. Also, we don't check types properly yet.

Signed-off-by: aszlig <[email protected]>

All of the types are just stubs right now, but I actually forgot
volgroupType, as I didn't do tests of LVM in nixpart so far yet.

Signed-off-by: aszlig <[email protected]>

Currently, this is still WIP and subject to change, but it helps to see
whether our storage configuration options actually work out the way we
want.

Still needs a lot of cleanup, especially regarding the -m option, where
I'm not sure whether we should do it with nixpart or write our own
lightweight solution to be built into NixOS.

Signed-off-by: aszlig <[email protected]>

Renames the test to closer match the NixOS module attribute and put it
into release.nix. Of course, those tests still fail, because nixpart is
still WIP and I haven't pushed the first version yet.

Signed-off-by: aszlig <[email protected]>

We have renamed the function to nixpart() already, so let's make sure we
rename it accordingly.

Signed-off-by: aszlig <[email protected]>

We're internally calling nix-instantiate to get the required options
from the configuration, so let's pass it through an option that skips
this step.

Signed-off-by: aszlig <[email protected]>

Let's use relative paths instead, because the version in <nixpkgs>
isn't necessarily the same as the current nixpkgs tree.

Signed-off-by: aszlig <[email protected]>

If we now use something like this:

pkgs.libselinux.override {
  enablePython = true;
  python = pkgs.python3;
}

... we now no longer will get the following error:

selinuxswig_wrap.c:143:21: fatal error: Python.h: No such file or directory

This happens because the Makefile is using the Python interpreter to get
the right library dir for Python:

PYTHONLIBDIR ?= $(shell $(PYTHON) -c "
  from distutils.sysconfig import *;
  print(get_python_lib(1))
")

Signed-off-by: aszlig <[email protected]>

This is basically multipath-tools patched into oblivion so that it's
resulting into a "mpathconf" script.

Although the patches apply to a lot more than just mpathconf and its
manpage, we use filterdiff to only apply all the patches that apply to
either mpathconf or mpathconf.[0-9]* so that we don't need to have the
original source of multipath-tools.

The reason I'm packaging this is that it's needed for libblockdev to
configure multipath devices.

As the description says, it's for editing /etc/multipath.conf so in the
long run let's see whether we actually need this if we can patch
libblockdev so it's not relying on a global multipath.conf at runtime.

Signed-off-by: aszlig <[email protected]>

This is a package that is required for running common tests against
projects from RedHat/Fedora, for example libbytesize.

I'm removing nix_run_setup.py before the check phase because that files
triggers a few pylint warnings and in turn causes the tests to fail.

Note that I don't use rm -f here to make sure the build fails once we no
longer need the nix_run_setup.py file so we can remove the reference
from the pocketlint as well.

Tested by building against Python 3.3, 3.4, 3.5 and 3.6.

Building against Python 3.6 has failed because of a test failure in
pylint, so it's only a transient failure.

Signed-off-by: aszlig <[email protected]>

This is needed for libblockdev, one of its implicit dependencies.

In order to successfully build the docs, we needed to pass
XML_CATALOG_FILES so that xsltproc (called by gtkdoc-mkhtml) is able to
find them.

The package only works with Python 3, so I'm explicitly depending on
python3Packages.

Tested by building against i686-linux and x86_64-linux, which is why I
set platforms to only Linux (don't have anything else to test on).

Signed-off-by: aszlig <[email protected]>

The patch is from Fedora:

http://pkgs.fedoraproject.org/cgit/rpms/parted.git/tree/0025-Add-libparted-fs-resize.pc.patch

It adds the pkgconfig file via configure.ac and Makefile.am, so we need
to use autoreconfHook and pkgconfig as well. The latter is needed so
that PKG_CHECK_MODULES macros are properly replaced in the resulting
configure script.

Built with success on i686-linux and x86_64-linux.

Signed-off-by: aszlig <[email protected]>

One of the requirements of libblockdev.

I'm using "python" without specifying an explicit Python version, so
that it's easy to override the Python version via something like this:

  volume_key.override { inherit (python3Packages) python; }

Tested by building against i686-linux and x86_64-linux with Python 2.7
and Python 3.5.

Signed-off-by: aszlig <[email protected]>

Needed for the current version of blivet.

This library does a lot of calls to external binaries, so we need to
patch in a lot of paths (bin-paths.patch).

The checkPhase currently only checks whether the paths to these binaries
are correct but doesn't run the real tests, which require root
permissions.

We also depend only on Python 3.x, because Python 2.x support seems
broken at the moment and it really doesn't make sense to support it via
patching on our side while it's eventually becoming obsolete.

Other than the default output there is also a "tests" output which is
going to be used by the upcoming NixOS VM test to run the libblockdev
tests within a VM.

Signed-off-by: aszlig <[email protected]>

Patch is originally from:

https://bug735669.bugzilla-attachments.gnome.org/attachment.cgi?id=289405

It's part of the following bug report:

https://bugzilla.gnome.org/show_bug.cgi?id=735669

I stumbled on this while running the libblockdev tests. Unfortunately we
haven't seen a new release of parted yet, so I'll add this patch for
now.

Tested by building against i686-linux and x86_64-linux.

Signed-off-by: aszlig <[email protected]>

The six library is a dependency of the library, so add it to the
propagatedBuildInputs.

In addition, pygobject3 and gobjectIntrospection need to be propagated
as well if we want to use this library from within Python, which is its
primary use-case.

If we don't want this propagation, we could add a flag "withPython" or
something similar so that the Python-specific parts are bound to a
condition. Right now however, the only package which is going to use
libblockdev is blivet.

Signed-off-by: aszlig <[email protected]>

The main reason I'm updating this is because the libblockdev tests
assume that lvm has proper default cache backend support, which only was
added in recent versions.

Upstream changelog can be found at:

https://git.fedorahosted.org/cgit/lvm2.git/tree/WHATS_NEW?h=v2_02_168

Tested against the NixOS installer test (tests.installer.lvm).

Signed-off-by: aszlig <[email protected]>
Cc: @7c6f434c

First of all the purity.patch is no longer referenced, so instead of
keeping it rotting around in the source tree, let's remove it. We can
still bring it back if we really need it.

Next, let's move version attribute to the derivation attributes, because
this is what we adopted as a standard for other packages. Plus I
couldn't find any package within the source tree where this is
referenced.

The systemd generator and unit files are now installed via the make
files provided by the upstream package. This also means that the name of
"blk_availability_systemd_red_hat.service" now is
"blk-availability.service" and resides in $out/lib/systemd/system
instead of $out/etc/systemd/system. Again, I haven't found anything that
references either of these paths.

We now also install the standard lvm config files in $out/etc/lvm,
although lvm does not search these paths by default. We might want to
fix this later.

Tested against the tests.installer.lvm NixOS VM test.

Signed-off-by: aszlig <[email protected]>
Cc: @7c6f434c

For example the following fails on NixOS:

lvcreate --profile thin-performance ...

It tries to look for a thin-performance.profile file in
/etc/lvm/profile, which by default doesn't exist.

So in order to make those profiles work we now set a default profile
directory which is used if lvm doesn't have a configuration file.

Tested against the tests.installer.lvm NixOS VM test.

Signed-off-by: aszlig <[email protected]>
Cc: @7c6f434c

This is going to be used for libblockdev. I'm a bit hesitant to enable
this by default though, to make sure we don't unnecessarily increase the
closure size of lvm2 (especially if it comes to the initrd, this could
be quite dangerous).

Tested against the tests.installer.lvm NixOS VM test even though it
shouldn't affect the result.

Signed-off-by: aszlig <[email protected]>

While it probably doesn't make sense to include lvm2 with enabled thin
provisioning to the initrd it certainly makes sense for libblockdev and
in turn for nixpart.

It remains to be seen whether we'd need to add this to a generated NixOS
configuration once someone actually wants to use it with rootfs volumes
but for now this is enough to let libblockdev's LVM tests pass.

Signed-off-by: aszlig <[email protected]>

These tests are irrelevant on Nix(OS) because we ship libblockdev with
all the binary paths directly built in (bin-paths.patch and "binPaths"
attribute), so there is no way to actually fake these utilities except
by rebuilding the package.

Apart from that the tests try to recompile libblockdev on-the-fly to
check whether plugin loading works correctly. This is also a non-issue
with Nix, because *all* plugins are always available.

So the tests.patch skips all of these tests involving these non-issues.

Signed-off-by: aszlig <[email protected]>

Once again a patch from Fedora:

http://pkgs.fedoraproject.org/cgit/rpms/parted.git/tree/0031-Use-BLKSSZGET-to-get-device-sector-size-in-_device_p.patch

Submitted upstream at:

http://lists.alioth.debian.org/pipermail/parted-devel/2016-March/004817.html

This fixes the remaining failing tests for libblockdev.

Tested by building against i686-linux and x86_64-linux.

Signed-off-by: aszlig <[email protected]>

This includes a lot of cruft that has accumulated during my previous
tries to get the tests working. After several tries (several revisions
of libblockdev and blivet) now the libblockdev tests succeed so we can
finally move over to get blivet working.

Right now the blivet subtest is commented out entirely because it will
fail to evaluate. First we need to refactor blivet and its tests and
then we need to refactor the whole NixOS test with both sutests so
they're as DRY as possible.

Signed-off-by: aszlig <[email protected]>

Upstream changes are a bit longer, see:

https://github.com/pyudev/pyudev/blob/v0.21.0/CHANGES.rst

While doCheck is implicitly true, the tests aren't actually hooked into
setup.py but need to be run via py.test.

I'm not running them on purpose because pyudev's tests assume they're on
a full-featured (GNU/)Linux system and thus the majority of the tests
fail.

Tested against Python 2.7, 3.4, 3.5 and 3.6.

Note that Python 3.3 doesn't work because the package requires the enum
module, which is only available since Python 3.4 so I've disabled
support for Python 3.3 for pyudev.

Signed-off-by: aszlig <[email protected]>

First of all, this gets rid of all the copying of the test sources by
providing them via a separate output directly from the corresponding
packages.

This also simplifies the way we retrieve environment variables needed
for running the tests. Previously we used one derivation for every
environment variable, the latter being defined in an attrset so we can
make customizations.

We no longer need these customizations, especially because libblockdev
and blivet are both rhinstaller projects and pretty much have the same
testing setup.

So now we gather these variables in one derivation and also do not fetch
LD_LIBRARY_PATH anymore, because all of the library path references are
built into the corresponding libraries.

I'm also no longer using "with pkgs.lib;" for the whole expression to
make sure we can catch eval errors very early on (my Vim config does
"nix-instantiate --parse" when writing the file).

Signed-off-by: aszlig <[email protected]>

This is basically a bump from the stone age to the current version, so
the changelog would be a bit long to summarize here, hence here is the
URL:

https://github.com/rhinstaller/blivet/blob/blivet-2.1.7/python-blivet.spec#L80-L1278

A few of the direct dependencies of blivet are now direct depencies of
libblockdev, which inself is a replacement for pyblock in C and uses
gobject-introspection to resolve the C symbols.

We also patch out hawkey and replace it with a wrapper around nix-store,
because on NixOS we can't use libsolv to resolve dependencies.

Right now, I'm also disabling the EDD tests, because I think they
shouldn't work within our VM test environments, but I need to dig a bit
more into that.

In the end we now still have 12 failing test cases which we need to
resolve:

https://headcounter.org/hydra/log/1rbpjrrjvc95ldcmmrqbangh5ssizygr-vm-test-run-blivet.drv

Signed-off-by: aszlig <[email protected]>

It's the volume_key _binary_ not the volume_key "something" :-)

Signed-off-by: aszlig <[email protected]>

The reason why it was in python-packages.nix is because we needed to
have a way to run this with different Python versions and also that the
project's API could be used by another program or library.

This is not intended so far and even if we're going to do that, we can
still move it back into python-packages.nix.

This now should make it easier to override the arguments of the package
and also should be easier to install inside a user env via "nix-env -iA
nixpart" instead of "nix-env -iA python3Packages.nixpart".

Signed-off-by: aszlig <[email protected]>

Running a specific test case is a bit icky if you need to comment out
the parts you don't want to run or wait for tests to succeed that you're
not even interested in.

This splits the subtests into its own derivations that can be simply
referenced using -A of nix-build/nix-instantiate.

Signed-off-by: aszlig <[email protected]>

I haven't yet stumbled on this because I have always set these options
within test configurations.

So this now allows to set empty disk options, which is fine (after all
we just need to reference them using the storage option in fileSystems
and swapDevices).

Signed-off-by: aszlig <[email protected]>

Using copyFileFromHost() doesn't work if the file contains single
quotes, because they're not escaped properly.

So let's move to a more robust way to provide storage.xml to the
guests (via environment.etc), because apart from that escaping issue we
really don't need anything like copyFileFromHost() anymore because every
subtest now resides in its own derivation.

Signed-off-by: aszlig <[email protected]>

I've changed this in NixOS/nixpart@7529c47.

So let's fix it here :-)

Signed-off-by: aszlig <[email protected]>

This is mainly for testing purposes during the WIP branch, but it should
actually do partitioning for some of the NixOS VM tests (particularily
the .btrfs test).

I've tagged this specifically as unstable-1.0.0 to make sure noone is
seriously going to use it yet (except for playing around of course).

Signed-off-by: aszlig <[email protected]>

The grow option now is no longer necessary, because the same effect can
be achieved by setting size to "fill". This also means that setting the
size option is now mandatory, thus it doesn't have a default value.

Instead of allowing a string for specifying size units we now use
attribute sets to do so, for example:

storage.partition.foo.size.mb = 123;

This would result into the "foo" partition being created with a size of
123 MB.

Of course it's possible to specify several units, for example:

storage.partition.foo.size = { mb = 123; kb = 456; b = 789; };

Now the type checking is also improved, so it actually shows more
information about which value is incorrectly set and why.

Signed-off-by: aszlig <[email protected]>

Use the new way to specify sizes as implemented in the previous commit.

Right now we only specify megabytes in the tests, so the tests do not
serve as very good examples on different size specifications which we
need to change soon.

Signed-off-by: aszlig <[email protected]>

BTRFS volumes are spanning over a range of physical devices and thus the
size option really doesn't apply here. Neither does it make sense to
have ordering.

In the future however we might want to allow setting a quota for a
particular volume.

Signed-off-by: aszlig <[email protected]>

Right now we have a print() at the end of the realize() function within
nixpart, which is going to print the device tree to stdout.

While I could print it to stderr instead it nevertheless make sense to
always show all the results from nixpart, regardless of whether it has
failed or not.

Signed-off-by: aszlig <[email protected]>

This was a typo I did in the first implementation and we really want to
check for the existance of a MBR on /dev/vdb instead of /dev/vdb4.

Signed-off-by: aszlig <[email protected]>

It makes more sense to default to plain bytes, so that we have the
lowest unit and also the principle of least astonishment because people
would usually assume if the didn't read the description of the option
that the amount is in bytes rather than some arbitrary value.

However, in terms of specifying sizes for partitioning, MiB or MB would
make more sense because it's highly unlikely that people want to have a
partition that's only a few bytes large.

Nevertheless having MiB vs. MB is probably also confusing because it's
clear whether people would assume the default based on 1024 units or
units of 1000.

Signed-off-by: aszlig <[email protected]>

The default size unit that's printed by the blivet device tree
representation is MiB, GiB and so on. This makes it more obvious whether
the correct size was used for partitioning without the need to convert
between mebibytes and megabytes.

Signed-off-by: aszlig <[email protected]>

This incorporates the changes revolving around the size option.

Signed-off-by: aszlig <[email protected]>

This is just the functionality of what's already documented in the
description of initlabel.

Signed-off-by: aszlig <[email protected]>

Every device specification is in the form "<type>.<name>" and so far the
type for referencing a specific device has been a plain types.str.

Now we're not only checking whether the device specification is a string
but also whether its syntax is correct and the type actually exists and
is valid for a particular option.

We now have a deviceTypes attribute set which is our main definition for
all available device specifications and it also categorizes them with
attributes like "resizable" or "orderable" which add the corresponding
options to the option set of the device specification submodule.

What's still missing are assertions on whether the actual name
references a device which actually has been defined.

Signed-off-by: aszlig <[email protected]>

I initially had these options in the storage module before actually
adding them to <nixpkgs>. Now it's time to put them back into the
storage module so that we have everything that's related to the module
in one place, so that we can do even more comprehensive type checking.

Signed-off-by: aszlig <[email protected]>

The sizes relative to the corresponding smaller units.

Signed-off-by: aszlig <[email protected]>

Whenever a device specification is cross-referenced we need to check
whether a definition for the exists. So for example if we have:

storage.mdraid.raid.devices = [ "partition.raid1" "partition.raid2" ];

We need to make sure here that storage.partition.raid1 and
storage.partition.raid2 are actually defined.

Of course we could check this within nixpart as well, but we want to
avoid such errors at run time.

Signed-off-by: aszlig <[email protected]>

This is an exception to the container types (isContainer in
deviceTypes) in that we *only* allow fsType to be "btrfs" for btrfs
subvolumes.

If this is set to something else than "btrfs", throw an assertion error
printing the conflicting options.

Signed-off-by: aszlig <[email protected]>

Currently only contains a small warning about the options being
experimental and is going to be written as we go with refactoring the
storage module.

Note that I've put it to be included *before* file-systems.xml because
the storage configuration also includes setting various fileSystems.*
options.

Signed-off-by: aszlig <[email protected]>

This is needed for the macefi and HFS tests and I'm adding solely for
the sake of completeness so that we have the biggest test coverage
possible.

Signed-off-by: aszlig <[email protected]>

So it turned out that the test failures we get with tmpfs are actually
an upstream problem because they seem to be not running tests that
require to be run as root. Here is a paste from an earlier run posted by
@vojtechtrefny:

https://paste.fedoraproject.org/518572/33694971/

The patch I'm using here is from @vojtechtrefny as well (pull request
storaged-project/blivet#532) and should not only fix the tmpfs tests but a
few other issues.

After running the test suite with this patch applied the tests are now
succeeding:

https://headcounter.org/hydra/log/ncy4wdpnhzww5yfqv9p8l9cl97dp3cac-vm-test-run-blivet.drv

Signed-off-by: aszlig <[email protected]>

We're going to break down various parts of the module into smaller
files, so that the main default.nix doesn't get cluttered up by
implementation details.

Signed-off-by: aszlig <[email protected]>

This should leave the default.nix with only option declarations and
without our custom types. The move of sizeUnits to lib.nix is currently
a bit of a workaround, but we're going to untangle that later.

Signed-off-by: aszlig <[email protected]>

This is preliminary because this needs to be properly type-constrained
and with a fallback to the device name.

Also I'm not yet sure whether we should create match.script via the
NixOS module system or within nixpart.

But before deciding on this, the current implementation at least serves
the purpose of documentation. Even though quite a bit is subject to
change the documentation on the individual matchers will still largely
apply.

However what's is going to be changed is how we're going to handle the
"match" option. Do we want it to be an additional submodule or do we
want it to be like it is now?

With the current state however it's a bit ugly because on one side we
allow multiple matchers but on the other side we don't allow the same
matcher to apply more than once.

Signed-off-by: aszlig <[email protected]>

We only had short descriptions for the options defined via deviceTypes
and thus we now also have a doc attribute that specifies a longer
description of the option.

This is needed in order for the user to know what's the default matching
method in case no particular matcher is specified.

Signed-off-by: aszlig <[email protected]>

This has the advantage of making it easier to set a default value and
also makes documentation a bit more "in place", so that we don't need to
write documentation about matches in storage.disk but in
storage.disk.*.match instead.

The default value is now match.name = diskName, where diskName is the
name defined by storage.disk.NAME.

Of course, we still have the limitation that we can't set multiple
matchers of the same type, but we can implement that easily by adding
another option "matchers" or "matches" that takes a list of matcher
submodules.

Signed-off-by: aszlig <[email protected]>

Having this as ready to be instantiated derivation is better than doing
this within nixpart, because we can make sure that all dependencies we
need for that script are in place.

Note that I'm using ${pkgs.bash}/bin/bash instead of
${pkgs.stdenv.shell}, so that it's guaranteed that the documentation is
right even if stdenv.shell should change someday.

Signed-off-by: aszlig <[email protected]>

This is to make it easier to evaluate the config with only the storage
options returned as JSON, along with all the stuff already built like
for example match.script options for disks.

If we'd only nix-instantiate the config we'd only get the .drv for the
script which we'd need to realize in another step.

In addition to that, it also makes it easier to do more extensive
validation on the storage options and we also have a way to even return
an entirely different JSON structure to what we have set in the options,
even though we don't intend to do that.

Signed-off-by: aszlig <[email protected]>

We want the evaluation to fail if one or more assertions arise in the
storage module. However, we don't want to raise unrelated assertions,
because these are clearly the job of system.build.toplevel and only are
relevant for a full system.

However, we still propagate the assertions down to config.assertions, so
that whenever a full system is built, the storage assertions get thrown
as well (if there are any).

Signed-off-by: aszlig <[email protected]>

Hawkey is normally used to get package versions and we have used
nix-store -qR to query the full dependency graph.

However, for just checking the versions for programs we have in $PATH it
should suffice to just traverse $PATH and run a regex on the package
name and version without walking the whole dependency graph.

I've used the latter only because we might have wrapped programs, but
after running the tests it turns out that we really don't need such an
overhead.

Also, it's quite a nuisance if we want to run tests in an isolated
environment where we don't have access to the store database (which
nix-store -q is trying to open).

Signed-off-by: aszlig <[email protected]>

The latest version of nixpart no longer needs XML nor do we now need to
gather the various option definitions from within the NixOS test but we
simply use config.system.build.nixpart-spec, which is the resulting JSON
file we need to pass to nixpart.

Signed-off-by: aszlig <[email protected]>

Adds support for JSON and drops XML support and thus brings nixpart in
par with our current storage module.

Signed-off-by: aszlig <[email protected]>

Having multiple matchers is a bit tricky if we don't know how they
should be combined. For example if we have a match on a label and a
device name and both produce valid matches, which one should we choose?

So let's restrict the use of device matchers to allow only one method
right now. If we later figure out a better way how to combine these
matchers, we can still lift this restriction easily.

Signed-off-by: aszlig <[email protected]>

We want to make sure that the options defined in disk.${name}.match are
working. So we define a bunch of disks with all currently available
matching methods and check afterwards if the devices get mounted.

Of course the "check afterwards" part is solely theoretical because this
is not yet supported in nixpart.

Signed-off-by: aszlig <[email protected]>

Just remove the redundant "array", because RAID already includes array
in the A.

Signed-off-by: aszlig <[email protected]>

We want to know the messages printed to stdout regardless of whether
nixpart -m has failed or not, primarily because it makes debugging
easier (just adding "print(something)" should suffice).

Signed-off-by: aszlig <[email protected]>

So far we checked whether /mnt is a valid mountpoint during invocation
of remountAndCheck. Now since we have the "matchers" sub test, we no
longer have anything mounted directly in /mnt, so it doesn't make sense
to check it unconditionally.

Signed-off-by: aszlig <[email protected]>

We're basically destroying the initial information that's relevant for
the matchers to actually match the corresponding devices, so we need
those labels to find the newly created dummy ext4 file systems again.

This also makes the definitions for fileSystems less redundant, because
we now generate it using listToAttrs and genList.

Signed-off-by: aszlig <[email protected]>

Adds support for mounting of file systems, which means that now the
"btrfs", "ext" and "matchers" storage tests are succeeding.

Signed-off-by: aszlig <[email protected]>

This fixes the following test failure on i686-linux:

https://headcounter.org/hydra/build/1562265/nixlog/13/raw

The reason we get an exec format error here is that we evaluate the
storage spec using the host system while the rest is evaluaten using the
system attribute from the test's args.

Signed-off-by: aszlig <[email protected]>

We want to have deterministic UUIDs for every device specification in
order to avoid the need to manually set labels all over the place.

Of course, we could internally set labels instead of precomputing UUIDs,
but labels have different length restrictions for every file system (for
example XFS has a maximum of 12 bytes, for ext4 it's 16 bytes). In
addition to that we remove the ability for people to set their own
labels during runtime.

The UUIDs generated here are based on version 5:

https://tools.ietf.org/html/rfc4122#section-4.1.3

Our variant deviates from this a bit in that we use string concatenation
to build up the input for the SHA1 hash instead of binaries. The results
however are pretty much the same and in our part the most important
aspect is determinism rather than having a truly unique value across the
whole planet.

Signed-off-by: aszlig <[email protected]>

Having just a single type for a device specification doesn't work out
well if we want to have an apply function, which we do want, because it
makes more sense if we want to resolve such a device specification
without using builtins.match all over the place.

It also improves a lot in readability of the option descriptions,
because every such option now has not only a description of what a
device specification is but also lists the valid types for the device
specification.

This has another advantage that instead for something like the
following:

Type: list of device specification of <type>.<name>s

The type description is now just:

Type: list of device specifications

We're also heading for more consistency, speaking about "device
specification" or shortly "devspec". Say if we have something like
"storage.foo.bar", "foo.bar" is the "device specification" and "foo" is
the "device specification type" and "bar" is the "device specification
name".

Signed-off-by: aszlig <[email protected]>

So far we passed the device specification as-is to nixpart, but from
within the module system it's quite tricky to validate or look up such a
string, because we need to parse it every time we need to do a look up
an a configuration value in "storage.*".

Now a device specification is an attribute set consisting of a `name'
and a `type' attribute. We also have a new applyTypeContainer attribute
we need to pass to mkDeviceSpecOption so that we can properly convert
things such as "listOf devspecType" into a list of valid internal
representations of device specifications.

Signed-off-by: aszlig <[email protected]>

This is handy if we want to look up configuration options for a specific
device specification, so with only the internal representation of a
devspec we can simply say devspec.uuid to get a generated UUID.

Signed-off-by: aszlig <[email protected]>

This implements the deterministically generated UUIDs to be used while
mounting file systems, but only if there is no label set already. So the
user still has a way to set labels (which are also applied by nixpart)
and use them accordingly, even though the UUIDs should be more
distincive.

Signed-off-by: aszlig <[email protected]>

While it may be handy to put the whole configuration of the
corresponding device specification into the values of the options
referring to them, this unfortunately blows up the size of the JSON
output we pass to nixpart.

This is unnecessary because we're only interested in the UUID.

Signed-off-by: aszlig <[email protected]>

I'm heading for a hybrid approach (using UUIDs and partition layout
holes) in nixpart for achieving storage tree determinism, so we need to
have support for setting UUIDs. Blivet currently doesn't yet support
this, so I've implemented it.

Upstream pull request:

storaged-project/blivet#537

Signed-off-by: aszlig <[email protected]>

These tools are needed in order to run tests for NTFS and for setting
the serial of a FAT file system after it has been created.

Signed-off-by: aszlig <[email protected]>

Even though the ntfs3g utilities are available inside our test
environment, the format didn't get advertised as formattable because the
_formattable attribute wasn't set to True.

Submitted upstream at:

storaged-project/blivet#536

Signed-off-by: aszlig <[email protected]>

Switches to using a dictionary for devspecs and supports setting UUIDs
for every device specification (currently only sets RFC4122-style
UUIDs).

Signed-off-by: aszlig <[email protected]>

We no longer need to use labels now that every file system gets assigned
an UUID in latest nixpart.

Signed-off-by: aszlig <[email protected]>

We already had stubs for some pyanaconda imports so far, but some
functionality like enable_installer_mode() inherently depends on it, so
let's remove enable_installer_mode().

Another occurence of pyanaconda import is in storage_initialize():

  from pyanaconda.flags import flags as anaconda_flags
  flags.update_from_anaconda_flags(anaconda_flags)

This is an installer-specific function which should also be quite tied
to pyanaconda, but instead of removing this function altogether, we just
remove the import, because it only appends certain flags from the
pyanaconda module.

Signed-off-by: aszlig <[email protected]>

This just contains one additional commit which fixes various pylint
errors.

Signed-off-by: aszlig <[email protected]>

So far we had disabled the tests while referring to the NixOS VM test
instead. However, it's desirable to run as much tests as we can, so
let's run the pocketlint tests in checkPhase instead of skipping it
altogether.

In my case this is very useful because it would have caught a few errors
during development of the UUIDs pull request:

storaged-project/blivet#537

But even if we're not directly developing for the upstream project, this
also catches Nix-related errors, such as references against pyanaconda
which might still exist or exist again after an update.

I'm using a list to accumulate find arguments because I wanted to avoid
endless repetitions of -o -path xyz -prune.

Signed-off-by: aszlig <[email protected]>

This is also part of blivet's "make check", so I've included it for
completeness and almost verbatim because blivet does not comply to a few
points in PEP8, like when it comes to line length.

Signed-off-by: aszlig <[email protected]>

This version largely differs from the previous version in that we now
set the UUID via the "uuid" keyword argument rather than introducing a
new "new_uuid" kwarg.

We now need to reorder the uuids.patch and the ntfs-formattable.patch,
because the latter got merged into the upstream 2.1-devel branch and the
uuids.patch has been rebased against the newest HEAD of 2.1-devel.

Signed-off-by: aszlig <[email protected]>

This only renames the "new_uuid" keyword argument to be just "uuid", as
changed in the previous nixpkgs commit on blivet.

Signed-off-by: aszlig <[email protected]>