fuse-overlayfs: add C plugins system

[giuseppe]

Alternative for #79

Add a simple plugin mechanism that will help to expand fuse-overlayfs
functionalities, in particular it allows to load data from a layer on
demand.

A plugin is loaded into fuse-overlayfs using the option:

-o plugins=path/to/plugin.so:path/to/another/plugin.so

A layer can use a plugin with the syntax:

-o lowerdir=//plugin-name/DATA-FOR-THE-PLUGIN/path

Each time a file/directory is looked up, if a plugin is registered for
a layer, the plugin is first notified about the request.

After the callback is invoked, fuse-overlayfs still expects the data
to be accessible at the specified directory.

Signed-off-by: Giuseppe Scrivano [email protected]

[lgtm-com]

This pull request introduces 3 alerts when merging dd49509 into 74fb3dd - view on LGTM.com

new alerts:

• 3 for Comparison result is always the same

[giuseppe]

a test plugin: https://gist.github.com/giuseppe/a669ed7248de557a9b5fd272ffe2a4f4

It allows to use a shared storage owned by another user as a lower layer.

as root:

# podman create --name foo fedora ls
# cp -r $(podman mount -l) rootfs
# podman umount -l
# podman rm -l
# ./convert rootfs

The last command convert each file to mode 0755, the original permission is stored in an extended attribute.

As rootless then we can do:

$ podman unshare
# mkdir upper workdir merged
# ./fuse-overlayfs -o plugins=$(pwd)/test-plugin.so -o lowerdir=//test//rootfs/,upperdir=upper,workdir=workdir merged
# podman run --rm -ti --rootfs $(pwd)/merged bash

The new syntax for lowerdir starting with // says to fuse-overlayfs to use a plugin to handle that layer.

The original files from the image are owned by root, the fuse-overlayfs plugin makes sure the original permissions are used inside of the container.

A second rootless user could use the same command and re-use the lower layer from the root storage.

[giuseppe]

@vrothberg @rhatdan ⬆️ this is the idea I had in mind for sharing the storage for rootless users.

[rhatdan]

This would only allow us to copy one image. How would I set this up to allow the use of 100 images?
Does convert switch all file protections to 755? Even ones that are looser?

[giuseppe]

@rhatdan yes, it must switch all protections to 0755, also ones that are looser. No file must be world writeable as it can affect other users of the image.

root, or more in general the owner of a storage, must explicitly convert or copy+clone (if it is still needed with native overlay) each layer that is going to be shared. If the underlying file system support reflinks though, there is only the additional cost of creating inodes.

[rhatdan]

We might be able to take advantage of Overlay MetaCopy=on as well.

[rhatdan]

Could container storage do this for us, and just take a group of images or all images and mount them up to be used by fuse-overlay?

[giuseppe]

Could container storage do this for us, and just take a group of images or all images and mount them up to be used by fuse-overlay?

yes once the low level bits are working, I think we'll need to move the logic of creating such images into containers/storage so that we could have something: podman image make-shared $IMAGE

[vrothberg]

Currently under water with tackling bugs. Will have a look as soon as I find time. Thanks for the ping, @giuseppe !

[giuseppe]

ready for review

[giuseppe]

a plugin for CRFS: https://github.com/giuseppe/crfs-plugin

[giuseppe]

@rhatdan let's merge if there are no problems and I will cut a new release

[rhatdan]

LGTM