Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add: extension env-injector #4925

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
1689d29
add: extension env-injector
locnnil Jul 17, 2024
c439436
test: add spread tests for env-injector extension
locnnil Jul 23, 2024
774f067
feat(extensions): pass app name to `get_app_snippet()`
mr-cal Jul 17, 2024
04dd07e
refact: address review comments
locnnil Jul 24, 2024
a6374ac
feat: address reviews, add more edge cases to test
locnnil Aug 2, 2024
0a280d9
many: Address reviews
locnnil Aug 2, 2024
2d276b8
feat: improve sentence.
locnnil Aug 2, 2024
d0629a7
fix: typo error
locnnil Aug 2, 2024
8bd2300
fix: improve sentence
locnnil Aug 2, 2024
d8f9e68
fix: commentaries not being discarted in envfiles
locnnil Aug 4, 2024
3e17cec
feat: add logs to check test execution
locnnil Aug 5, 2024
b65a0c6
fix: add missing configure hook
locnnil Aug 8, 2024
6ab0b23
Update tests/spread/extensions/env-injector/task.yaml
locnnil Aug 8, 2024
09a883a
feat: improve tests, drop C code
locnnil Aug 12, 2024
84ce910
fix: shellchek error
locnnil Aug 12, 2024
984dd5b
fix(ci): fix spread test
locnnil Aug 12, 2024
5406f55
fix: outdated test
locnnil Aug 13, 2024
9466377
fix: use absolute path for envfiles
locnnil Aug 13, 2024
bd60e48
feat: change test snap to classic confinement
locnnil Aug 13, 2024
03a188d
fix: (try to) fix envfile path location problem
locnnil Aug 14, 2024
0d09a15
test|refact: Add unit tests and rename class
locnnil Aug 14, 2024
18d6c5c
refact: formating python code
locnnil Aug 14, 2024
7bc6f24
test: add parts to dedicated class
locnnil Aug 14, 2024
0c57814
fix: address lint errors
locnnil Aug 14, 2024
cb72589
fix: missing fields
locnnil Aug 14, 2024
297de3a
fix: PEP8 formating
locnnil Aug 14, 2024
6b1dafe
refact: formating code
locnnil Aug 14, 2024
498ad20
fix: solving last(I hope) TOX/linter complaining
locnnil Aug 14, 2024
5e15598
cleanup: remove comment
locnnil Aug 15, 2024
c20cac8
many: reviews addressed
locnnil Aug 16, 2024
e30ed89
refact: using common directory
locnnil Aug 16, 2024
645fb35
fix: missing filename
locnnil Aug 17, 2024
d839fe2
tests: add tests to toolchain fetch method
locnnil Aug 17, 2024
76e51e6
fix: don't copy compressed bin to snap
locnnil Sep 3, 2024
ecbde85
refact: move exporter program to separate repo
locnnil Sep 3, 2024
48378ca
fix: lint error
locnnil Sep 3, 2024
0ce514a
fix: lint include error
locnnil Sep 4, 2024
a97df2d
feat|fix: set tagged version of snappy-env and fix unit test
locnnil Sep 26, 2024
b5f6370
test: check error value for toolchain selection
locnnil Sep 26, 2024
fb432b8
refact: formating
locnnil Sep 26, 2024
01c5c25
fix: revert change to spread
locnnil Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions schema/snapcraft.json
Original file line number Diff line number Diff line change
Expand Up @@ -906,6 +906,7 @@
"uniqueItems": true,
"items": {
"enum": [
"env-injector",
"flutter-stable",
"flutter-beta",
"flutter-dev",
Expand Down
4 changes: 2 additions & 2 deletions snapcraft/extensions/_ros2_humble_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ def get_root_snippet(self) -> Dict[str, Any]:
return root_snippet

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
app_snippet = super().get_app_snippet()
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
app_snippet = super().get_app_snippet(app_name=app_name)
python_paths = app_snippet["environment"]["PYTHONPATH"]
new_python_paths = [
f"$SNAP/opt/ros/underlay_ws/opt/ros/{self.ROS_DISTRO}/lib/python3.10/site-packages",
Expand Down
4 changes: 2 additions & 2 deletions snapcraft/extensions/_ros2_jazzy_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ def get_root_snippet(self) -> Dict[str, Any]:
return root_snippet

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
app_snippet = super().get_app_snippet()
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
app_snippet = super().get_app_snippet(app_name=app_name)
python_paths = app_snippet["environment"]["PYTHONPATH"]
new_python_paths = [
f"$SNAP/opt/ros/underlay_ws/opt/ros/{self.ROS_DISTRO}/lib/python3.12/site-packages",
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ def _apply_extension(
)

# Apply the app-specific components of the extension (if any)
app_extension = extension.get_app_snippet()
for app_name in app_names:
app_extension = extension.get_app_snippet(app_name=app_name)
app_definition = yaml_data["apps"][app_name]
for property_name, property_value in app_extension.items():
app_definition[property_name] = _apply_extension_property(
Expand Down
129 changes: 129 additions & 0 deletions snapcraft/extensions/env_injector.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
#
# Copyright 2024 Canonical Ltd.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 3 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

"""Extension to automatically set environment variables on snaps."""

from typing import Any, Dict, Optional, Tuple

from overrides import overrides

from .extension import Extension


class EnvInjector(Extension):
"""Extension to automatically set environment variables on snaps.

This extension allows you to transform snap options into environment
variables

It configures your application to run a command-chain that transforms the
snap options into environment variables automatically.

- To set global environment variables for all applications **inside** the snap:

.. code-block:: shell
sudo snap set <snap-name> env.<key>=<value>

- To set environment variables for a specific application **inside** the snap:

.. code-block:: shell
sudo snap set <snap-name> apps.<app-name>.env.<key>=<value>

- To set environment file inside the snap:

.. code-block:: shell
sudo snap set <snap-name> env-file=<path-to-env-file>
locnnil marked this conversation as resolved.
Show resolved Hide resolved
- To set environment file for a specific app:

.. code-block:: shell
sudo snap set <snap-name> apps.<app-name>.envfile=<path-to-env-file>

"""

@staticmethod
@overrides
def get_supported_bases() -> Tuple[str, ...]:
return ("core24",)

@staticmethod
@overrides
def get_supported_confinement() -> Tuple[str, ...]:
return ("strict", "devmode", "classic")

@staticmethod
@overrides
def is_experimental(base: Optional[str]) -> bool:
return True

@overrides
def get_root_snippet(self) -> Dict[str, Any]:
return {}

@overrides
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
"""Return the app snippet to apply."""
return {
"command-chain": ["bin/command-chain/env-exporter"],
"environment": {
"env_alias": f"{app_name}",
},
Comment on lines +79 to +82
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very simple snippet and snap authors are quite used to apply even more complex patterns to parts or applications in order to implement certain features, so this alone may not warrant the existence of an extension.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The selected snippet adds the env exporter program and the alias. The alias gets added only when the user hasn't set it (this is intended and based on snapcraft's internal handling of the dictionary).

The extension does more than that: It adds the source code for the exporter program, builds it and places it in the right location. The design also envisions additional functions, such as adding a configure hook when it isn't a present, or adding configuration validation.

If this was to be done manually, without the extension, the developers would have to:

  1. Fetch the exporter source from trusted location, pick and appropriate version, and hope that it stays compatible with the snap. *
  2. Add build logic to compile and place the executable in a suitable location *
  3. For each application that needs the functionality:
    i. Add the exporter program to the command chain
    ii. Set the application alias

In comparison, when the extension is used:

  1. Add the extension to each application that needs the functionality
  2. Add alias if an app's name needs to be overridden

The manual way requires average snapping knowledge, which early adopters lack. I'd emphasize that the goal is to simplify the snapping of apps and all this complexity thrown at the developer defeats its purpose.


* We could make a snap or a debian package and let the developer stage it, but then we need to separately maintain that package.

}

@overrides
def get_part_snippet(self, *, plugin_name: str) -> Dict[str, Any]:
return {}

@overrides
def get_parts_snippet(self) -> Dict[str, Any]:
toolchain = self.get_toolchain(self.arch)
if toolchain is None:
raise ValueError(
f"Unsupported architecture for env-injector extension: {self.arch}"
)

return {
"env-injector/env-injector": {
"source": "https://github.com/canonical/snappy-env.git",
locnnil marked this conversation as resolved.
Show resolved Hide resolved
"source-tag": "v1.0.0-beta",
"plugin": "nil",
lengau marked this conversation as resolved.
Show resolved Hide resolved
"build-snaps": ["rustup"],
"build-packages": ["upx-ucl"], # for binary compression
"override-build": f"""
rustup default stable
rustup target add {toolchain}

cargo build --target {toolchain} --release
mkdir -p $SNAPCRAFT_PART_INSTALL/bin/command-chain

# compress the binary
upx --best --lzma target/{toolchain}/release/env-exporter

cp target/{toolchain}/release/env-exporter $SNAPCRAFT_PART_INSTALL/bin/command-chain
""",
}
}

def get_toolchain(self, arch: str):
"""Get the Rust toolchain for the current architecture."""
toolchain = {
"amd64": "x86_64-unknown-linux-gnu",
"arm64": "aarch64-unknown-linux-gnu",
# 'armhf': 'armv8-unknown-linux-gnueabihf', # Tier 2 toolchain
# 'riscv64': 'riscv64gc-unknown-linux-gnu', # Tier 2 toolchain
# 'ppc64el': 'powerpc64-unknown-linux-gnu', # Tier 2 toolchain
# 's390x': 's390x-unknown-linux-gnu', # Tier 2 toolchain
}
return toolchain.get(arch)
7 changes: 5 additions & 2 deletions snapcraft/extensions/extension.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,11 @@ def get_root_snippet(self) -> Dict[str, Any]:
"""Return the root snippet to apply."""

@abc.abstractmethod
def get_app_snippet(self) -> Dict[str, Any]:
"""Return the app snippet to apply."""
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
"""Return the app snippet to apply.

:param app_name: the name of the app where the snippet will be applied
"""

@abc.abstractmethod
def get_part_snippet(self, *, plugin_name: str) -> Dict[str, Any]:
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/gnome.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def is_experimental(base: Optional[str]) -> bool:
return False

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
command_chain = ["snap/command-chain/desktop-launch"]
if self.yaml_data["base"] == "core24":
command_chain.insert(0, "snap/command-chain/gpu-2404-wrapper")
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/kde_neon.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def is_experimental(base: Optional[str]) -> bool:
return False

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
return {
"command-chain": ["snap/command-chain/desktop-launch"],
"plugs": ["desktop", "desktop-legacy", "opengl", "wayland", "x11"],
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/kde_neon_6.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def is_experimental(base: Optional[str]) -> bool:
return False

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
return {
"command-chain": ["snap/command-chain/desktop-launch6"],
"plugs": [
Expand Down
2 changes: 2 additions & 0 deletions snapcraft/extensions/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

from snapcraft import errors

from .env_injector import EnvInjector
from .gnome import GNOME
from .kde_neon import KDENeon
from .kde_neon_6 import KDENeon6
Expand All @@ -38,6 +39,7 @@
ExtensionType = Type[Extension]

_EXTENSIONS: Dict[str, "ExtensionType"] = {
"env-injector": EnvInjector,
"gnome": GNOME,
"ros2-humble": ROS2HumbleExtension,
"ros2-humble-ros-core": ROS2HumbleRosCoreExtension,
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/ros2_humble.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def get_root_snippet(self) -> Dict[str, Any]:
}

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
python_paths = [
f"$SNAP/opt/ros/{self.ROS_DISTRO}/lib/python3.10/site-packages",
"$SNAP/usr/lib/python3/dist-packages",
Expand Down
2 changes: 1 addition & 1 deletion snapcraft/extensions/ros2_jazzy.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def get_root_snippet(self) -> Dict[str, Any]:
}

@overrides
def get_app_snippet(self) -> Dict[str, Any]:
def get_app_snippet(self, *, app_name: str) -> Dict[str, Any]:
python_paths = [
f"$SNAP/opt/ros/{self.ROS_DISTRO}/lib/python3.12/site-packages",
"$SNAP/usr/lib/python3/dist-packages",
Expand Down
127 changes: 127 additions & 0 deletions tests/spread/extensions/env-injector/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
summary: Build and run a basic hello-world snap using extensions

systems:
- ubuntu-24.04*

environment:

SNAPCRAFT_ENABLE_EXPERIMENTAL_EXTENSIONS: "1"
SNAP_DIR: ../snaps/env-injector-hello
SNAP: env-injector-hello

prepare: |

#shellcheck source=tests/spread/tools/snapcraft-yaml.sh
. "$TOOLS_DIR/snapcraft-yaml.sh"
set_base "$SNAP_DIR/snap/snapcraft.yaml"

restore: |

cd "$SNAP_DIR"
snapcraft clean
rm -f /var/snap/"${SNAP}"/common/*.env
rm -f ./*.snap
rm -rf ./squashfs-root
#shellcheck source=tests/spread/tools/snapcraft-yaml.sh
. "$TOOLS_DIR/snapcraft-yaml.sh"
restore_yaml "snap/snapcraft.yaml"

execute: |

assert_env() {
local snap_app="$1"
local env_name="$2"
local exp_value="$3"

local actual_value

if ! eval "$snap_app" | grep -q "^${env_name}="; then
echo "Environment variable '$env_name' is not set."
return 1
fi

if [ -z "$env_name" ]; then
empty=$( "$snap_app" | grep "=${exp_value}")
[ -z "$empty" ] || return 1
fi

actual_value=$( "$snap_app" | grep "^${env_name}=" | cut -d'=' -f2-)

if [ "$actual_value" != "$exp_value" ]; then
echo "Environment variable '$env_name' does not match the expected value."
echo "Expected: '$env_name=$exp_value', but got: '$env_name=$actual_value'"
return 1
fi

return 0
}

cd "$SNAP_DIR"
SNAPCRAFT_ENABLE_EXPERIMENTAL_EXTENSIONS=1 snapcraft

unsquashfs "${SNAP}"_1.0_*.snap

# Check that the env-exporter program is present
[ -f squashfs-root/bin/command-chain/env-exporter ]

# Check that the exec-env script is present
[ -f squashfs-root/usr/bin/exec-env ]

snap install "${SNAP}"_1.0_*.snap --dangerous

echo "[env-injector] Creating global envfile"
echo 'HELLO_WORLD="Hello World"' >> /var/snap/"${SNAP}"/common/global.env

# Load global envfile
snap set env-injector-hello envfile=/var/snap/"${SNAP}"/common/global.env
echo "[TEST] - Check if the global envfile is loaded for all apps"
assert_env "env-injector-hello.hello1" "HELLO_WORLD" "Hello World" || exit 1
assert_env "env-injector-hello.hello2" "HELLO_WORLD" "Hello World" || exit 1
assert_env "env-injector-hello.hello-demo" "HELLO_WORLD" "Hello World" || exit 1

echo "[env-injector] Creating app-specific envfile"
echo 'SCOPED=Scoped' >> /var/snap/"${SNAP}"/common/appenv.env

# Load app-specific envfile
snap set env-injector-hello apps.hello1.envfile=/var/snap/"${SNAP}"/common/appenv.env
echo "[TEST] - Check if the app-specific envfile is loaded for the app"
assert_env "env-injector-hello.hello1" "SCOPED" "Scoped" || exit 1

echo "[env-injector] Setting global env variable"
# Set env vars: Global
snap set env-injector-hello env.global="World"
locnnil marked this conversation as resolved.
Show resolved Hide resolved
echo "[TEST] - Check if the global env var is set for all apps"
assert_env "env-injector-hello.hello1" "GLOBAL" "World" || exit 1
assert_env "env-injector-hello.hello2" "GLOBAL" "World" || exit 1
assert_env "env-injector-hello.hello-demo" "GLOBAL" "World" || exit 1

echo "[env-injector] Setting app-specific env variable"
# Set env vars: specific to each app
snap set env-injector-hello apps.hello1.env.hello="Hello"
snap set env-injector-hello apps.hello2.env.specific="City"

echo "[TEST] - Check if the app-specific env var IS SET for the app hello1"
assert_env "env-injector-hello.hello1" "HELLO" "Hello" || exit 1
echo "[TEST] - Check if the app-specific env var IS NOT SET for the app hello2"
! assert_env "env-injector-hello.hello2" "HELLO" "Hello" || exit 1

echo "[TEST] - Check if the app-specific env var IS SET for the app hello2"
assert_env "env-injector-hello.hello2" "SPECIFIC" "City" || exit 1
echo "[TEST] - Check if the app-specific env var IS NOT SET for the app hello1"
! assert_env "env-injector-hello.hello1" "SPECIFIC" "City" || exit 1

snap set env-injector-hello env.word.dot="wrong"
echo "[TEST] - Check if the key with dot was ignored"
! assert_env "env-injector-hello.hello1" "" "wrong" || exit 1

echo "[env-injector] Testing order of env vars"
echo 'ORDER="From envfile"' >> /var/snap/"${SNAP}"/common/local.env
snap set env-injector-hello apps.hello1.env.order="from app-specific"
snap set env-injector-hello apps.hello1.envfile=/var/snap/"${SNAP}"/common/local.env
echo "[TEST] - Check if local overrites global"
assert_env "env-injector-hello.hello1" "ORDER" "from app-specific" || exit 1

echo "[env-injector] Run hello-demo app"
snap set env-injector-hello apps.myapp.env.specific="City"
echo "[TEST] Make sure that alias is NOT rewritten"
assert_env "env-injector-hello.hello-demo" "SPECIFIC" "City" || exit 1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#!/bin/bash
Loading
Loading