Skip to content

Commit

Permalink
Add support for Hyper-V platform
Browse files Browse the repository at this point in the history
Read the Ignition config from the Hyper-V Data Exchange Service ("KVP"),
which can be configured on the host via WMI.

KVP on Linux is normally handled by a combination of a kernel module
(hv_utils) and a daemon shipped with the kernel source (hv_kvp_daemon).
The latter writes world-readable files to /var/lib/hyperv; these contain
an array of binary structs.  The host-guest protocol has the guest OS as
the passive peer; hv_kvp_daemon connects and is sent all the host's
key-value pairs.

We don't want to require hv_kvp_daemon as a dependency, so we use a
pure Go reimplementation (libhvee) that supports just enough of the
protocol to receive the KVPs.  However, if we disconnect and then
hv_kvp_daemon reconnects later, the host won't re-send the KVPs, so we
also need to save them to /var/lib/hyperv in the same format used by
the daemon.  Unlike the daemon, we set the files to mode 600, since the
Ignition config is potentially sensitive; the daemon does not change
the permissions of existing files.

Small configs can be stored in the `ignition.config` KVP.  However,
individual values have severe length limitations (~1 KiB of UTF-8), so
we also support concatenating `ignition.config.0`, `ignition.config.1`,
etc. to form a larger config.

See also:
coreos/fedora-coreos-tracker#1411
coreos/fedora-coreos-tracker#1424

Almost all of this work was done by Brent Baude; I just added the final
polish.  Thanks Brent!

Co-authored-by: Brent Baude <[email protected]>
  • Loading branch information
bgilbert and baude committed Jun 28, 2023
1 parent 153df40 commit 005f5b8
Show file tree
Hide file tree
Showing 11 changed files with 616 additions and 0 deletions.
1 change: 1 addition & 0 deletions docs/release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Starting with this release, ignition-validate binaries are signed with the

### Features

- Support Hyper-V platform

### Changes

Expand Down
2 changes: 2 additions & 0 deletions docs/supported-platforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Ignition is currently only supported for the following platforms:
* [DigitalOcean] (`digitalocean`) - Ignition will read its configuration from the droplet userdata. Cloud SSH keys and network configuration are handled separately.
* [Exoscale] (`exoscale`) - Ignition will read its configuration from the instance userdata. Cloud SSH keys are handled separately.
* [Google Cloud] (`gcp`) - Ignition will read its configuration from the instance metadata entry named "user-data". Cloud SSH keys are handled separately.
* [Microsoft Hyper-V] (`hyperv`) - Ignition will read its configuration from the `ignition.config` key in pool 0 of the Hyper-V Data Exchange Service (KVP). Values are limited to approximately 1 KiB of text, so Ignition can also read and concatenate multiple keys named `ignition.config.0`, `ignition.config.1`, and so on.
* [IBM Cloud] (`ibmcloud`) - Ignition will read its configuration from the instance userdata. Cloud SSH keys are handled separately.
* [KubeVirt] (`kubevirt`) - Ignition will read its configuration from the instance userdata via config drive. Cloud SSH keys are handled separately.
* Bare Metal (`metal`) - Use the `ignition.config.url` kernel parameter to provide a URL to the configuration. The URL can use the `http://`, `https://`, `tftp://`, `s3://`, `arn:`, or `gs://` schemes to specify a remote config.
Expand All @@ -41,6 +42,7 @@ For most cloud providers, cloud SSH keys and custom network configuration are ha
[DigitalOcean]: https://www.digitalocean.com/products/droplets/
[Exoscale]: https://www.exoscale.com/compute/
[Google Cloud]: https://cloud.google.com/compute
[Microsoft Hyper-V]: https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/
[IBM Cloud]: https://www.ibm.com/cloud/vpc
[KubeVirt]: https://kubevirt.io
[Nutanix]: https://www.nutanix.com/products/ahv
Expand Down
5 changes: 5 additions & 0 deletions dracut/30ignition/module-setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,3 +96,8 @@ install() {
# needed for openstack config drive support
inst_rules 60-cdrom_id.rules
}

installkernel() {
# required by hyperv platform to read kvp from the kernel
instmods -c hv_utils
}
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ require (
cloud.google.com/go/storage v1.30.1
github.com/aws/aws-sdk-go v1.44.284
github.com/beevik/etree v1.2.0
github.com/containers/libhvee v0.0.5
github.com/coreos/go-semver v0.3.1
github.com/coreos/go-systemd/v22 v22.5.0
github.com/coreos/vcontext v0.0.0-20230201181013-d72178a18687
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XP
github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/containers/libhvee v0.0.5 h1:5tUiF2eVe8XbVSPD/Os4dIU1gJWoQgtkQHIjQ5X7wpE=
github.com/containers/libhvee v0.0.5/go.mod h1:AYsyMe44w9ylWWEZNW+IOzA7oZ2i/P9TChNljavhYMI=
github.com/coreos/go-json v0.0.0-20230131223807-18775e0fb4fb h1:rmqyI19j3Z/74bIRhuC59RB442rXUazKNueVpfJPxg4=
github.com/coreos/go-json v0.0.0-20230131223807-18775e0fb4fb/go.mod h1:rcFZM3uxVvdyNmsAV2jopgPD1cs5SPWJWU5dOz2LUnw=
github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4=
Expand Down
106 changes: 106 additions & 0 deletions internal/providers/hyperv/kvp.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
// Copyright 2023 Red Hat
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package hyperv

import (
"fmt"
"os/exec"
"path/filepath"

"github.com/containers/libhvee/pkg/kvp"
"github.com/coreos/ignition/v2/config/shared/errors"
"github.com/coreos/ignition/v2/config/v3_5_experimental/types"
"github.com/coreos/ignition/v2/internal/distro"
"github.com/coreos/ignition/v2/internal/platform"
"github.com/coreos/ignition/v2/internal/providers/util"
"github.com/coreos/ignition/v2/internal/resource"
"github.com/coreos/vcontext/report"
)

const singleKey = "ignition.config"

// Prefix for multiple config fragments to reassemble. The suffix is a
// sequential integer starting from 0.
const splitKeyPrefix = "ignition.config."

func init() {
platform.Register(platform.Provider{
Name: "hyperv",
FetchWithFiles: fetchConfig,
})
}

func fetchConfig(f *resource.Fetcher) ([]types.File, types.Config, report.Report, error) {
var kvpFiles []types.File

// To read key-value pairs from the Windows host, the hv_util kernel
// module must be loaded to create the kernel device
_, err := f.Logger.LogCmd(exec.Command(distro.ModprobeCmd(), "hv_utils"), "loading hv_utils kernel module")
if err != nil {
return nil, types.Config{}, report.Report{}, fmt.Errorf("loading hv_utils kernel module: %w", err)
}

keyValuePairs, err := kvp.GetKeyValuePairs()
if err != nil {
return nil, types.Config{}, report.Report{}, fmt.Errorf("reading key-value pairs: %w", err)
}

var ign string
for _, kv := range keyValuePairs[kvp.DefaultKVPPoolID] {
if kv.Key == singleKey {
f.Logger.Debug("found single KVP key")
ign = kv.Value
break
}
}

if ign == "" {
ign, _, err = keyValuePairs.GetSplitKeyValues(splitKeyPrefix, kvp.DefaultKVPPoolID)
if err == nil {
f.Logger.Debug("found concatenated KVP keys")
} else if err != kvp.ErrNoKeyValuePairsFound {
return nil, types.Config{}, report.Report{}, fmt.Errorf("reassembling split config: %w", err)
}
}

// hv_kvp_daemon writes pools to the filesystem in /var/lib/hyperv.
// We've already read the pool data, and the host won't send it again
// on this boot, so we need to write the files ourselves.
for poolID := range keyValuePairs {
// hv_kvp_daemon writes the pool files with mode 644 in a
// directory with mode 755. This isn't safe for us, since
// it leaks the config to non-root users, including on
// subsequent boots.
// - There's no API that lets us delete the KVPs from the host.
// - We could filter out the KVPs when writing the pools,
// but if hv_kvp_daemon runs on subsequent boots, it could
// re-add them.
// - The caller doesn't give us a way to create directory
// entries, only files; and we probably shouldn't set
// restrictive permissions on /var/lib/hyperv because it
// hypothetically might be used for other purposes.
// Avoid the issue by setting the files to mode 600.
// hv_kvp_daemon won't change the mode afterward.
poolPath := filepath.Join(kvp.DefaultKVPFilePath, fmt.Sprintf("%s%d", kvp.DefaultKVPBaseName, poolID))
kvpFiles = append(kvpFiles, util.MakeProviderOutputFile(poolPath, 0600, keyValuePairs.EncodePoolFile(poolID)))
}

if ign == "" {
return kvpFiles, types.Config{}, report.Report{}, errors.ErrEmpty
}

c, r, err := util.ParseConfig(f.Logger, []byte(ign))
return kvpFiles, c, r, err
}
1 change: 1 addition & 0 deletions internal/register/providers.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
_ "github.com/coreos/ignition/v2/internal/providers/exoscale"
_ "github.com/coreos/ignition/v2/internal/providers/file"
_ "github.com/coreos/ignition/v2/internal/providers/gcp"
_ "github.com/coreos/ignition/v2/internal/providers/hyperv"
_ "github.com/coreos/ignition/v2/internal/providers/ibmcloud"
_ "github.com/coreos/ignition/v2/internal/providers/kubevirt"
_ "github.com/coreos/ignition/v2/internal/providers/metal"
Expand Down
201 changes: 201 additions & 0 deletions vendor/github.com/containers/libhvee/LICENSE

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 005f5b8

Please sign in to comment.