Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Hyper-V support #1424

Closed
26 tasks done
baude opened this issue Feb 17, 2023 · 7 comments
Closed
26 tasks done

Implement Hyper-V support #1424

baude opened this issue Feb 17, 2023 · 7 comments

Comments

@baude
Copy link
Contributor

baude commented Feb 17, 2023

Implementing a new supported platform

During Development

Create PR's addressing the following:

At Release

  1. Merge metadata changes:

    • stream-metadata-go
    • fedora-coreos-tracker
    • fedora website
    • fedora-coreos-browser
  2. Create and push signed tags with appropriate versions

    # Ensure gpg key for signing in github settings that is associated to redhat email.
    # Verify you are on the upstream repo's main branch.
    
    git status
    
    RELEASE_VER=vx.y.z
    # Replace 'x.y.z' with the appropriate numbers.
    
    git tag -s ${RELEASE_VER}
    # Give appropriate detail to tag, check previous tags with 'git show ${RELEASE_VER}'
    
    git push [email protected]:coreos/targeted-repo.git ${RELEASE_VER}
    # Navigate to the targeted-repo's tag section to ensure a valid signed tag is listed.
    # e.g. https://github.com/...repo/tags 
    
    1. Tag stream-metadata-go following the above steps. After tagging, ensure that dependabot has picked up latest version, and merged it into fedora-coreos-stream-generator && coreos-assembler.

    2. Tag fedora-coreos-stream-generator following the above steps.

  3. Merge the following changes:

    • coreos-assembler
  4. Wait for updates made to coreos-assembler to be propagated to latest container

    • Download latest version of coreos-assembler container. Verify platform support functionality.
  5. Merge changes for:

    • Build pipeline
  6. Wait for new images to reach stable then merge documentation.

    • fedora-coreos-docs merged
@baude baude mentioned this issue Feb 17, 2023
29 tasks
baude added a commit to baude/stream-metadata-go that referenced this issue Feb 17, 2023
Following new platform support checklist for Hyperv

coreos/fedora-coreos-tracker#1424
baude added a commit to baude/fedora-coreos-browser that referenced this issue Feb 17, 2023
baude added a commit to baude/fedora-coreos-pipeline that referenced this issue Feb 17, 2023
baude added a commit to baude/coreos-assembler that referenced this issue Feb 17, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
@baude baude changed the title Implement HyperV support #1421 Implement HyperV support Feb 17, 2023
@bgilbert
Copy link
Contributor

bgilbert commented Feb 17, 2023

Updates for the checklist template:

  • Add steps for adding Ignition and Afterburn support
  • Replace link to pipeline example PR
  • Add platforms.yaml update

Applied in #1439.

baude added a commit to baude/ignition that referenced this issue Feb 17, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/fedora-coreos-tracker that referenced this issue Feb 17, 2023
Add example metadata for new platform support of hyperv. This supports:

  * coreos#1411
  * coreos#1424
@baude
Copy link
Contributor Author

baude commented Feb 17, 2023

the fedora-web PR can be found -> https://pagure.io/fedora-web/websites/pull-request/282

@jlebon jlebon changed the title Implement HyperV support Implement Hyper-V support Feb 22, 2023
@bgilbert
Copy link
Contributor

We never decided in #1411 whether serial console is needed. I've added a checklist step to enable that, and we can skip the checkbox if we decide to just stick with graphical console.

baude added a commit to baude/coreos-assembler that referenced this issue Mar 21, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/coreos-assembler that referenced this issue Apr 12, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/ignition that referenced this issue Apr 12, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/coreos-assembler that referenced this issue Apr 12, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/ignition that referenced this issue Apr 12, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/ignition that referenced this issue Apr 13, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/ignition that referenced this issue Apr 14, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/coreos-assembler that referenced this issue Apr 14, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/coreos-assembler that referenced this issue Apr 14, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
bgilbert pushed a commit to coreos/coreos-assembler that referenced this issue Apr 18, 2023
buildextend --compress supports gzip as an alternate compression over
xz using a key called "gzip" where the value is a bool.  This is now
refactored to a key called "compression" with a value of "gzip", "skip",
or "zip" (see below).  The "skip-compression" key is no longer
applicable.

HyperV images which are used on Windows need to be compressed with zip
because xz and gzip are not supported natively.  Added zip support to
buildextend and set the hyperv platform compression to "zip".

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/ignition that referenced this issue Apr 19, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/fedora-coreos-docs that referenced this issue May 8, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft HyperV
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.
This is in support of coreos/fedora-coreos-tracker#1424

Note: I cannot provide a link to HDX images in this documentation
because they are not being made yet.  This will need to be updated.

Signed-off-by: Brent Baude <[email protected]>
baude added a commit to baude/fedora-coreos-docs that referenced this issue May 8, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft HyperV
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.
This is in support of coreos/fedora-coreos-tracker#1424

Note: I cannot provide a link to HDX images in this documentation
because they are not being made yet.  This will need to be updated.

Signed-off-by: Brent Baude <[email protected]>
@baude
Copy link
Contributor Author

baude commented May 8, 2023

Note: serial console is not needed

bgilbert pushed a commit to baude/ignition that referenced this issue Jun 28, 2023
This PR adds initial support for the Microsoft HyperV platform.  It uses
HyperV and MS WMI to read key/value pairs as provided by the Windows
host.  Because the size allocation of the value is quite small, the
ignition values will likely be split into at least 2 and more like more
parts which must be read and reassembled in ignition.

Supports:
  * coreos/fedora-coreos-tracker#1411
  * coreos/fedora-coreos-tracker#1424

Signed-off-by: Brent Baude <[email protected]>
bgilbert added a commit to baude/ignition that referenced this issue Jun 28, 2023
Read the Ignition config from the Hyper-V Data Exchange Service ("KVP"),
which can be configured on the host via WMI.

KVP on Linux is normally handled by a combination of a kernel module
(hv_utils) and a daemon shipped with the kernel source (hv_kvp_daemon).
The latter writes world-readable files to /var/lib/hyperv; these contain
an array of binary structs.  The host-guest protocol has the guest OS as
the passive peer; hv_kvp_daemon connects and is sent all the host's
key-value pairs.

We don't want to require hv_kvp_daemon as a dependency, so we use a
pure Go reimplementation (libhvee) that supports just enough of the
protocol to receive the KVPs.  However, if we disconnect and then
hv_kvp_daemon reconnects later, the host won't re-send the KVPs, so we
also need to save them to /var/lib/hyperv in the same format used by
the daemon.  Unlike the daemon, we set the files to mode 600, since the
Ignition config is potentially sensitive; the daemon does not change
the permissions of existing files.

Small configs can be stored in the `ignition.config` KVP.  However,
individual values have severe length limitations (~1 KiB of UTF-8), so
we also support concatenating `ignition.config.0`, `ignition.config.1`,
etc. to form a larger config.

See also:
coreos/fedora-coreos-tracker#1411
coreos/fedora-coreos-tracker#1424

Almost all of this work was done by Brent Baude; I just added the final
polish.  Thanks Brent!

Co-authored-by: Brent Baude <[email protected]>
bgilbert pushed a commit to baude/stream-metadata-go that referenced this issue Jun 28, 2023
Following new platform support checklist for Hyperv

coreos/fedora-coreos-tracker#1424
bgilbert added a commit to baude/stream-metadata-go that referenced this issue Jun 28, 2023
bgilbert added a commit to baude/fedora-coreos-browser that referenced this issue Jun 28, 2023
bgilbert added a commit to baude/ignition that referenced this issue Jun 28, 2023
Read the Ignition config from the Hyper-V Data Exchange Service ("KVP"),
which can be configured on the host via WMI.

KVP on Linux is normally handled by a combination of a kernel module
(hv_utils) and a daemon shipped with the kernel source (hv_kvp_daemon).
The latter writes world-readable files to /var/lib/hyperv; these contain
an array of binary structs.  The host-guest protocol has the guest OS as
the passive peer; hv_kvp_daemon connects and is sent all the host's
key-value pairs.

We don't want to require hv_kvp_daemon as a dependency, so we use a
pure Go reimplementation (libhvee) that supports just enough of the
protocol to receive the KVPs.  However, if we disconnect and then
hv_kvp_daemon reconnects later, the host won't re-send the KVPs, so we
also need to save them to /var/lib/hyperv in the same format used by
the daemon.  Unlike the daemon, we set the files to mode 600, since the
Ignition config is potentially sensitive; the daemon does not change
the permissions of existing files.

Small configs can be stored in the `ignition.config` KVP.  However,
individual values have severe length limitations (~1 KiB of UTF-8), so
we also support concatenating `ignition.config.0`, `ignition.config.1`,
etc. to form a larger config.

See also:
coreos/fedora-coreos-tracker#1411
coreos/fedora-coreos-tracker#1424

Almost all of this work was done by Brent Baude; I just added the final
polish.  Thanks Brent!

Co-authored-by: Brent Baude <[email protected]>
bgilbert pushed a commit to baude/fedora-coreos-tracker that referenced this issue Jun 28, 2023
Add example metadata for new platform support of hyperv. This supports:

  * coreos#1411
  * coreos#1424
bgilbert added a commit to baude/fedora-coreos-tracker that referenced this issue Jun 28, 2023
Add example metadata for hyperv platform. This supports:

  * coreos#1411
  * coreos#1424

Co-authored-by: Brent Baude <[email protected]>
bgilbert added a commit to baude/ignition that referenced this issue Jun 29, 2023
Read the Ignition config from the Hyper-V Data Exchange Service ("KVP"),
which can be configured on the host via WMI.

KVP on Linux is normally handled by a combination of a kernel module
(hv_utils) and a daemon shipped with the kernel source (hv_kvp_daemon).
The latter writes world-readable files to /var/lib/hyperv; these contain
an array of binary structs.  The host-guest protocol has the guest OS as
the passive peer; hv_kvp_daemon connects and is sent all the host's
key-value pairs.

We don't want to require hv_kvp_daemon as a dependency, so we use a
pure Go reimplementation (libhvee) that supports just enough of the
protocol to receive the KVPs.  However, if we disconnect and then
hv_kvp_daemon reconnects later, the host won't re-send the KVPs, so we
also need to save them to /var/lib/hyperv in the same format used by
the daemon.  Unlike the daemon, we set the files to mode 600, since the
Ignition config is potentially sensitive; the daemon does not change
the permissions of existing files.

Small configs can be stored in the `ignition.config` KVP.  However,
individual values have severe length limitations (~1 KiB of UTF-8), so
we also support concatenating `ignition.config.0`, `ignition.config.1`,
etc. to form a larger config.

See also:
coreos/fedora-coreos-tracker#1411
coreos/fedora-coreos-tracker#1424

Almost all of this work was done by Brent Baude; I just added the final
polish.  Thanks Brent!

Co-authored-by: Brent Baude <[email protected]>
bgilbert pushed a commit to baude/fedora-coreos-docs that referenced this issue Jul 10, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft HyperV
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.
This is in support of coreos/fedora-coreos-tracker#1424

Note: I cannot provide a link to HDX images in this documentation
because they are not being made yet.  This will need to be updated.

Co-authored-by: Nikolas Grottendieck <[email protected]>
bgilbert added a commit to baude/fedora-coreos-docs that referenced this issue Jul 10, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft Hyper-V
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.

See: coreos/fedora-coreos-tracker#1424

Co-authored-by: Nikolas Grottendieck <[email protected]>
Co-authored-by: Benjamin Gilbert <[email protected]>
bgilbert added a commit to baude/fedora-coreos-docs that referenced this issue Jul 19, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft Hyper-V
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.

See coreos/fedora-coreos-tracker#1424.

Co-authored-by: Brent Baude <[email protected]>
Co-authored-by: Nikolas Grottendieck <[email protected]>
bgilbert added a commit to baude/fedora-coreos-docs that referenced this issue Jul 21, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft Hyper-V
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.

See coreos/fedora-coreos-tracker#1424.

Co-authored-by: Brent Baude <[email protected]>
Co-authored-by: Nikolas Grottendieck <[email protected]>
bgilbert added a commit to baude/fedora-coreos-docs that referenced this issue Jul 21, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft Hyper-V
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.

See coreos/fedora-coreos-tracker#1424.

Co-authored-by: Brent Baude <[email protected]>
Co-authored-by: Nikolas Grottendieck <[email protected]>
bgilbert added a commit to coreos/fedora-coreos-docs that referenced this issue Jul 21, 2023
Add instructions on how to boot Fedora CoreOS using the Microsoft Hyper-V
hypervisor.  The instructions cover a basic how-to for setting up the VM
as well as specific information needed for Ignition to work properly.

See coreos/fedora-coreos-tracker#1424.

Co-authored-by: Brent Baude <[email protected]>
Co-authored-by: Nikolas Grottendieck <[email protected]>
@bgilbert
Copy link
Contributor

Hyper-V images will be available in the next set of FCOS releases. I've gone ahead and merged the docs PR in advance.

@travier
Copy link
Member

travier commented Jul 22, 2023

Thanks a lot work all the work here @baude @bgilbert @dustymabe

@bgilbert
Copy link
Contributor

Looks like we missed adding hyperv to cmd-generate-release-meta. Fix in coreos/coreos-assembler#3546. The example coreos-assembler PR does do that, so no checklist update is necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants