-
Notifications
You must be signed in to change notification settings - Fork 106
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: support lts and production nvidia modules
Support LTS and production versions of NVIDIA kernel modules as per https://docs.nvidia.com/datacenter/tesla/drivers/index.html#lifecycle Part of: siderolabs/talos#9086 Signed-off-by: Noel Georgi <[email protected]>
- Loading branch information
Showing
48 changed files
with
625 additions
and
71 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
2 changes: 1 addition & 1 deletion
2
nvidia-gpu/nonfree/kmod-nvidia/manifest.yaml → ...gpu/nonfree/kmod-nvidia/lts/manifest.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
nvidia-gpu/nonfree/kmod-nvidia/pkg.yaml → nvidia-gpu/nonfree/kmod-nvidia/lts/pkg.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
nvidia-gpu/nonfree/kmod-nvidia/vars.yaml → nvidia-gpu/nonfree/kmod-nvidia/lts/vars.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,2 @@ | ||
| # the first part is the driver version and the second the talos version for which the module is built against | ||
| VERSION: "{{ .NVIDIA_DRIVER_VERSION }}-{{ .BUILD_ARG_TAG }}" | ||
| VERSION: "{{ .NVIDIA_DRIVER_LTS_VERSION }}-{{ .BUILD_ARG_TAG }}" |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| version: v1alpha1 | ||
| metadata: | ||
| name: nonfree-kmod-nvidia-production | ||
| version: "$VERSION" | ||
| author: Sidero Labs | ||
| description: | | ||
| This system extension provides nvidia proprietary kernel modules built against a specific Talos version. | ||
| compatibility: | ||
| talos: | ||
| version: ">= v1.5.0" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| name: nonfree-kmod-nvidia-production | ||
| variant: scratch | ||
| shell: /toolchain/bin/bash | ||
| dependencies: | ||
| - stage: base | ||
| # The pkgs version for a particular release of Talos as defined in | ||
| # https://github.com/siderolabs/talos/blob/<talos version>/pkg/machinery/gendata/data/pkgs | ||
| - image: "{{ .BUILD_ARG_PKGS_PREFIX }}/nonfree-kmod-nvidia-production-pkg:{{ .BUILD_ARG_PKGS }}" | ||
| steps: | ||
| - prepare: | ||
| - | | ||
| sed -i 's#$VERSION#{{ .VERSION }}#' /pkg/manifest.yaml | ||
| - install: | ||
| - | | ||
| mkdir -p /rootfs/lib/modules \ | ||
| /rootfs/usr/local/lib/modprobe.d | ||
| cp /pkg/files/nvidia.conf /rootfs/usr/local/lib/modprobe.d/nvidia.conf | ||
| cp -R /lib/modules/* /rootfs/lib/modules | ||
| test: | ||
| - | | ||
| mkdir -p /extensions-validator-rootfs | ||
| cp -r /rootfs/ /extensions-validator-rootfs/rootfs | ||
| cp /pkg/manifest.yaml /extensions-validator-rootfs/manifest.yaml | ||
| /extensions-validator validate --rootfs=/extensions-validator-rootfs --pkg-name="${PKG_NAME}" | ||
| finalize: | ||
| - from: /rootfs | ||
| to: /rootfs | ||
| - from: /pkg/manifest.yaml | ||
| to: / |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # the first part is the driver version and the second the talos version for which the module is built against | ||
| VERSION: "{{ .NVIDIA_DRIVER_PRODUCTION_VERSION }}-{{ .BUILD_ARG_TAG }}" |
2 changes: 1 addition & 1 deletion
2
...pu/nvidia-container-toolkit/manifest.yaml → ...vidia-container-toolkit/lts/manifest.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
4 changes: 2 additions & 2 deletions
4
nvidia-gpu/nvidia-container-toolkit/pkg.yaml → ...gpu/nvidia-container-toolkit/lts/pkg.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...ia-gpu/nvidia-container-toolkit/vars.yaml → ...pu/nvidia-container-toolkit/lts/vars.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,2 @@ | ||
| # the first part is the driver version and the second the container-toolkit version | ||
| VERSION: "{{ .NVIDIA_DRIVER_VERSION }}-{{ .CONTAINER_TOOLKIT_VERSION }}" | ||
| VERSION: "{{ .NVIDIA_DRIVER_LTS_VERSION }}-{{ .CONTAINER_TOOLKIT_VERSION }}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
67 changes: 67 additions & 0 deletions
67
nvidia-gpu/nvidia-container-toolkit/nvidia-container-cli/production/pkg.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| name: nvidia-container-cli-production | ||
| variant: scratch | ||
| shell: /bin/bash | ||
| install: | ||
| - build-base | ||
| - bash | ||
| - go | ||
| - coreutils | ||
| - sed | ||
| - curl | ||
| - rpcsvc-proto | ||
| - patch | ||
| dependencies: | ||
| - image: cgr.dev/chainguard/wolfi-base@{{ .WOLFI_BASE_REF }} | ||
| # nvidia-pkgs depends on glibc, | ||
| # so any stage depending on nvidia-container-cli will have the updated ld.so.cache, | ||
| # from both nvidia-pkgs and nvidia-container-cli | ||
| - stage: nvidia-pkgs-production | ||
| - stage: libseccomp | ||
| from: /rootfs | ||
| - stage: libcap | ||
| from: /rootfs | ||
| - stage: elfutils | ||
| from: /rootfs | ||
| - stage: zlib | ||
| from: /rootfs | ||
| - stage: libtirpc | ||
| from: /rootfs | ||
| steps: | ||
| - sources: | ||
| - url: https://github.com/NVIDIA/libnvidia-container/archive/refs/tags/{{ .LIBNVIDIA_CONTAINER_VERSION }}.tar.gz | ||
| destination: libnvidia-container.tar.gz | ||
| sha256: cbc1dda7ee90b8b729c5f178292cd07b421863015d84b84c37e69c8d580ab3ff | ||
| sha512: b304c284c5ab0c3544362307dc16ffcca8d34497e4356a520dc6da81a86a62b2a262b528cba559bb0d7a3addf018c3b50b6cb78669c82c1b4acae159e5922548 | ||
| env: | ||
| SOURCE_DATE_EPOCH: {{ .BUILD_ARG_SOURCE_DATE_EPOCH }} | ||
| REVISION: {{ .LIBNVIDIA_CONTAINER_REF }} | ||
| LIB_VERSION: {{ .LIBNVIDIA_CONTAINER_VERSION | replace "v" "" }} | ||
| WITH_NVCGO: yes | ||
| WITH_LIBELF: yes | ||
| WITH_TIRPC: no # setting no means we'll use the system libtirpc | ||
| WITH_SECCOMP: yes | ||
| PKG_CONFIG_PATH: /usr/local/glibc/lib/pkgconfig # to find runtime libraries compiled in extensions (libseccomp) | ||
| PATH: "/usr/bin:{{ .PATH }}" # bldr doesn't have /usr/bin in PATH | ||
| prepare: | ||
| - | | ||
| mkdir libnvidia-container | ||
| tar -xzf libnvidia-container.tar.gz --strip-components=1 -C libnvidia-container | ||
| build: | ||
| - | | ||
| cd libnvidia-container | ||
| # LDLIBS=-L/usr/local/glibc/lib is set so that libnvidia-container-cli libs which are hardcoded as -llibname and not using pkg-config | ||
| CPPFLAGS="-I/usr/local/glibc/include/tirpc" LDLIBS="-L/usr/local/glibc/lib -ltirpc -lelf -lseccomp" LDFLAGS='-Wl,--rpath=\$$ORIGIN/../glibc/\$$LIB' make | ||
| install: | ||
| - | | ||
| mkdir -p /rootfs | ||
| cd libnvidia-container | ||
| make install DESTDIR=/rootfs | ||
| # run ldconfig to update the cache | ||
| /rootfs/usr/local/glibc/sbin/ldconfig -r /rootfs | ||
| finalize: | ||
| - from: /rootfs | ||
| to: /rootfs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
Oops, something went wrong.