-
Notifications
You must be signed in to change notification settings - Fork 399
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(nvmf): add code for parsing the NBFT
Add code to parse the Nvme-oF Boot Firmware Table (NBFT) according
to the NVM Express Boot Specification 1.0 [1]. The implementation in
dracut follows a similar general approach as iBFT support in the
iscsi module.
NBFT support requires two steps:
(1) Setting up the network and routing according to the
HFI ("Host Fabric Interface") records in the NBFT,
(2) Establishing the actual NVMe-oF connection.
(1) is accomplished by reading the NBFT using JSON output from
the "nvme nbft show" command, and transforming it into command
line options ("ip=", "rd.neednet", etc.) understood by dracut's
network module and its backends. The resulting network setup code
is backend-agnostic. It has been tested with the "network-legacy"
and "network-manager" network backend modules. The network setup
code supports IPv4 and IPv6 with static, RA, or DHCP configurations,
802.1q VLANs, and simple routing / gateway setup.
(2) is done using the "nvme connect-all" command [2] in the netroot handler,
which is invoked by networking backends when an interface gets fully
configured. This patch adds support for "netboot=nbft". The "nbftroot"
handler calls nvmf-autoconnect.sh, which contains the actual connect
logic. nvmf-autoconnect.sh itself is preserved, because there are
other NVMe-oF setups like NVMe over FC which don't depend on the
network.
The various ways to configure NVMe-oF are prioritized like this:
1 FC autoconnect from kernel commandline (rd.nvmf.discover=fc,auto)
2 NBFT, if present
3 discovery.conf or config.json, if present, and cmdline.d parameters,
if present (rd.nvmf.discovery=...)
4 FC autoconnect (without kernel command line)
The reason for this priorization is that in the initial RAM fs, we try
to activate only those connections that are necessary to mount the root
file system. This avoids confusion, possible contradicting or ambiguous
configuration, and timeouts from unavailable targets.
A retry logic is implemented for enabling the NVMe-oF connections,
using the "settled" initqueue, the netroot handler, and eventually, the
"timeout" initqueue. This is similar to the retry logic of the iscsi module.
In the "timeout" case, connection to all possible NVMe-oF subsystems
is attempted.
Two new command line parameters are introduced to make it possible to
change the priorities above:
- "rd.nvmf.nonbft" causes the NBFT to be ignored,
- "rd.nvmf.nostatic" causes any statically configured NVMe-oF targets
(config.json, discovery.conf, and cmdline.d) to be ignored.
These parameters may be helpful to skip attempts to set up broken
configurations.
At initramfs build time, the nvmf module is now enabled if an NBFT
table is detected in the system.
[1] https://nvmexpress.org/wp-content/uploads/NVM-Express-Boot-Specification-2022.11.15-Ratified.pdf
[2] NBFT support in nvme-cli requires the latest upstream code (> v2.4).
Signed-off-by: Martin Wilck <[email protected]>
Co-authored-by: John Meneghini <[email protected]>
Co-authored-by: Charles Rose <[email protected]>- Loading branch information
1 parent
f07117d
commit b490f6f
Showing
5 changed files
with
305 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| #! /bin/sh | ||
| # This script is called from /sbin/netroot | ||
|
|
||
| /sbin/nvmf-autoconnect.sh online | ||
| exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,54 @@ | ||
| #!/bin/bash | ||
| #!/bin/sh | ||
| # Argument $1 is "settled", "online", or "timeout", indicating | ||
| # the queue from which the script is called. | ||
| # In the "timeout" case, try everything. | ||
| # Otherwise, try options according to the priorities below. | ||
|
|
||
| [ -f /sys/class/fc/fc_udev_device/nvme_discovery ] || exit 1 | ||
| echo add > /sys/class/fc/fc_udev_device/nvme_discovery | ||
| [ "$RD_DEBUG" != yes ] || set -x | ||
|
|
||
| if [ "$1" = timeout ]; then | ||
| [ ! -f /sys/class/fc/fc_udev_device/nvme_discovery ] \ | ||
| || echo add > /sys/class/fc/fc_udev_device/nvme_discovery | ||
| /usr/sbin/nvme connect-all | ||
| exit 0 | ||
| fi | ||
|
|
||
| NVMF_HOSTNQN_OK= | ||
| [ ! -f "/etc/nvme/hostnqn" ] || [ ! -f "/etc/nvme/hostid" ] || NVMF_HOSTNQN_OK=1 | ||
|
|
||
| # Only nvme-cli 2.5 or newer supports the options --nbft and --no-nbft | ||
| # for the connect-all command. | ||
| # Make sure we don't use unsupported options with earlier versions. | ||
| NBFT_SUPPORTED= | ||
| # shellcheck disable=SC2016 | ||
| /usr/sbin/nvme connect-all --help 2>&1 | sed -n '/[[:space:]]--nbft[[:space:]]/q1;$q0' \ | ||
| || NBFT_SUPPORTED=1 | ||
|
|
||
| if [ -e /tmp/nvmf-fc-auto ] && [ "$NVMF_HOSTNQN_OK" ] \ | ||
| && [ -f /sys/class/fc/fc_udev_device/nvme_discovery ]; then | ||
| # prio 1: cmdline override "rd.nvmf.discovery=fc,auto" | ||
| echo add > /sys/class/fc/fc_udev_device/nvme_discovery | ||
| exit 0 | ||
| fi | ||
| if [ "$NBFT_SUPPORTED" ] && [ -e /tmp/valid_nbft_entry_found ]; then | ||
| # prio 2: NBFT | ||
| /usr/sbin/nvme connect-all --nbft | ||
| exit 0 | ||
| fi | ||
| if [ -f /etc/nvme/discovery.conf ] || [ -f /etc/nvme/config.json ] \ | ||
| && [ "$NVMF_HOSTNQN_OK" ]; then | ||
| # prio 3: configuration from initrd and/or kernel command line | ||
| # We can get here even if "rd.nvmf.nonbft" was given, thus use --no-nbft | ||
| if [ "$NBFT_SUPPORTED" ]; then | ||
| /usr/sbin/nvme connect-all --no-nbft | ||
| else | ||
| /usr/sbin/nvme connect-all | ||
| fi | ||
| exit 0 | ||
| fi | ||
| if [ "$NVMF_HOSTNQN_OK" ] \ | ||
| && [ -f /sys/class/fc/fc_udev_device/nvme_discovery ]; then | ||
| # prio 4: no discovery entries, try NVMeoFC autoconnect | ||
| echo add > /sys/class/fc/fc_udev_device/nvme_discovery | ||
| fi | ||
| exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters