Add firmware_update phase

[nancyc12]

Description

Base on CR033 - Firmware updates for regression testing, introduce a new optional phase firmware_update after provision phase to handle request for firmware update on the device. As in the spec, this new phase is implemented in DefaultDevice, so it won't need any change on existing DeviceConnector codes.

To trigger firmware_update phase, the following section needs to be provided in job.yaml. If the firmware_update_data section is missing from the job, this phase will not run.

firmware_update_data:
  version: < latest >
  ignore_failure: < false | true >

Variables in firmware_update_data:

• version: The desired firmware level on the device. Currently it only supports latest (upgrading all components in the device with the latest firmware release). More options are planned to be implemented in the future once we gathering data for working firmware baseline on devices.
• ignore_failure: When set to false, Testflinger Agent will suspend the job if firmware_update phase return a status other than 0, which implies there's a failure during firmware_update phase. If set to true, the job will continue regardless the status of firmware_update phase.

Code changes

Server

• Update schema.py to let testflinger server accept job with firmware_update_data section.
• Update job_detail.html to let it display firmware_update phase information.

Agent

• Add firmware_update phase in job.py and agent.py.
• Update schema.py to handle the phase with firmware_update_command.

Important

For each Testflinger agent instance, extra configuration needs to be added to testflinger-agent.conf. For example,
firmware_update_command: testflinger-device-connector noprovision firmware_update -c default.yaml testflinger.json.
However, it won't break the agent if you don't provide firmware_update_command in the conf file, it just loses the support for firmware_update phase.

Device Connector

Add firmware_update phase in DefaultDevice to handle the firmware update for all child DeviceConnector class. It will gather DMI chassis_vendor and chassis_type from the DUT to decide whether firmware update is applicable. The current support scope is limited the devices as below.

chassis_vendor: HP, Lenovo, Dell
chassis_type: All In One, Desktop, Mini PC, Rack Mount Chassis

Resolved issues

N/A

Documentation

As discussed with @tang-mm, the documentation for firmware_update phase should include the following:

• General Introduction:
  covered in Doc: add test phase reference #134

• Testflinger agents config:
  definition and usage for firmware_update_command, which has been updated in add reference docs for configuration options #113

• Testflinger jobs config:
  covered in Doc: add test phase reference #134

• Troubleshooting:
  provided necessary command output and detailed firmware update status in testflinger log

Tests

It's tested by building a dev testflinger server with docker along with modified agent and device connector.

• Unit tests for server

• Unit tests for agent

• Unit tests for device connector

• Testflinger server is able to accept job.yaml with firmware_update_data section, and propagate the job json to the agent. Agent is able to work with Device Connector (tested with noprovision) to run firmware_update phase, and populate the data to firmware_update_data, firmware_update_status and firmware_update_output section in server job webpage.

**************************************************************************

* Starting testflinger firmware_update phase on dell-optiplex3070-c26629 *

**************************************************************************
2023-10-13 12:06:46,578 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: BEGIN firmware_update
2023-10-13 12:06:47,130 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: 10.102.163.206 is a Dell Inc. LVFSDevice
2023-10-13 12:06:57,154 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: collect firmware info
2023-10-13 12:06:58,600 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: [ST500LM021-1KJ152] current version: 0005SDM1 - no available firmware on LVFS
2023-10-13 12:06:58,601 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: [System Firmware] current version: 1.21.0 - LVFS upgradable version(s): 1.22.0 - LVFS downgradable version(s): 1.20.0
2023-10-13 12:06:58,601 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: [UEFI dbx] current version: 77 - no available firmware on LVFS
2023-10-13 12:06:58,601 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: start upgrading
2023-10-13 12:06:58,601 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: [System Firmware] try upgrading to 1.22.0
2023-10-13 12:07:05,600 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: reboot DUT
2023-10-13 12:07:16,680 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: check and wait for 900s until 10.102.163.206 is SSHable
2023-10-13 12:08:53,927 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: 10.102.163.206 is SSHable after 97s
2023-10-13 12:09:01,255 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: [System Firmware] FWUPD_UPDATE_STATE_FAILED: failed to run update on reboot
2023-10-13 12:09:01,255 dell-optiplex3070-c26629 INFO: DEVICE CONNECTOR: END firmware_update

[plars]

Thanks for getting this all figured out! it's a complicated piece and what you've done here is fantastic, has unit tests, and I know the documentation has landed separately already. So I think it's in pretty good shape already, but there are a couple of minor things that I mentioned below, that I think would really improve it even more. Take a look and see what you think.

[plars]

+1 great work!