Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

U-Boot integration [WIP] #1411

Open
wants to merge 62 commits into
base: master
Choose a base branch
from

Conversation

sjg20
Copy link
Contributor

@sjg20 sjg20 commented May 21, 2024

Cover the drivers, scripts, pytest and the Gitlab information.

Note: This implementation is the result of working through what is
needed in Labgrid to support U-Boot labs. I did initially file quite
a few issues[1] but there has not been a lot of traction and I got
feedback that I have overwhelmed people with too many. So I have
stopped filing issues on the things I hit along the way. I have carried
on with the implementation in the hope that this can be a better basis
for discussion.

Note that all of these patches are work-in-progress, but it is becoming
more stable.

Feedback may still change the implementation and documentation but the
time for wholesale changes of approach has passed.

Cover-letter:
U-Boot integration
END

Changes in v6:

  • Show a more friendly error when the wrong query arguments are given
  • Add new patch to report an error if QEMU is not turned on
  • Improve handling of dual-build targets so pytest works correctly
  • Support QEMU on ARM
  • Support writing a VBE (Verified Boot for Embedded) TPL file
  • Fix reference to crossbar in _ub-bisect-try and ub-smoke
  • Add -x option to specify the extra board directory
  • Add comments to the example lg-env
  • Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
  • Add a few more comments to ub-pyt
  • Show the build path with ub-pyt -v
  • Add --allow-unmatched to ub-smoke to avoid QEMU error
  • Add a patch with an example lab

Changes in v4:

  • Get internal console working with qemu
  • Show a prompt when starting, to indicate it is waiting for the board

Simon Glass (62):
util/helper: Allow setting environment and cwd
util/helper: Change command output level to debug
helper: Support long-running processes
Provide variables to control drivers from the command line
driver: Add driver for Dediprog SPI-flash emulator
protocol: Add a new recovery protocol
protocol: Allow more control of reset
labgrid: Add a way to query information from a driver
remote/client: Support finding a place by role
driver/qemudriver: Make extra_args, cpu and machine optional
driver/qemudriver: Delay setting of the QEMU arguments
driver/qemudriver: Allow the BIOS filename to be changed
driver/qemudriver: Report an error if QEMU is not turned on
driver: Add a digital-output driver for recovery
driver: Add a driver for the Servo board
driver: Add a way to build U-Boot images
driver: Provide a driver for writing U-Boot to a board
Add a driver for a device which is always powered
Correct binding for HIDRelayDriver
bootstrapprotocol: Support a boot phase
driver/usbloader: Add a USB loader for sunxi / Allwinner
driver/usbloader: Add a loader for Tegra
driver/usbloader: Add a loader for samsung
driver/usbstoragedriver: Allow block size in write_image()
driver/usbstoragedriver: Allow block count in write_image()
driver/usbstoragedriver: Adjust how sync works
driver/usbstoragedriver: Write silently
driver/usbstoragedriver: Report time take to write
pytestplugin: Allow selecting a particular target
pytestplugin: Allow setting variables
usbhidrelay: Support generating a recovery signal
remote/exporter: Reduce the verbosity
remote/exporter: Reduce the verbosity further
remote/exporter: Indicate when the exporter is ready
remote/client: Provide an option to acquire a place
remote/client: Using logging for progress output
remote/client: Show the result of a failed command
remote/client: Allow releasing a target silently
remote/client: Fix environemnt typo
remote/client: Use logging when acquiring / releasing
remote/client: Flush the console device only when needed
remote/client: Move initial-state code into a function
remote/client: Provide a way to set the strategy end-state
remote/client: Allow checking if the place is acquired
remote/client: Move terminal handling into a separate file
remote/client: Provide an internal console
ssh: Avoid output when running rsync
target: Add documentation
target: Add documentation for get_driver()
target: Allow looking for an optional driver
ubootdriver: Support collecting the version string
ubootdriver: Provide a way to collect output
strategy: Show the U-Boot version when booting
strategy: Add a new state to start U-Boot
strategy: Allow a board to be bootstrapped
strategy: Allow sending U-Boot over USB
strategy: Show console output on failure
strategy: Support boards which need a recovery button
strategy: Support lab mode
contrib: Provide some useful scripts for U-Boot
Add an example lab
doc: Add documentation for U-Boot integration [v2]

contrib/u-boot/.gitignore | 1 +
contrib/u-boot/_ub-bisect-try | 47 +
contrib/u-boot/conftest.py | 21 +
contrib/u-boot/example_env.cfg | 1245 +++++++++++++++++++++++++
contrib/u-boot/example_export.yaml | 938 +++++++++++++++++++
contrib/u-boot/example_places.yaml | 152 +++
contrib/u-boot/get_args.sh | 136 +++
contrib/u-boot/index.rst | 232 +++++
contrib/u-boot/lg-client | 11 +
contrib/u-boot/lg-env | 24 +
contrib/u-boot/test_smoke.py | 3 +
contrib/u-boot/ub-bisect | 45 +
contrib/u-boot/ub-cli | 40 +
contrib/u-boot/ub-int | 42 +
contrib/u-boot/ub-pyt | 74 ++
contrib/u-boot/ub-smoke | 49 +
doc/configuration.rst | 611 +++++++++++-
doc/usage.rst | 401 ++++++++
labgrid/driver/init.py | 10 +-
labgrid/driver/common.py | 11 +
labgrid/driver/consoleexpectmixin.py | 7 +
labgrid/driver/powerdriver.py | 29 +
labgrid/driver/qemudriver.py | 86 +-
labgrid/driver/recoverydriver.py | 25 +
labgrid/driver/resetdriver.py | 7 +
labgrid/driver/servodriver.py | 157 ++++
labgrid/driver/sfemulatordriver.py | 102 ++
labgrid/driver/ubootdriver.py | 28 +-
labgrid/driver/ubootproviderdriver.py | 326 +++++++
labgrid/driver/ubootwriterdriver.py | 169 ++++
labgrid/driver/usbhidrelay.py | 7 +-
labgrid/driver/usbloader.py | 170 +++-
labgrid/driver/usbstoragedriver.py | 18 +-
labgrid/factory.py | 4 +-
labgrid/protocol/init.py | 1 +
labgrid/protocol/bootstrapprotocol.py | 8 +-
labgrid/protocol/recoveryprotocol.py | 14 +
labgrid/protocol/resetprotocol.py | 12 +
labgrid/pytestplugin/fixtures.py | 21 +-
labgrid/pytestplugin/hooks.py | 8 +
labgrid/remote/client.py | 266 ++++--
labgrid/remote/config.py | 7 +-
labgrid/remote/exporter.py | 123 ++-
labgrid/resource/init.py | 6 +
labgrid/resource/remote.py | 24 +
labgrid/resource/servo.py | 485 ++++++++++
labgrid/resource/sfemulator.py | 32 +
labgrid/resource/suggest.py | 6 +
labgrid/resource/udev.py | 33 +
labgrid/strategy/ubootstrategy.py | 148 ++-
labgrid/target.py | 120 ++-
labgrid/util/helper.py | 222 +++--
labgrid/util/ssh.py | 3 +-
labgrid/util/term.py | 196 ++++
labgrid/var_dict.py | 8 +
man/labgrid-client.1 | 6 +
man/labgrid-client.rst | 4 +
man/labgrid-device-config.5 | 4 +
58 files changed, 6715 insertions(+), 270 deletions(-)
create mode 100644 contrib/u-boot/.gitignore
create mode 100755 contrib/u-boot/_ub-bisect-try
create mode 100644 contrib/u-boot/conftest.py
create mode 100644 contrib/u-boot/example_env.cfg
create mode 100644 contrib/u-boot/example_export.yaml
create mode 100644 contrib/u-boot/example_places.yaml
create mode 100755 contrib/u-boot/get_args.sh
create mode 100644 contrib/u-boot/index.rst
create mode 100755 contrib/u-boot/lg-client
create mode 100755 contrib/u-boot/lg-env
create mode 100644 contrib/u-boot/test_smoke.py
create mode 100755 contrib/u-boot/ub-bisect
create mode 100755 contrib/u-boot/ub-cli
create mode 100755 contrib/u-boot/ub-int
create mode 100755 contrib/u-boot/ub-pyt
create mode 100755 contrib/u-boot/ub-smoke
create mode 100644 labgrid/driver/recoverydriver.py
create mode 100644 labgrid/driver/servodriver.py
create mode 100644 labgrid/driver/sfemulatordriver.py
create mode 100644 labgrid/driver/ubootproviderdriver.py
create mode 100644 labgrid/driver/ubootwriterdriver.py
create mode 100644 labgrid/protocol/recoveryprotocol.py
create mode 100644 labgrid/resource/servo.py
create mode 100644 labgrid/resource/sfemulator.py
create mode 100644 labgrid/util/term.py
create mode 100644 labgrid/var_dict.py

@sjg20 sjg20 force-pushed the u-boot-integration branch 2 times, most recently from 4cdfb54 to c4b13af Compare June 10, 2024 22:14
@sjg20 sjg20 changed the title U boot integration [WIP] U-Boot integration [WIP] Jun 25, 2024
@sjg20
Copy link
Contributor Author

sjg20 commented Jun 25, 2024

Changes for v3:

  • Rebase on top of grpc branch
  • Don't mess with terminal setting unless stdin is a terminal
  • Don't show an error if there are no resources when auto-acquiring
  • Support QEMU in UBootWriter

trini added a commit to trini/u-boot that referenced this pull request Jul 9, 2024
Simon Glass <[email protected]> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrote a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html
@sjg20 sjg20 requested a review from krevsbech as a code owner July 29, 2024 02:22
@sjg20
Copy link
Contributor Author

sjg20 commented Jul 29, 2024

Changes in v4:

  • Support for Beagleplay, which needs files from two separate U-Boot
    builds
  • Support for a 'recovery' button needed to boot the image
  • Tidy up the internal-console support
  • Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab
    mode)

@sjg20 sjg20 force-pushed the u-boot-integration branch 2 times, most recently from 6259b9a to 8f0e69e Compare August 16, 2024 13:47
When running a process it is sometimes necessary to pass environment
variables and to run the process in a particular directory. Add support
for these features.

Signed-off-by: Simon Glass <[email protected]>
It isn't generally useful to print out every command before executing
it. Now that the client shows failed commands fully, it seems better to
change this to show only when debugging.

Signed-off-by: Simon Glass <[email protected]>
The current subprocess helper assumes it is the only thing running. In
some cases (such as the Dediprog em100 tool) we want to start a process
and have it run for a while in the background.

Rather than invent another helper, modify the existing helper to support
this.

For existing use cases, no functional change is intended.

Signed-off-by: Simon Glass <[email protected]>
Some drivers need to be told what to do from the command line. This is
particularly true of complex strategy drivers, such as the one to write
U-Boot to board media, or send it over USB.

Provide a way to specify driver-specific variables with a string value,
so that they can control the strategy. Examples include whether to
bootstrap the device or just boot the existing image, whether to build a
new image or use an existing one and whether to write the image to boot
media or send it over USB.

With these variables a useful selection of shell scripts can provide a
highly productive development and test environment, without a huge
prolification of very similar drivers. This approach also avoids the
complex interactions that would be necessary between different drivers
aimed towards the same goal.

Signed-off-by: Simon Glass <[email protected]>
The EM100-pro is a SPI-flash emulator which supports a variety of chips.
It can be used to provide an image to a booting system.

Add a driver which allows images to be loaded ready for use. Provide a
trace facility for use when the board does not boot.

Signed-off-by: Simon Glass <[email protected]>
Some boards have a way to put them into a 'recovery' mode provided by
the SoC. Recovery is a signal asserted at power-on (or when the board is
reset). It causes the SoC to listen on USB.

Add a protocol for this feature.

Signed-off-by: Simon Glass <[email protected]>
The reset protocol only supports doing a reset. Sometimes we want to
assert reset while something is done (e.g. a power cycle or a recovery
signal), then de-assert it afterwards.

Add a new set_reset_enable() method for this.

The name is a little strange: one might expect set_enable() instead. But
with things like the PowerResetMixin we want to avoid ambiguity over
whether this refers to powering the device or resetting it.

Signed-off-by: Simon Glass <[email protected]>
When integrating Labgrid into other projects it is useful to be able to
query information known by a particular driver. For example, U-Boot
wants to know the board name of a particular target, so that it can
pass this to pytest.

Add a new driver method to provide this feature.

Series-changes: 6
- Show a more friendly error when the wrong query arguments are given

Signed-off-by: Simon Glass <[email protected]>
The role (or is it target?) is used in the environment and allows
multiple roles to use the same place. It is therefore often better to
specify the role to use rather than the place, since the place can be
determined from the role.

Add a new -r option to support this.

Signed-off-by: Simon Glass <[email protected]>
At least for x86 devices it is not necessary to provide these arguments.
Make them optional.

Signed-off-by: Simon Glass <[email protected]>
These arguments are currently set on activation. Where the driver is
being used by a strategy, it may need to set the arguments after that.
There is no need to have everything set in stone until the on() method
is called, so move it there.

Signed-off-by: Simon Glass <[email protected]>
The U-Boot strategy needs to provide the filename of the BIOS directly.
Add a method to support this.

Signed-off-by: Simon Glass <[email protected]>
Rather than provide a strange exception, complain when a strategy does
not use the QEMU driver properly, perhaps due to user settings.

Series-changes: 6
- Add new patch to report an error if QEMU is not turned on

Signed-off-by: Simon Glass <[email protected]>
It is common to use a digital output (such as a relay) to assert the
recovery signal. Add a driver to handle this.

Signed-off-by: Simon Glass <[email protected]>
Servo is a range of boards used by ChromiumOS devices to provide reset,
recovery and power control, as well as multiple serial consoles.

Add a resource and driver which provides these basic facilities.

Signed-off-by: Simon Glass <[email protected]>
Building firmware images is fairly complex these days, due to the wide
variety of binaries that need to be assembled together.

U-Boot provides two useful tools here:
- buildman can build U-Boot itself, selecting a suitable toolchain
- binman can assemble / sign images, pulling in required binaries

It is convenient to build these images within Labgrid, since each board
has a selection of necessary binaries, which can be provided as part
of the lab configuration.

This allows a fully functional firmware image to be produced for any
board as part of setting up a Labgrid configuration for that board.

A particularly useful case is for interactive use, where the current
source tree is built and booted on a selected board, resulting in a
console for that board (if everything goes well). The driver includes
some features which make this easy to implement.

Note that Labgrid itself does not build U-Boot. It just calls buildman
to make it happen.

Series-changes: 6
- Improve handling of dual-build targets so pytest works correctly

Signed-off-by: Simon Glass <[email protected]>
Each SoC has its own way of booting and this means that the firmware
layout for each SoC is typically different. On top of that, some SoCs
allow writing an image over USB. which uses a different format, e.g.
separate binaries for each stage.

Trying to make all this work is extremely challenging. Labgrid provides
a nice unification point, where the method appropriate for each SoC can
be included, with a small amount of configuration to control things.

Binary blobs can be provided to make the writer work correctly.

This driver is capable of writing SD-card images for several popular
devices as well as handling USB booting in some cases. It also supports
QEMU. It can be easily extended as necessary.

Series-changes: 6
- Support QEMU on ARM
- Support writing a VBE (Verified Boot for Embedded) TPL file

Signed-off-by: Simon Glass <[email protected]>
It isn't obvious when the exporter has finished setting things up and
the labe is ready for use. This seems to take quite a while - e.g. 15
seconds for 125 resources.

Add a message to provide this signal, including how many resources are
exported.

Signed-off-by: Simon Glass <[email protected]>
It is common to acquire a place and then use it to access the console.
Add a -a option to support this, to avoid running labgrid-client twice,
which takes longer and makes error handling harder.

If the place is acquired, it is released on exit.

Signed-off-by: Simon Glass <[email protected]>
Some messages are printed regardless of the logging level. Convert them
to logging statements so they can be suppressed.

Signed-off-by: Simon Glass <[email protected]>
When a command fails in labgrid-client it just shows the return code.
This makes it very hard to see what actually went wrong. Handle this
exception explicitly, showing the output of the tool which failed. This
allows easy debugging of the problem.

This is particularly important with things like USB loaders.

Signed-off-by: Simon Glass <[email protected]>
When a target may or may not be acquired, it is useful to be able to
release it without checking first. Add an option for this, so that it
is not necessary to check first.

Signed-off-by: Simon Glass <[email protected]>
Fix a small typo in a message.

Signed-off-by: Simon Glass <[email protected]>
Use logging rather than print so that this output can be controlled.

Signed-off-by: Simon Glass <[email protected]>
At present if a console device is used by a strategy driver it is then
removed by the client immediately afterwards. If the 'console' command
is used, this means that any console data is lost.

This is done so that the console device is release for microcom, so
move the logic in with the microcom logic. With the forthcoming internal
terminal, it will not be needed.

Signed-off-by: Simon Glass <[email protected]>
Put this code into a function to reduce the size of the _get_target()
function.

Signed-off-by: Simon Glass <[email protected]>
It is possible to set the initial state of a strategy, but not the final
state. In many cases we want to power the board off and otherwise clean
things up at the end. Provide a -e option to accomplish this.

Signed-off-by: Simon Glass <[email protected]>
The current _check_allowed() function is not very friendly in that it
produces an exception if something is wrong. Refactor it to add a new
function which just returns an error message in that case. This will
allow other modules to call it without creating an exception.

Signed-off-by: Simon Glass <[email protected]>
There is quite a lot of code here, so move the terminal function into
its own file. This will make it easier to extend it later.

Signed-off-by: Simon Glass <[email protected]>
At present Labgrid uses microcom as its console. This has some
limitations:

- console output is lost between when the board is reset and microcom
  connects
- txdelay cannot be handled in microcom, meaning that boards may fail
  to receive expected output
- the console may echo a few characters back to the caller in the time
  between when 'labgrid-client console' is executed and when microcom
  starts (which causes failures with U-Boot test system)

For many use cases, microcom is more than is needed, so provide a simple
internal terminal which resolved the above problems.

It is enabled by a '-i' option to the 'console' command, as well as an
environment variable, so that it can be adjustly without updating a lot
of scripts.

To exit, press Ctrl-] twice, quickly.

Series-changes: 4
- Get internal console working with qemu
- Show a prompt when starting, to indicate it is waiting for the board

Signed-off-by: Simon Glass <[email protected]>
Transferring a file around is an operation which happens internally
within Labgrid. It is confusing to show these transfers as they happen.
Use quiet mode with rsync to avoid unwanted output.

Signed-off-by: Simon Glass <[email protected]>
This class has a lot of members whose type and purpose is hard to figure
out. Add some documentation at the top.

Signed-off-by: Simon Glass <[email protected]>
This function lacks documentation. Tidy it up.

While we are here, reverse the order of the 'active' and 'activate'
arguments, to separate the arguments into those at the start which
provide the filter and the one at the end which requests an action.

Signed-off-by: Simon Glass <[email protected]>
Some strategies may want to use a driver if it exists, without failing
if it does not. Add an argument to get_driver() to support this.

Signed-off-by: Simon Glass <[email protected]>
U-Boot prints a banner when it starts up, which contains the version
string. Collect this so it can be used by the strategy driver as needed.

Signed-off-by: Simon Glass <[email protected]>
When something goes wrong it is useful to see what output was processed
as a clue to what happened. Collect everything up to the autoboot
prompt, since once it hits that things are normally working OK.

Signed-off-by: Simon Glass <[email protected]>
At present the 'start' state of U-Boot consumes all the initial U-Boot
output, including the SPL and proper headers and any version
information. This makes it impossible to run the U-Boot pytests, since
this output is expected from the board.

There are two ways to fix this:
- Show the output
- Print a message indicating that U-Boot has booted, so tests don't
  wait for the headers

Do both, the first to make tests work existing U-Boot work with tests,
the second to resolve the problem for future U-Boot versions, once the
U-Boot pytest patch is accepted:

   test: Allow signaling that U-Boot is ready

Signed-off-by: Simon Glass <[email protected]>
The U-Boot strategy only allows bringing up a U-Boot prompt. If for some
reason this fails (e.g. due to a bug), nothing is shown on the console.

For interactive use, it is helpful to see any output, even if it does
not ultimately result in reach the U-Boot prompt.

Add a new 'start' state, to start U-Boot. Use this from the 'uboot'
state.

Signed-off-by: Simon Glass <[email protected]>
It is useful to write a new U-Boot version to the board before starting
it up. This is particularly valuable when using labgrid interactively,
to test out / debug U-Boot on various boards.

We end up with several different paths through the strategy:

- just power on/reset the board
- bootstrap: write an existing U-Boot to the board, then power it on
- build and bootstrap: build a new U-Boot, write it to the board, then
  power it on

All of these are useful in an interactive context.

Implement this by calling the U-Boot provider to build U-Boot, then the
U-Boot writer to write it to the board. Finally, power on/reset the
board as normal.

This is controlled by two variables:

   do-bootstrap: 1 to bootstrap the board, 0 to leave it along
   do-build:     1 to build U-Boot, 0 to use the existing build

Signed-off-by: Simon Glass <[email protected]>
Some boards do not have boot media, or it cannot be accessed by the test
system.

In this case it is useful to send U-Boot to the board using the
USB-recovery mechanism. Add the logic for this, controlled by a new
'do-send' variable.

Even for boards which do have boot media, it can still be useful to use
the USB mechanism.

Add the logic for this, using a new 'do-send' variable to control it. If
'do-send' is 1, U-Boot is written over USB. If it is 0, then U-Boot is
written to the boot media.

Signed-off-by: Simon Glass <[email protected]>
When Labgrid fails to get to the U-Boot command line, show the output
that was received, so that the user can see what is going on.

Signed-off-by: Simon Glass <[email protected]>
Some boards boot from internal flash unless a 'recovery' button is
pressed. Add support for this, so that these boards can be used in a
lab.

When Labgrid fails to get to the U-Boot command line, show the output
that was received, so that the user can see what is going on.

Signed-off-by: Simon Glass <[email protected]>
U-Boot's pytest framework has lots of logic to detect the U-Boot banner
and deal with failures to do so.

Labgrid supports this in its U-Boot driver.

We don't need both and Labgrid's implementation is better. It has more
knowledge of what is going on, since it is in charge of bringing the
board to life. It can be configured to know how long the board takes to
start up.

Add a 'lab mode' where Labgrid does all the work and just announces when
things are ready. This is enabled by presence of the U_BOOT_SOURCE_DIR
environment variable, which is set by U-Boot's pytest system.

This greatly simplifies operation of lab testing and ensures that pytest
is not trying to guess what happened.

Signed-off-by: Simon Glass <[email protected]>
Provide some scripts which facilitate U-Boot testing and development
using Labgrid.

Series-changes: 6
- Fix reference to crossbar in _ub-bisect-try and ub-smoke
- Add -x option to specify the extra board directory
- Add comments to the example lg-env
- Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
- Add a few more comments to ub-pyt
- Show the build path with ub-pyt -v
- Add --allow-unmatched to ub-smoke to avoid QEMU error

Signed-off-by: Simon Glass <[email protected]>
Add an example lab, based on my one, so people can see how to configure
things for this integration.

Series-changes: 6
- Add a patch with an example lab

Signed-off-by: Simon Glass <[email protected]>
Cover the drivers, scripts, pytest and the Gitlab information.

Note: This implementation is the result of working through what is
needed in Labgrid to support U-Boot labs. I did initially file quite
a few issues[1] but there has not been a lot of traction and I got
feedback that I have overwhelmed people with too many. So I have
stopped filing issues on the things I hit along the way. I have carried
on with the implementation in the hope that this can be a better basis
for discussion.

Note that all of these patches are work-in-progress, but it is becoming
more stable.

Feedback may still change the implementation and documentation but the
time for wholesale changes of approach has passed.

Cover-letter:
U-Boot integration
END

Changes in v6:
- Fix reference to crossbar in _ub-bisect-try and ub-smoke
- Add -x option to specify the extra board directory
- Add comments to the example lg-env
- Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
- Add a few more comments to ub-pyt
- Show the build path with ub-pyt -v
- Add --allow-unmatched to ub-smoke to avoid QEMU error
- Add a new section on compatibility
- Add a new section on script usage

Changes in v5:
- Rebase on latest grpc branch
- Fix pylint errors and warnings

Changes in v4:
- Support for Beagleplay, which needs files from two separate U-Boot
  builds
- Support for a 'recovery' button needed to boot the image
- Tidy up the internal-console support
- Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab
  mode)

Changes in v3:
- Rebase on top of grpc branch
- Don't mess with terminal setting unless stdin is a terminal
- Don't show an error if there are no resources when auto-acquiring
- Support QEMU in UBootWriter

Some changes in v2:
- Rationalise the flags for the U-Boot scripts
- Support tracing with em100
- Support an internal terminal instead of microcom
- Add a -D flag for debugging
- Support send-only boards
- Add a way to build the U-Boot config
- Add a control for buildman's process-limit
- allow the build-dir to be specified in a variable
- add documentation about U-Boot-pytest integration
- add source_dir and config_file to UBootProviderDriver
- add an internal terminal
- expand the U-Boot scripts
- significantly improve the U-Boot-pytest integration

The approximate diffstat is:
 contrib/sync-places.py                |  23 +-
 contrib/u-boot/.gitignore             |   1 +
 contrib/u-boot/_ub-bisect-try         |  47 ++
 contrib/u-boot/conftest.py            |  21 +
 contrib/u-boot/get_args.sh            | 128 +++++
 contrib/u-boot/index.rst              | 232 +++++++++
 contrib/u-boot/lg-client              |  11 +
 contrib/u-boot/lg-env                 |  10 +
 contrib/u-boot/test_smoke.py          |   3 +
 contrib/u-boot/ub-bisect              |  44 ++
 contrib/u-boot/ub-cli                 |  39 ++
 contrib/u-boot/ub-int                 |  41 ++
 contrib/u-boot/ub-pyt                 |  69 +++
 contrib/u-boot/ub-smoke               |  48 ++
 doc/configuration.rst                 | 611 ++++++++++++++++++++++-
 doc/usage.rst                         | 288 +++++++++++
 labgrid/driver/__init__.py            |  10 +-
 labgrid/driver/common.py              |  11 +
 labgrid/driver/consoleexpectmixin.py  |   7 +
 labgrid/driver/powerdriver.py         |  29 ++
 labgrid/driver/qemudriver.py          |  84 ++--
 labgrid/driver/recoverydriver.py      |  25 +
 labgrid/driver/resetdriver.py         |   7 +
 labgrid/driver/servodriver.py         | 157 ++++++
 labgrid/driver/sfemulatordriver.py    | 102 ++++
 labgrid/driver/ubootdriver.py         |  27 +-
 labgrid/driver/ubootproviderdriver.py | 323 ++++++++++++
 labgrid/driver/ubootwriterdriver.py   | 160 ++++++
 labgrid/driver/usbhidrelay.py         |   7 +-
 labgrid/driver/usbloader.py           | 170 ++++++-
 labgrid/driver/usbstoragedriver.py    |  18 +-
 labgrid/factory.py                    |   4 +-
 labgrid/protocol/__init__.py          |   1 +
 labgrid/protocol/bootstrapprotocol.py |   8 +-
 labgrid/protocol/recoveryprotocol.py  |  14 +
 labgrid/protocol/resetprotocol.py     |  12 +
 labgrid/pytestplugin/fixtures.py      |  21 +-
 labgrid/pytestplugin/hooks.py         |   8 +
 labgrid/remote/client.py              | 351 ++++++++-----
 labgrid/remote/config.py              |   7 +-
 labgrid/remote/exporter.py            | 122 ++++-
 labgrid/resource/__init__.py          |   6 +
 labgrid/resource/remote.py            |  24 +
 labgrid/resource/servo.py             | 485 ++++++++++++++++++
 labgrid/resource/sfemulator.py        |  33 ++
 labgrid/resource/suggest.py           |   6 +
 labgrid/resource/udev.py              |  33 ++
 labgrid/strategy/ubootstrategy.py     | 151 +++++-
 labgrid/target.py                     | 120 ++++-
 labgrid/util/helper.py                | 220 ++++----
 labgrid/util/ssh.py                   |   3 +-
 labgrid/util/term.py                  | 184 +++++++
 labgrid/var_dict.py                   |   8 +
 man/labgrid-client.1                  |   6 +
 man/labgrid-client.rst                |   4 +
 man/labgrid-device-config.5           |   4 +
 56 files changed, 4267 insertions(+), 321 deletions(-)

[1] https://github.com/labgrid-project/labgrid/issues/created_by/sjg20

Signed-off-by: Simon Glass <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants