samd51: MicroPython on the Seeed Wio Terminal

[scruss]

Thanks to robert-hh's amazing efforts — made all the more amazing when you realize he hasn't got a Wio Terminal to test this on himself — MicroPython works quite well on the Wio Terminal.

In order to get folks started, I wrote a tutorial that has working examples for all operating sensors and outputs on that board: MicroPython on the Seeed Studio Wio Terminal: it works! – We Saw a Chicken …

There are also shorter code snippets here: scruss/WioTerminal-MicroPython: short MicroPython examples for the Seeed Wio Terminal

Huge thanks to Robert for making this port work so well.

[davefes]

Just read through the blurb on the website ... does the WiFi work on Micropython?

[scruss]

No; it only ever worked under Arduino. There is a small amount of hope that it might eventually, though:

1. Realtek developers have expressed interest in supporting MicroPython on and with their chips;
2. The Seeed Studio API has been changed to expose an API compatible with the ESP32 firmware used in other boards and other vendors (perhaps the same as the Nina wifi interface that's on the Arduino RP2040 Connect)

Both routes would take considerable development time, so I'm not expecting it to happen soon.

[robert-hh]

@scruss Thank you for the tutorial. It looks good and reads well. Two remarks:

• RTC is enabled on all boards with a 32kHz crystal. So it is enabled on the WIO terminal, and can be enabled on other boards.
• The flash chip can be used for a file system using @peterhinch 's flash driver at https://github.com/peterhinch/micropython_eeprom, similar to using the SD card with the sdcard.py driver. The SPI device for the flash should be 5. The flash chip is connected to the QSPI interface, so access is only by SoftSPI. One topic on my list is to use a board's external flash for the default files system. That would especially be useful for the SAM21 port, which is at the limit of the flash capacity. But that would end up in a more differentiated board definition files.
• WiFi is not (yet) supported. The Seeed Wio terminal is the only board I know with WiFi support. I did not look into the capabilities of that module, but in any case it will require quite a lot of flash space. I do not know if it will fit.

Edit:
While reading through your tutorial: When I saw the connections of the Touch device I thought: Hey, that looks like a XT2046 interface. But according to the schematics it is directly connected to the MCU, and the only AF functions available at these pins are thos from the SAMD touch module, which is NOT DOCUMENTED, at least not at a level that one could write software for. If you want to integrate it, you have to use a closed software blob provided by Atmel/Microchip, which in turn requires to use their software development environment, which is the extreme opposite to their excellent data sheets. For the SAMD21, someone (adafruit?) reverse engineered the code and made a useful API for the touch module. For the SAMD51, this is still due.

[robert-hh]

The mount script should better be:

import os
from machine import SoftSPI, Pin
from flashbdev import FlashBdev
wp = Pin("PA10", Pin.OUT, value=1)
hold = Pin("PA11", Pin.OUT, value=1)
cs = Pin("PB11", Pin.OUT, value=1)
spi = SoftSPI(sck=Pin("PB10"), mosi=Pin("PA08"), miso=Pin("PA09"), baudrate=1_000_000)

flash=FlashBdev(spi, cs)
try:
    vfs = os.VfsLfs1(flash, progsize=256)
except OSError as e:
    print("Mount failed with error", e)
    print("Recreate the file system")
    os.VfsLfs1.mkfs(flash, progsize=256)
    vfs = os.VfsLfs1(flash, progsize=256)

os.mount(vfs, "/flash")

Better use LFS1. LFS2 is pretty slow with SoftSPI, because if make gazillion small reads, when parsing through the directory structure. And if you mount the flash in boot.py, it is visible for mpremote.py.

[robert-hh]

The images in https://github.com/robert-hh/Shared-Stuff/tree/master/samd_firmware now contains a setting for maximum Baudrate with SoftSPI. You have to set baudrate to at least 1_000_000. Then, flash file access is substantially faster.

[peterhinch]

As a matter of interest did you compare the RAM consumption of your driver with mine? I'd be interested to know how big a saving you've achieved.

[robert-hh]

Actually no. Until now I just looked at the mpy file sizes. which is 1.3k vs. 3.7k. If the driver is loaded to RAM, that would bea difference. But the data RAM usage difference should be at least that of the 4k cache buffer, which is allocated by your script and makes it faster. Besides that, my driver only allocates a few bytes for local variables and the address message buffer (4-5 bytes). These size differences are important for the samd21 port with it's ~20k heap and less important for the samd51 port.

[peterhinch]

That's a substantial improvement in code size. I'm unconvinced about the buffer. Your ioctl returns a block size of 4096 which surely forces the littlefs driver to allocate a 4KiB RAM buffer. It seems to me that the buffer is a logical necessity because of the way Flash works.

Taking a top-down view, say you open a file in append mode, then every so often add a byte. The hardware only lets you erase 4KiB sectors. To avoid endless slow (and damaging) erases you must buffer those bytes in RAM until you have a sector full. This seems true regardless of how the device and filesystem drivers are coded.

If I'm missing something, please do tell!

[robert-hh]

So I made a comparison, loading both drivers into RAM from the .py files and mounting the device, using gc.collect();gc.mem_free() before and after the load & mount.
bdevice.py + flash_spi.py: difference 12160 bytes
flashbdev.py: difference 4720 bytes
So the difference between the two is 7440 bytes or about 3k + 4k. 3k for the mpy file size difference, and 4 k for the additional cache buffer in your driver.

[peterhinch]

Thank you. That is a big improvement.

I'm still baffled by the buffering. My code has a 4K buffer with 0.5K block size. Yours has no buffer and a 4K block size. On my reckoning the difference should be 0.5K. I'm clearly wrong.

Ff littlefs doesn't create a 4K buffer, what happens in code like this:

with open("file", "a") as f:
    f.write("a")
    time.sleep(0.1)

The bytes must be accumulating somewhere until 4K of them have been sent. Then the chip's sector is erased and the 4K written out. Perhaps the buffer is created dynamically by extending a list? If so, this could be tested by writing 4K - 1 bytes and checking RAM.

If I'm talking nonsense please explain because I'm foxed :)

[robert-hh]

LFS1 seems to buffer data as give by the progsize parameter in the vfs constructor.
Maybe even stranger, with your flash_spi.py I have to set block_size to 12. Otherwise, LFS1 will not work properly.

[robert-hh]

There is a difference between LFS1 and LFS2.

• When with LFS2 the progsize parameter is omitted and the default of 32 is used, both lfs_test.py and my test of writing large buffers succeed, not matter which file flash driver with it's default blocksize parameter is used. When progsize is set to 256, using a blocksize of 512 fails with your driver. I did not try other value combinations.
• When with LFS1 the progsize parameter is omitted, your driver works, but the flashbdev.py driver refuses to write files >= 4096 bytes. With a progsize of e.g. 256, both drivers work. And the blocksize returned by my driver must be 4096, the physical sector size.

[peterhinch]

Thank you, that explains my results. I never tested with LFS1 - it's not clear to me what is its use-case. Nor did I experiment with progsize. Are there any docs on the readsize, progsize and lookahead args?

Unless there is a case for supporting LFS1 (nobody has requested it) my feeling is that I should just update the docs to specify supported test conditions.

[robert-hh]

The SAMD21 port only supports LFS1. Enabling LFS2 increased the build size by ~16kByte, and flash is scarce at the SAMD21. The SAMD51 variant supports LFS2 and FAT as well. LFS1 appeared to me a little bit faster in reading a directory, at the cost of worse flash utilization.
The only documentation about the readsize, progsize and lookahead args seems to be here: https://github.com/littlefs-project/littlefs/blob/master/lfs.h. More details about the implementation are here: https://github.com/littlefs-project/littlefs/blob/master/DESIGN.md
I noticed that for LFS1 the ratio blocksize/progsize must not be larger than 32, but I did not find it mentioned in the documentation. So with the 4k blocksize progsize must not be smaller than 128. Surely some implementation detail.

[peterhinch]

I think the way forward is for me to recommend LFS2 to users of my driver. For users needing a minimal flash size, I will recommend your driver with LFS1. Do you agree?

I have added a note in this para of my docs making this observation and pointing to your solution.

[robert-hh]

Agreed. With samd21, LFS1 is anyhow the only option. If the both file systems are present, LFS2 should be used with every driver.

[ThinkTransit]

Quick question regarding the mount script, should the flash be mounted to "/" instead of "/flash"?

When mounting at "/flash" I have to install libraries in "/flash/lib" and import using

from flash.lib import mylib

which doesn't seem correct, but I'm probably missing something!

[robert-hh]

The internal flash file system is already mounted at "/", so an additional file system has to use another path. What is the value of your sys.path?

import sys
print(sys.path)

[ThinkTransit]

>>> import sys
>>> print(sys.path)
['', '.frozen']
>>> 

I'm using the nightly firmware from the mp website, nothing custom.

[robert-hh]

You have to add /lib to sys,path, the files in /lib are found by import:

import sys
sys.path.append("/lib")

[ThinkTransit]

Thanks @robert-hh too easy!

[ThinkTransit]

The ili9341 micropython driver is a bit slow, would it be possible to build a c driver like https://github.com/russhughes/ili9342c_mpy into the firmware? I noticed it has been developed/tested on an esp32 but I assume if the code is c it could be also used on the samd51?

[robert-hh]

@ThinkTransit Did you check the Python ILI9341 driver with the other baudrate as well for speed? And there is as well a much simpler IL9341 driver using the framebuffer by @peterhinch https://github.com/peterhinch/micropython-nano-gui/tree/master/drivers/ili93xx

[russhughes]

I saw the same results using the driver https://github.com/rdagger/micropython-ili9341. While https://microchipsupport.force.com/s/article/SPI-max-clock-frequency-in-SAMD-SAMR-devices specifically references the samd21 I suspect this limit applies to the samd51 as well.

[ThinkTransit]

@robert-hh Yes I tested Peter's python driver @24_000_000 and it works fine.

I have micro-gui running on the WIO now which is pretty cool. The only issue I have is that it complains about complex numbers not being supported. Do you know if this is an architecture limitation or is there work required to get complex numbers working?

https://www.youtube.com/watch?v=3nsCb7HmfaE&ab_channel=DunnyRoll

[robert-hh]

Complex numbers are simply not enabled in the config options. See mcu/samd51/mpconfigmcu.h, where the options are set to 0 to make it obvious. Can be enabled.

[ThinkTransit]

Thanks @robert-hh that worked!

[ThinkTransit]

@scruss great article

One trap for new players is that if the directory "SD" doesn't exist then the SD card will not mount and report "SD card not found", could be good to add a note in the article to create "SD" or catch the OSError: [Errno 1] EPERM error.

[scruss]

I have never needed to create the directory: os.mount(sd, "/SD") appears to do it for me. Are you sure this is necessary? It is under Linux, but there's no OS to worry about on a SAMD51 board

[robert-hh]

os.mount() creates the entry for the mount point.

[ThinkTransit]

Apologies @scruss It must have been a coincidence that it failed and then worked after I created the directory!

[robert-hh]

As an exercise, I made a C version of the raw flaw driver class in flashbdev.py. It's a little bit smaller than the .py version, and when operating with a hard SPI interface, file accesses are more snappy, especially the short ones. With SoftSPI, there is no noticeable difference. As next step I plan to add a QSPI flash driver, which gives hardware speed to the flash of SAMD51 devices like the WIO terminal or Adafruit SAMD51 boards. The branch is here: https://github.com/robert-hh/micropython/tree/samd_spiflash

[robert-hh]

The branch at https://github.com/robert-hh/micropython/tree/samd_spiflash is now extended by a C-Code QSPI flash class in machine. It is based on a Adafruit QSPI driver. That speeds up flash access on SAMD51 devices with QSPI Flash substantially. It uses the same small flashbdev.py driver as the SPI flash class. If you build the firmware yourself, this driver is included in the frozen bytecode. Otherwise, it's below.

SECTOR_SIZE = const(4096)


class FlashBdev:
    def __init__(self, flash, sectorsize=SECTOR_SIZE):
        self.flash = flash
        self.sectorsize = sectorsize

    def readblocks(self, n, buf, offset=0):
        self.flash.readblock(n * self.sectorsize + offset, buf)

    def writeblocks(self, n, buf, offset=0):
        if offset == 0:
            self.flash.erase(n * self.sectorsize)
        self.flash.writeblock(n * self.sectorsize + offset, buf)

    def ioctl(self, op, arg):
        if op == 4:  # MP_BLOCKDEV_IOCTL_BLOCK_COUNT
            return self.flash.getsize() // self.sectorsize
        if op == 5:  # MP_BLOCKDEV_IOCTL_BLOCK_SIZE
            return self.sectorsize
        if op == 6:  # MP_BLOCKDEV_IOCTL_BLOCK_ERASE
            self.flash.erase(arg * self.sectorsize)
            return 0

When I was almost finished, I found Damiens spiflash.c driver in drivers/memory. It is used by the STM32 port. Not sure how to use that. It would need an interface for the hardware and to the Python API. The new SAMD drivers mostly of these interface parts, and the code in /drivers/memory would add a large layer in between.

[robert-hh]

I renamed the samd_spiflash branch into samd_flash. The commits are rearranged to have fewer of them, and the last commit changes the startup code _boot.py to use the board's flash drive as default drive. With SAMD51 QSPIflash, that increase the CPU speed as well. My test code pystone_lowmem.py runs about two times faster. With SAM21 devices there is no speed gain, but at least a larger file system. I moved the additional flash drivers from machine into samd. It still may be optimized. Having three different flash drivers does not look nice.