Embassy HAL implementation for Espressif chips

[MabezDev]

We're looking to implement the required traits/executors for Espressif chips, do you have any pointers on what we'll need to implement in our HAL's?

So far it looks like we'll need:

• Embassy executor for RISCV arch. - Preliminary RISCV support #804
• Embassy executor for Xtensa arch - Preliminary Xtensa support #805 .
• Add embedded-hal alpha traits.
• Add at least one embassy-time::Driver implementation for each chip
• Implement embedded-hal-async where appropriate.
• Add an InterruptExecutor for each chip

[lulf]

Great to see the interested of running embassy on espressif! 🚀

We're looking to implement the required traits/executors for Espressif chips, do you have any pointers on what we'll need to implement in our HAL's?

So far it looks like we'll need:

* Embassy executor for RISCV arch.

* Embassy executor for Xtensa arch.

👍 ! The 'core' of the executor is arch-independent, but the the different arch-dependent executor parts can be found here:

https://github.com/embassy-rs/embassy/tree/master/embassy/src/executor/arch

The cortex_m one might be the most relevant to look at.

* Add embedded-hal alpha traits.

* Implement embedded-hal-async where appropriate.

In addition to the embedded-hal(-async) traits, embassy has the concept of an opt-in 'timer driver' system timer that is implemented for most HALs, which makes async timers just work. For instance:

https://github.com/embassy-rs/embassy/blob/master/embassy/src/time/driver.rs

and an implementation:
https://github.com/embassy-rs/embassy/blob/master/embassy-nrf/src/time_driver.rs#L122-L126

There is also the critical-section crate that must have implementations for the target architecture (https://crates.io/crates/critical-section). Risc V seems to have some support there already.

There is also some hardware specific stuff in the macros that's needed for users to use the #[embassy::main] macro:

https://github.com/embassy-rs/embassy/blob/master/embassy-macros/src/macros/main.rs#L8-L15

[MabezDev]

Thanks for the pointers!

For HAL peripheral implementations I also saw the Unborrow trait used sporadically, I understand what it's doing but I don't know why it's needed? Could you point me in the direction of some docs to explain the requirement of this trait and when to use it?

[Dirbaio]

It's to allow creating drivers with &mut singletons. It's very handy when you want to create a driver temporarily, for example for low power. It avoids moving the singletons into the driver and then back out with some .free() method other HALs have, which usually requires Options and unwraps, which is annoying and error-prone.

(It has nothing to do with async, it's a design decision the embassy HALs make, you can do different design decisions if you like)

[osobiehl]

Hi everyone,
I've been trying to port embassy to the esp32c3 for a couple of weeks now, I have a VERY rough and patchy port working here that awaits a timer and prints hello, world!, among with some other small tests.

I made it by stitching together the assembler code responsible for setting up the interrupt table from [esp-hal] (https://github.com/esp-rs/esp-hal) , as well as using some of the existing traits / implementations from embedded-hal to write to the UART0 peripheral.

Some of what I have working is:

• thread mode executor using a global flag since riscv doesn't have something similar to cortex_m's asm::sev (thanks @Dirbaio for the tip)
• time driver using SYSTIMER peripheral, (requires adding 16 MHz to clock frequencies in embassy) allowing allocation of up to 3 alarms (We probably shouldn't use all 3 in case some other peripheral requires a systimer irq, but that's outside my scope). The peripheral uses a 52-bit timer so it should be low-risk in terms of overflowing

Interrupts are handled differently on the ESP32C3 board, I'm not sure if the same interrupt controller is used for other ESP boards, either way, another area that would need porting is interrupts, this would require

• Removing the blanket implementation for InterruptExt for the Interrupt trait, or at least defining an interrupt_esp32xx.rs without a blanket implementation and using conditional compilation to use that when targeting ESP boards.
• Finding a nice way to declare CPU interrupts on the ESP32C3 board without having clashes, since there are more interrupt sources than potential interrupts, the approach is not as clear-cut as cortex_m's NVIC, and requires a lot more manual work in terms of enabling and disabling interrupts. I just piggybacked off the interrupt module in esp32-hal and it seems to work reasonably well.

Finally, one last thing to do would be to have compatibility with embassy_macros. I kind of got them working to use the #[riscv-rt::entry] macro for the esp board, but the compiler seems to complain when it comes to cfg attributes in procedural macros. I also don't have any other board than the ESP32C3 so I haven't really been doing compatibility checks.

[MabezDev]

This is awesome! Thanks @osobiehl, I'll take a in-depth look soon!

Interrupts are handled differently on the ESP32C3 board, I'm not sure if the same interrupt controller is used for other ESP boards, either way, another area that would need porting is interrupts, this would require

Yes the c3 uses a custom interrupt controller, and the next few chips will too. Longer-term I believe we will integrate the standard PLIC. Perhaps we can find an abstraction that supports PLIC & custom interrupt controllers (I know there are other RISCV chips out there with them) without special behaviour inside embassy.

[MabezDev]

I've just opened #804 based on your work @osobiehl, thank you for laying the groundwork! I've also been working on the esp-hal side, including a SystemTimer implementation in esp-rs/esp-hal#76. My current embassy dev branch is here.

[jessebraham]

I've opened a PR in esp-hal implementing a number of the embedded-hal alpha traits: esp-rs/esp-hal#82

Edit: this has now been merged.

[jessebraham]

In case anybody wants to follow progress, I've created some tracking issues in esp-hal regarding the Trait implementations:

embedded-hal alpha traits: esp-rs/esp-hal#69
embedded-hal-async traits: esp-rs/esp-hal#70

[markfarnan]

How close is this to useable ? Especially on the Xtensa ESP32 base device (not S/C etc)
Failing that, the C3

[jessebraham]

@markfarnan as things stand currently:

• We have have time driver implementations for all currently supported chips (ESP32, ESP32-C2/C3, ESP32-S2/S3) using either TIMG as the timer source or SYSTIMER (with the exception of the ESP32, as is does not have this peripheral). Timer source is selectable via features.
• We do not presently have async peripheral drivers, however implementing these is high on our priority list and should begin in the coming weeks. Hopefully this will not take too long, I would expect these to be done in Q1, and I will be sure to update the tracking issue linked above as traits are implemented.

[MabezDev]

A quick update 6~ weeks on:

We have two async drivers in esp-hal:

• Async: GPIO esp-rs/esp-hal#333
• Async SPI esp-rs/esp-hal#385

and we also have two forms of async networking through esp-wifi

• Async WiFi esp-rs/esp-wifi-sys#125 - async wifi support with embassy_net
• Add esp-now support esp-rs/esp-wifi-sys#121 - async esp-now, esp-now is a peer-to-peer communication protocol

[TheButlah]

@Dirbaio embassy switched from using atomic_polyfill in #1125, which breaks riscv. The esp32c3 does not support atomics, and AtomicBool and other atomics don't exist on the platform and do not exist in core.

Some further context from this discussion:

The difference between the riscv32imc-* and the thumbv6m-* rustc compilation targets is that the RISCV one has broken LLVM codegen and that's why it doesn't have the Atomic* API available at all.

In other words, even though embassy doesn't require CAS, rustc doesn't emit even the types themselves.

error[E0432]: unresolved import `core::sync::atomic::AtomicBool`
 --> /home/ryan/.cargo/git/checkouts/embassy-9312dcb0ed774b29/26474ce/embassy-executor/src/arch/riscv32.rs:3:26
  |
3 | use core::sync::atomic::{AtomicBool, Ordering};
  |                          ^^^^^^^^^^ no `AtomicBool` in `sync::atomic`
  |
help: consider importing this struct instead
  |
3 | use core::sync::atomic::{atomic_polyfill::AtomicBool;
  |                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For more information about this error, try `rustc --explain E0432`.
error: could not compile `embassy-executor` due to previous error

Could #1125 be reverted to restore embassy support to esp32c3?

Side note: Could someone more knowledgeable on the subject open an issue in rustc to get the codegen working for this and link to the issue?

[MabezDev]

I don't think anything needs to be changed in embassy. We have atomic emulation in esp-hal for the esp32c3 and esp32c2 enabled by default. You just need to add the following rustflags to your project from the riscv-atomic-emulation-trap handler: https://github.com/esp-rs/riscv-atomic-emulation-trap#usage.

Atomic load/stores on RISCV are implemented as a normal load/store (essentially zero cost) and I believe CAS will actually be handled by atomic-polyfill where possible because it is set to polyfill for any riscv target: https://github.com/embassy-rs/atomic-polyfill/blob/80cee72cf0e583e6fccefa8ee463d66ad9aa05d8/build.rs#L31. To ensure that is the case, you could remove the CAS related cfg's from the rustflags list above.

[TheButlah]

doesn't this require trap handlers get taken on any load or store on an atomic? I would assume that is bad for performance.
So far I have been able to avoid the use of imac.

If we don't need CAS, but we do need atomic loads/stores, is using the trap handler really the right approach?

Disclaimer: I have no experience with atomics at the assembly level or codegen. I'm asking about this because I am trying to understand and learn, and am trying to understand what the performance tradeoffs here are.

[Dirbaio]

#1125 removed atomic_polyfill for uses that were only load/store (no CAS). The esp32c3 hardware (riscv32imc) is capable of doing that.

It is (IMO) a rust bug that load/store atomics aren't exposed in riscv32imc. They are exposed in thumbv6m, which is the same case (can do load/store but no CAS). See rust-lang/rust#99668

That's what should be fixed, instead of adding workarounds to Embassy.

[TheButlah]

Awesome, thanks for the added context :)
I think I misunderstood mabez's message - I thought he meant this would require me to use the trap handler, but I see now that I can just pass the rustflags for Load/Store.

[proegssilb]

I'm a little confused by current status.

• Are we still waiting for embedded-hal-async implementations? Or are those entirely complete?
• Interrupts are still basically not supported at this time, correct?

I'm looking at maybe switching from CircuitPython to Embassy for broader hardware support. In order for the port to make sense, I'd need BLE, i2c, i2s, spi/qspi, and gpio (preferably with interrupts) already working across nrf52840, esp32-s3, and esp32-c6 (as implemented in the Adafruit Feather boards, for the nrf and the s3).

If there's anything I can do to help out, I'm all ears, but I'll be the first to admit limited expertise with low-level hardware work.

[MabezDev]

All the eha traits are now implemented (I've updated the check list). The remaining async drivers can be tracked here: esp-rs/esp-hal#361. From your list async i2c,spi,gpio and BLE are already available. Async I2S is being worked on.

[markfarnan]

Does this make Embassy now useable, (for the peripherals which are done), or are there other blocking issues ?

…

On Wed, Sep 20, 2023 at 2:24 PM Scott Mabin ***@***.***> wrote: All the eha traits are now implemented (I've updated the check list). The remaining async drivers can be tracked here: esp-rs/esp-hal#361 <esp-rs/esp-hal#361>. From your list async i2c,spi,gpio and BLE are already available. Async I2S is being worked on. — Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFRFHIAL2NZV7JNEGOVGBIDX3LN6PANCNFSM5U3UEYFA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[MabezDev]

Does this make Embassy now useable, (for the peripherals which are done),
or are there other blocking issues ?

Yes, its been more than usable since February this year :)

[proegssilb]

Thanks for the update, MabezDev. I noticed after the update, interrupts still remain unchecked, and your message didn't mention them. Just for clarity, do you know if interrupts are still a work in progress?

[MabezDev]

An interrupt executor != interrupts (See the note about them here: https://embassy.dev/book/dev/runtime.html). Interrupts are essentially a requirements for async, and have been supported in esp-hal since esp-rs/esp-hal#118 & esp-rs/esp-hal#103.

Thanks to @bugadani we do actually have an interrupt executor inside esp-hal, but currently it's only for our Xtensa chips at the moment, which is why I haven't checked it off.

[markfarnan]

Is there a guide to using Embassy with ESP32 ? (It's not listed on in the Embassy main guide, less I'm blind)

…

On Wed, Sep 20, 2023 at 5:45 PM Scott Mabin ***@***.***> wrote: An interrupt executor != interrupts (See the note about them here: https://embassy.dev/book/dev/runtime.html). Interrupts are essentially a requirements for async, and have been supported in esp-hal since esp-rs/esp-hal#118 <esp-rs/esp-hal#118> & esp-rs/esp-hal#103 <esp-rs/esp-hal#103>. Thanks to @bugadani <https://github.com/bugadani> we do actually have an interrupt executor inside esp-hal, but currently it's only for our Xtensa chips at the moment, which is why I haven't checked it off. — Reply to this email directly, view it on GitHub <#745 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFRFHIGLP4BTL4Y4OB7G6K3X3MFSPANCNFSM5U3UEYFA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[TheButlah]

Check https://github.com/esp-rs/awesome-esp-rust for general esp32 rust stuff, and if you want a more complex firmware that uses it with embassy, you can reference https://github.com/SlimeVR/SlimeVR-Rust/tree/main/firmware.

The best source of information though is the example code in esp-hal: https://github.com/esp-rs/esp-hal/blob/main/esp32c3-hal/examples/embassy_hello_world.rs

[MabezDev]

I think we can consider this done, we have excellent embassy support in esp-hal including but not limited to:

• An Interrupt executor
• Thread mode, multicore aware, executor
• Multiple time-driver implementations
• Many async drivers capable of running on any executor, including embassy-executor
• async networking with esp-wifi using embassy-net
• async Bluetooth using the bleps stack, and soon trouble support too.

Thanks to everyone involved in making this possible! ❤️