Update for latest Zephyr version (ESF-149)

[marekmatej]

This provides multiple fixes to make the Zephyr example work on the ESP32 boards with uart1 available.

Related PRs:

• Fix esp-serial-flasher build by renaming HAL files zephyrproject-rtos/hal_espressif#314 - fix filename conflicts between HAL and flasher sources.
• boards: espressif: Add uart1 to esp_wrover_kit zephyrproject-rtos/zephyr#77073 - extends the flasher support on the esp_wrover_kit.

[Dzarda7]

Hi @marekmatej, thank you for the fix. LGTM. Will wait for the merges to test it properly. Just a nitpick, can you please change the commit message? We use conventional commits which is checked by CI. Something like fix: Make changes for newest Zephyr? Thanks.

@DNedic PTAL.

[DNedic]

Lots of lines have been just reformatted to remove indentation, can we revert these?

[marekmatej]

yeah, so the order of the statements was a bit strange, and also I didn't like the unnecessary indentation so I fix it according to my taste. However, I can revert all that eye sugar if you don't like it :-)

[DNedic]

I don't have a strong opinion on CMake formatting, but it would be more consistent with the rest of esp-serial-flasher to leave it as is, and also make the diffs smaller and git blame more meaningful.

[DNedic]

Could you explain the reasoning behind removing the board dt overlay? I'm a bit confused on this part.

[marekmatej]

Sometimes there are too many boards overlay that are copy and paste stuff. For those cases, there is the socs folder where you can put all overlays/conf common on the SoC level. This was the exact case.

[DNedic]

Hi @marekmatej , nice work and thanks for helping out!

[DNedic]

Also, I would use the feat(zephyr): format for the conventional commit here and specify the Zephyr SDK version in the commit message.

[Dzarda7]

Thanks for taking a look @DNedic, this should also fix the small Flash size for the images, that's why I wanted it to be fix, but I agree with the feat. Just so you understand.

[Dzarda7]

@marekmatej can you please revert the CMakeLists changes? After that, I am going to merge it. I tested it and it works with minor fix not related to this PR. Thank you so much for your help.

[Dzarda7]

LGTM, thanks.

[beriberikix]

Which version of Zephyr was this tested against? I'm testing against main (d4b7bf9) and it doesn't compile. Probably an easy fix but I'm getting example_common.c:22:10: fatal error: esp_loader_io.h: No such file or director.

[marekmatej]

Which version of Zephyr was this tested against? I'm testing against main (d4b7bf9) and it doesn't compile. Probably an easy fix but I'm getting example_common.c:22:10: fatal error: esp_loader_io.h: No such file or director.

Hi @beriberikix, the fix used the upstream version as well, but that was weeks ago. Try to find esp_loader.h and rename it to esp_loader_io.h. It seems there was further renaming of the file.

[igrr]

Try to find esp_loader.h and rename it to esp_loader_io.h

But both files exist in the same directory: https://github.com/espressif/esp-serial-flasher/tree/master/include...

[Dzarda7]

@beriberikix I am still able to build the zephyr example with this fix. I do not know much about zephyr so I am following the CI build process which is described here. I just use newest releases of all tools and master branch of zephyr. The build should be successful with this.

You also need to remove fflush(stdout); line here, mcu will throw panic error during runtime otherwise. I also needed to change the pins as the present ones are used for flash memory connection in ESP32-DevKitC V4.

Sorry for inconvenience, these changes will hopefully be merged soon, we are waiting for next zephyr release so we can mark it as supported. Thanks for understanding, I already proposed that tested versions should be stated with all ports.

If you have some troubles, let me know.

[beriberikix]

Totally understandable! I retried with a clean Zephyr environment and got the same issue. If I explicitly add the include and port directories to CMake, I get past the missing file errors:

zephyr_library_include_directories(
  ../common
  ../binaries
  ../../include
  ../../port
  )

However, linking fails because of undefined references. Ex.

/home/jberi/toolchains/zephyr-sdk-0.17.0/xtensa-espressif_esp32_zephyr-elf/bin/../lib/gcc/xtensa-espressif_esp32_zephyr-elf/12.2.0/../../../../xtensa-espressif_esp32_zephyr-elf/bin/ld.bfd: app/libapp.a(example_common.c.obj):(.literal.connect_to_target+0x1c): undefined reference to `esp_loader_connect'
/home/jberi/toolchains/zephyr-sdk-0.17.0/xtensa-espressif_esp32_zephyr-elf/bin/../lib/gcc/xtensa-espressif_esp32_zephyr-elf/12.2.0/../../../../xtensa-espressif_esp32_zephyr-elf/bin/ld.bfd: app/libapp.a(example_common.c.obj):(.literal.connect_to_target+0x20): undefined reference to `esp_loader_get_target'
...

Any ideas?

[Dzarda7]

Did you clone the this repo and checked out correct branch? If so, you should be able to build the example using following commands:

cd esp-serial-flasher
mkdir zephyrproject-rtos
cd zephyrproject-rtos
git clone https://github.com/zephyrproject-rtos/zephyr.git
mkdir zephyr-sdk
cd zephyr-sdk
wget --no-verbose -O zephyr_sdk.tar.xz https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.17.0/zephyr-sdk-0.17.0_linux-aarch64_minimal.tar.xz
tar xvf zephyr_sdk.tar.xz --strip-components=1
wget --no-verbose -O esp32_toolchain.tar.xz https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.3/toolchain_linux-x86_64_xtensa-espressif_esp32_zephyr-elf.tar.xz
tar xvf esp32_toolchain.tar.xz
export ZEPHYR_SDK_INSTALL_DIR=$(pwd)
export ZEPHYR_TOOLCHAIN_VARIANT="zephyr"
cd ../zephyr
export ZEPHYR_BASE=$(pwd)
pip install -r scripts/requirements.txt
pip install wheel
west init -l .
unset IDF_PATH
west zephyr-export
west update hal_espressif
west build -p -b esp32_devkitc_wroom <<ABSOLUTE_PATH_TO_zephyr_example_FOLDER>> -DZEPHYR_EXTRA_MODULES=<<ABSOLUTE_PATH_TO_esp-serial-flasher_FOLDER>>


example of last command: west build -p -b esp32_devkitc_wroom /home/jarda/Documents/Espressif/zephyr_test/esp-serial-flasher/examples/zephyr_example -DZEPHYR_EXTRA_MODULES=/home/jarda/Documents/Espressif/zephyr_test/esp-serial-flasher

I am able to build it using these commands.

[beriberikix]

Hmm...yes, and just checked again. I didn't pass in the ZEPHYR_EXTRA_MODULES originally by that didn't make a difference.

[beriberikix]

Actually, I'm not sure what's the issue with my machine but I was able to build using Docker. Here's the Dockerfile I pulled together:

FROM ghcr.io/embeddedcontainers/zephyr:xtensa-espressif_esp32-0.17.0SDK

WORKDIR /workdir

RUN \
  git clone https://github.com/marekmatej/esp-serial-flasher \
  && cd esp-serial-flasher \
  && git checkout bugfix/zephyr_port_example \
  && cd ..

RUN \
  west init \
  && west update --narrow -o=--depth=1 \
  && pip3 install -r zephyr/scripts/requirements.txt \
  && west zephyr-export \
  && west update hal_espressif

# docker build -t esf-zephyr-esp:latest -f ./Dockerfile .
# docker run --rm -it -v ./zephyr-build:/workdir/build esf-zephyr-esp:latest
# west build -p -b esp32_devkitc_wroom /workdir/esp-serial-flasher/examples/zephyr_example -DZEPHYR_EXTRA_MODULES=/workdir/esp-serial-flasher

[Dzarda7]

That is interesting. Do not you have exported some variable that messes up with the zephyr build? For example the IDF_PATH? Someone has ESP-IDF export script in .bashrc which might cause some issues.