Upgrade to ESP32 - Wemos S2/C3 mini and S3

[CheeseE]

Question
Has anyone tried this or this board to upgrade the old ESP8266 to ESP32? Is it pin-to-pin compatible with the D1 mini? It seems to me but it would be good for someone to confirm, thanks.

[proddy]

The esp32-s2 is a different chip so the code needs to compiled. I've haven't tried it myself.

https://docs.platformio.org/en/stable/boards/espressif32/lolin_s2_mini.html

[bbqkees]

It's the easiest to just get a MH-ET LIVE D1 Mini with ESP32 chip.
Pin compatible with the ESP8266 D1 Mini and it already has a board profile in the firmware.

[CheeseE]

Thank you, I have the MH-ET, just wanted to know if I have to cut the enclosure, as these mentioned boards are the same size as the D1 mini. Ordered them, and will try them out if it works.

[bbqkees]

The MH-ET is wider so you have to make a few adjustments to the housing. See the wiki:
https://bbqkees-electronics.nl/wiki/gateway/upgrade-the-gateway-to-an-esp32.html

[MichaelDvP]

If you dont want to cut the housing, you can also cut the board and sidepanel:

[CheeseE]

What I'm trying to avoid is to cut the housing. But this is a good idea to cut the board itself, thank you :)

[MichaelDvP]

You also have to remove the plastic spacer from the board pins to make the usb fit without cutting the housing.

[ajvdw]

I did build for S2: there are 3 minor adjustments necessary NO UART2 moved to UART1 needs testing, NO BT skip include esp_bt.h and I modified the rename_fw script to generate a S2 specific bin file. Only problem: the settings are not stored in NVS. It's already mentioned as an issue in 3.4.2. If that part is fixed I can do some testing. I will create an PR if it works.

[CheeseE]

@ajvdw There's already a working version committed for Lolin C3 mini.

[ajvdw]

@ajvdw There's already a working version committed for Lolin C3 mini.

I followed the same path ;-)

I'd like to propose the bin files to follow the names in the PlatformIO Project environment, the rename_fw script would be something like, so no need to mention the flash size separately:

chip_target = env.get('PIOENV').upper()

print("app version: "+app_version)
print("chip_target: "+chip_target)
print("platform: "+platform)
# print("flash size: "+flash_size)

# convert . to _ so Windows doesn't complain
variant = "EMS-ESP-" + app_version.replace(".", "_") + "-" + chip_target 

[proddy]

That's a good recommendation. I had already change the file in the latest flash_optmization branch (which I'll merge into dev soon):

EMS-ESP32/scripts/rename_fw.py

Line 39 in 58a0ec9

variant = "EMS-ESP-" + app_version.replace(".", "_") + "-" + platform

but your change is better. thanks

[MichaelDvP]

Today i've got a C3-Mini and a S2-Mini from lolin store. I'll check memory and issues the next days. S2 has less ram, but PSRAM. I'm curious how this works.

[proddy]

PSRAM

I'm also curious how to use it (there are a few ways described here). I'm also going to start looking at how to measure performance using some kind of profile. There must be something out there that can monitor. The alternative is to build a test into test.cpp which will stress test EMS-ESP by loading data, printing it, publishing it, and repeating for a few minutes, and then logging CPU and memory consumption. There are 2 cores in the ESP32 standard and I don't think we're even using the 2nd one fully.

[MichaelDvP]

C3 is running now with ems, dallas, analog, had some issues with gpio settings. Program size is larger, free ram a bit less then esp32, but working good.
S2 seems to have uart1 fixed on gpio17/18, not pin compatible with D1 mini and no plug in replacement for gateway premium.
psram is standard config as malloc, the dynamic json can use it and i think it is needed (without ems connection <50k free heap).

S2 without ems, dallas, analog connection, program size for single core is smaller:

[MichaelDvP]

S2 uart pins have to be configured before driver loading. Now it works as plugin in gateway premium.
Psram is used without code change, no issues when calling api with large output. All working fast.
Info with ems, dallas, analog connected:

[MichaelDvP]

@proddy To identify the platform in a string at central place i've modified the version.h to

#define EMSESP_APP_VERSION "3.5.0b5"

if defined(ARDUINO_LOLIN_C3_MINI)
#define EMSESP_PLATFORM "ESP32-C3";
#else
#if defined(ARDUINO_LOLIN_S2_MINI)
#define EMSESP_PLATFORM "ESP32-S2";
#else
#define EMSESP_PLATFORM "ESP32";
#endif
#endif

Is this ok, or should i add a platform.h with this defines?

[proddy]

Yes! Nice.

[MichaelDvP]

It's better to check the chip, not the specific board.

#if CONFIG_IDF_TARGET_ESP32C3
#define EMSESP_PLATFORM "ESP32-C3";
#elif CONFIG_IDF_TARGET_ESP32S2
#define EMSESP_PLATFORM "ESP32-S2";
#elif CONFIG_IDF_TARGET_ESP32
#define EMSESP_PLATFORM "ESP32";
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif

The C3-Mini board have a rgb-LED (WS2812B), and can not be switchted by gpio, needs a fast protocoll with ns timing. Adafruit NeoLED lbrary can handle this, but flash is nearly full for ESP32C3. For now i skip LED.

[proddy]

I've ordered an S2 and C3 because I'm missing out on the fun too.

regarding the RGB, I have some old C code from another project years ago that handled the RGB LED on an ESP32, I'll see if I can reuse it.

[ajvdw]

I've done some testing with the S2, I can't change to GPIO 7 to 9 because of the validators. More GPIO's should be probably allowed. My board used GPIO18 and GP19 on the ESP32 on the S2 the GPIO pins 7 and 9 are on that location.... I defined my board profile but I couldn't get it to connect to the EMS bus. Probably because of incorrect initialization? I remember a remark about setting GPIO's before initializing the UART. But I can't find the matching commit with that fix.....

[proddy]

Those validators are easy to adjust for the different ESPs. I'm sure Michael can figure out the GPIOs init. I'm waiting for my S2's to arrive.

[MichaelDvP]

@ajvdw I will push a working dev soon to my repo and make a pr. Here is a working bin from my testings (with some extra debug logs, experimental values, etc.)
EMS-ESP-LOLIN_S2_MINI-3_5_0b5.zip

[ajvdw]

@ajvdw I will push a working dev soon to my repo and make a pr. Here is a working bin from my testings (with some extra debug logs, experimental values, etc.) EMS-ESP-LOLIN_S2_MINI-3_5_0b5.zip

I've tested this version. It seems to work even with. the custom GPIO's set to 7 and 9.

[MichaelDvP]

The source is online now: https://github.com/MichaelDvP/EMS-ESP32/tree/dev_.

[proddy]

Got my S2 Mini and C3 Mini today so will join the fun

[MichaelDvP]

I think you have a configuration error with the two ems-esp, same bus-id, same mqtt-settings, same hostname, or something like that. But without log and system info it's just a look in a crystall ball.

[tp1de]

No different bus-id, s3 without mqtt, different hostname and ip-adresses.
Sorry I can not display system log since it is basically not displayed and/or system reboots in the meantime. Even with just gateway (s3) it does not work. No chance to flash with your flasher?

[MichaelDvP]

I have tried the official build, it crashes every few seconds. Seems i have made a fix in my testbuild, have to check what is was.
(Edit: removed testbuild to avoid confusion)

[MichaelDvP]

Seems to be a stack overflow in UART task, change line 92 of emsuart_esp32.cpp to
xTaskCreate(uart_event_task, "uart_event_task", 2560, NULL, configMAX_PRIORITIES - 1, NULL);
(change stack size from 2k to 2,5k) (my build uses 2,2k)

[tp1de]

@MichaelDvP yes uart stack size was causing the problems.
Now S3 works, I just have to find out why with the S3 version I do have 2 thermostat entities less.

[tp1de]

Under S3 the 2 following thermostat entities are missing: hc1/hc2.haclimate. Anyhow I do have no glue what these entities are used for. Same code for S3 (16MB) and ESP32 (4MB) with new partioning.

[MichaelDvP]

This is a dummy entity to generate the homeassistant climate card depending on thermostat configuration. It should show up in settings customizations, but only has a value if HA is activated.

[tp1de]

This is a dummy entity to generate the homeassistant climate card depending on thermostat configuration. It should show up in settings customizations, but only has a value if HA is activated.

Understood. Everything seems to work now. Just a final question:

while using the S3 I need to compile always by myself - isn't it? (I can't use the bin-files (4MB) provided by @proddy)

[bbqkees]

The current bin file on the repository is specific to the ESP32 architecture, the ESP32-S3 is a different architecture.
Maybe it's an idea to start offering a bin file for the S3 as well so more people can test it easily.

[MichaelDvP]

The esp32, s2, c3, s3 are different chip types and needs different binary files. The platformio.ini sets the right chiptype and partition size.
For now you need to compile yourself, maybe later there will be different binaries for download.
The web upload checks file header and you can not upload a wrong binary and get http:503 service unavailable for wrong binaries.

Set in your pio_local.ini

[platformio]
 default_envs = esp32_16M, lolin_s3

to build both binaries if you hit build button.

[proddy]

I've been running on S3 for a while now, very stable. @MichaelDvP do you think we can close this?

[MichaelDvP]

Yes, S3 is working good, i have 2 gateways connected MH-ET and S3. C3 was also good, but not tested with the latest 3.6. With S2 i remember that memory was a bit critical, but maybe this was also the stack overflow from NTP service. My S2-tests are long time ago. We can reopen for S2 if someone use it an report issues.

[CheeseE]

Before we close, can we add the bin files to the release, so people can use them without compiling the whole stuff?

[bbqkees]

Well maybe now is a good time to start offering a bin file for S3 on the repository.

[bbqkees]

@CheeseE Seems we had the same idea at the same time :-)

[tp1de]

Same for me. Bin files for S3 would be fine. Is there any chance to provide old version bin files in the repository as well?

[MichaelDvP]

Github actions has a issue for S3 with default platform, giving Error: Could not find the package with 'espressif/toolchain-riscv32-esp @ 8.4.0+2021r2-patch3' requirements for your system 'linux_x86_64'
The dev-platform 8.4.0+2021r2-patch5 works, so use this for the dev-builds?
The S3 boardprofile defaults to the lilgo pinout.

[MichaelDvP]

Tested the S2 again with actual v3.6dev.

• Because of low heap mqtt is not working, we should check for PSRAM and heap to make it work again. But with HA enabled still ~20 mqtt errors happen while publishing the config.
• I get some TX-read errors independend of tx-mode. After a tx-read it receives a poll and then the answer, If i ignore this poll when waiting for reply, all is well (zero tx-errors). The liligo S3 also shows this, but not as often as S2.

[proddy]

Tested the S2 again with actual v3.6dev.

• Because of low heap mqtt is not working, we should check for PSRAM and heap to make it work again. But with HA enabled still ~20 mqtt errors happen while publishing the config.

Do you know what is causing the mqtt errors? Is it because of the heap check we added? I've been looking into https://github.com/bertmelis/espMqttClient which is new version of asyncMqttClient from one of the maintainers. It looks more lightweight and we can probably not use async and just send out the messages immediately because of the ESP32's dual core. I'm not sure though.

• I get some TX-read errors independend of tx-mode. After a tx-read it receives a poll and then the answer, If i ignore this poll when waiting for reply, all is well (zero tx-errors). The liligo S3 also shows this, but not as often as S2.

Good catch. We can add that code to ignore it.

Also, do you think it's worth investing time into making EMS-ESP work on an S2?

[MichaelDvP]

Do you know what is causing the mqtt errors?

Yes, mostly. The async-mqtt also checks for free heap (4k) and gives message id 0 back if lower. For the HA messages we can change the mqtt publish functions to bool, checking the message-Id and resend later if not sent.
(Our heap check is 60k and has to be disabled for psram present, otherwise the s2 never sends)

I've been looking into https://github.com/bertmelis/espMqttClient which is new version of asyncMqttClient from one of the maintainers. It looks more lightweight and we can probably not use async and just send out the messages immediately because of the ESP32's dual core. I'm not sure though.

looks nice. Also the current async-mqtt has a queue and processes qos, works also direct sending. The ems-esp queue is only usefull to queue before connected. But the new lib looks good. We should test it.

Also, do you think it's worth investing time into making EMS-ESP work on an S2?

No. But it's a nice test to see what happens if heap gets low and PSRam is active. Should be possible to make it stable.

[CheeseE]

@MichaelDvP could we add C3 binaries as well? I'm using that one for months now without issues.

[MichaelDvP]

The C3 works good, has more ram than S2 and with reduced tx-power buspowernig is possible. The Wemos C3 mini fits in the old gateway premium. It has only 4MB flash, but asymetric partitions with loader work.

But there are different versions of the wemos, v1.0.0 needs a special wifi init and have a single LED, v2.1.0 has a programable RGB-LED. So we need different binaries. I think it's better to compile yourself, or forc the repo and edit the github action to build your version.

@proddy what do you think about adding C3 bnary?

[proddy]

agree to make special instructions on how to adjust for C3

[bbqkees]

I'll add an update to my wiki soon with instruction on selecting the correct bin file.

[MichaelDvP]

Just an update to modules:
Wemos/lolin have introduced a S3 mini https://www.wemos.cc/en/latest/s3/s3_mini.html with 4MB Flash and 2 MB PSRAM. But the lolin store is shut down https://lolin.aliexpress.com/ and the module can not be purcased https://de.aliexpress.com/item/1005005449219195.html Same for the C3 mini https://de.aliexpress.com/item/1005004740051202.html and S2, only S2 module is offered by different sellers on aliexpress. But S2 is to low with ram.
The liligo S3 module is available in liligo shop again https://www.lilygo.cc/products/t7-s3. Price is lower now, but no tax handling by lilgo for EU customers. On ebay the module is still 23-27€ from chinese resellers.

[proddy]

Thanks for sharing. Hopefully bbqkees has enough S3's in stock for his next batch of gateway boards. Then next I'd love to play with the C6 with its fast RISC-V and WiFi6 support.