Issues with v23.9.11 , had to revoke merge #440
Replies: 10 comments 13 replies
-
|
Hi, |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @madmartin wasn't aware of this command. |
Beta Was this translation helpful? Give feedback.
-
|
I found this: Update espressif32 from 6.3.2 to 6.4.0 This results in crashes: Can anybody confirm this? I'm wondering why there are no issues with @tbnobody upstream version |
Beta Was this translation helpful? Give feedback.
-
|
OK, without this upgrading espressif32 the merge works on my system. Will merge it to developement. But I need some feedback from others how this update behave on their system. |
Beta Was this translation helpful? Give feedback.
-
Interesting, this would have been my first guess/try if there would have been no news until this evening. Is it always a similar backtrace? Does this happen repeatedly on your system? Why this error does not occur in the upstream project is probably explained easily: There is no Odd. I hope I can contribute this evening. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @schlimmchen Yes the the backtrace is still the same. What I also see, that the wifi connection becomes unreliable. |
Beta Was this translation helpful? Give feedback.
-
|
Cannot confirm. 🤷♂️ At least not within the first ten minutes of runtime. Your log suggests that this happens after only 12 seconds of runtime on your board. I enabled HA integration but that did not do it either. |
Beta Was this translation helpful? Give feedback.
-
|
Regarding the backtrace... There is multiple things very wrong that make me roll my eyes, but the one thing that might be a real candidate is this: Have a look at
So my best idea right now is that somehow the JSON increased and the serialized JSON is now larger than 512 Bytes. The buffer is then implicitly converted to a There definitely are issues here, but I am not very confident that I found the culprit. Is this reproducible well at your end? Maybe increase those buffers (double them to match the memory pool of the DynamicJsonDocument) and see if that fixes things. (BTW: Doing so is not the proper way to fix the issue(s), and there are more things to do, but it might prevent the crashes you observe, so we can know that this is indeed the issue). |
Beta Was this translation helpful? Give feedback.
-
|
Checked the logs again. It happens also in MqttHandleHassClass |
Beta Was this translation helpful? Give feedback.
-
|
Interesting. What's the memory consumption? |
Beta Was this translation helpful? Give feedback.
-
|
I was not able to login. But my telegraf call to the prometheus api is logged says: I saw one measurment in prometheus for free heap around 62 kByte free heap. Normal is around 98 kB free heap. |
Beta Was this translation helpful? Give feedback.
-
|
I can't seem to find a difference in heap consumption between f7bd4a4 (plus work in progress code from myself) compared to the same workspace with [email protected]. Also, there is 160kB of free heap on my board. It is odd that you usually see only 98kB of free heap through prometheus. Maybe that's because the moment that value is determined that insanely huge buffer is allocated to create the prometheus response, so the heap consumption appears much larger. Otherwise I am quite confused that your heap consumption is so much larger. |
Beta Was this translation helpful? Give feedback.
-
|
I have to invertes.
But even on a nearly unconfigured test device (only network setup), I don't get 160kB
|
Beta Was this translation helpful? Give feedback.
-
|
Interesting. My test device has one inverter configured and still 160kB left. Are there significant differences between the ESP32 and ESP-S3? Also, mine uses 86% of the sketch, not 92% as yours does. |
Beta Was this translation helpful? Give feedback.
-
|
FYI: Commit f7bd4a4 unintentionally (?) reverts JK BMS: Support for MQTT (#432) (24018a1). |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, I will check this. I cherry picked it. Should be back in development. |
Beta Was this translation helpful? Give feedback.
-
|
I nailed it down, that I get different results for serializeJson() Using branch https://github.com/helgeerbe/OpenDTU-OnBattery/tree/espressif_6.4.0 with and with Luna is the second Inverter. Solar ist my first one and is published always correctly. |
Beta Was this translation helpful? Give feedback.
-
|
Interesting! I looked at the code you pushed to the You already increased the |
Beta Was this translation helpful? Give feedback.
-
|
No, as soon you try to publish or print the buffer the system crashes (when it has an odd length). I will try to use Serial instead of MessageOutput |
Beta Was this translation helpful? Give feedback.
-
|
I have no idea how to solve this. This was the simpliest test in the code: void MqttHandleHassClass::publishInverterNumber(
std::shared_ptr<InverterAbstract> inv, const char* caption, const char* icon, const char* category,
const char* commandTopic, const char* stateTopic, const char* unitOfMeasure,
int16_t min, int16_t max)
{
char buffer[2048];
size_t byte = 0;
String serial = inv->serialString();
String buttonId = caption;
buttonId.replace(" ", "_");
buttonId.toLowerCase();
String configTopic = "number/dtu_" + serial
+ "/" + buttonId
+ "/config";
String cmdTopic = MqttSettings.getPrefix() + serial + "/" + commandTopic;
String statTopic = MqttSettings.getPrefix() + serial + "/" + stateTopic;
DynamicJsonDocument root(2048);
root["name"] = String(inv->name()) + " " + caption;
Serial.printf("[MqttHandleHassClass::publishInverterNumber] root capacity of : %d \r\n", root.capacity());
byte = serializeJson(root, buffer);
MessageOutput.printf("[MqttHandleHassClass::publishInverterNumber] size of buffer(name): %d (%d) \r\n", byte, measureJson(root));
Serial.println(buffer);For my first inverter solar I get: For my second inverter luna I always get: Dumping the free heap in MqttHandleHassClass::publishField shows the decrease in memory: At the moment I have no idea where to look for the memory leak. |
Beta Was this translation helpful? Give feedback.
-
|
Can you confirm this? v6.4.0 v6.3.2 If you can confirm this, I would open a issue at platform-espressif32. Even if the code is bad... the total amount of heap should not change except for maybe changes in the linker scripts? |
Beta Was this translation helpful? Give feedback.
-
|
I'm using a D1 mini board. @schlimmchen seems to have other values. If I'm using v6.4.0 in my fully configured productive board within seconds all free heap space is gone (see above). So values below are taken from a separate D1 mini board (unconfigured except network). Until now I couldn't find out, where the leak is. v6.3.2 v6.4.0 But yes, I could confirm that 34kB of heap is missing. |
Beta Was this translation helpful? Give feedback.
-
|
The heap issue seems to be something which is fixed in arduino core 2.0.12 (which depends on IDF 4.4.5 https://github.com/espressif/esp-idf/releases/tag/v4.4.5) |
Beta Was this translation helpful? Give feedback.
-
Unfortunately after merging v23.9.11 I encountered several issues, so I had to revoke the upstream code.
What I observed so far:
At the moment, I don't know which changes from the upstream repo are causing this. So best strategy I have for the moment, to cherry pick each commit from the upstream repo into a separate branch one after the other and test it. This will take some time.
If there is someone out there with more time, please feel free to review the changes in commit eb1c2db in the development branch.
Beta Was this translation helpful? Give feedback.
All reactions