Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[P2P] unit with latest build not receiving values of shared plugins #4718

Closed
chromoxdor opened this issue Jun 26, 2023 · 30 comments · Fixed by #4720
Closed

[P2P] unit with latest build not receiving values of shared plugins #4718

chromoxdor opened this issue Jun 26, 2023 · 30 comments · Fixed by #4720

Comments

@chromoxdor
Copy link
Contributor

chromoxdor commented Jun 26, 2023

@TD-er if i remember correctly, you recently made changes in the P2P code.
I compiled myself a fresh ESPeasy (Build: ESP_Easy_mega_20230626_normal_ESP8266_4M1M Jun 26 2023).
It shows the plugins shared by other nodes but the values are always zero.
Just that you know....

Edit:
For the context: I have a network of nodes sharing successfully plugins and values for a long time ... its only the new one that doesn´t want to receive.

@chromoxdor
Copy link
Contributor Author

Hmm..just shared from my oldest node... that seems to work. strange

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

That's really strange as I am working on that code right now to optimize it and as far as I know it wasn't broken when I started working on this code 2 days ago.

Can you check with the code of this PR: #4354
?

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 26, 2023

Can you check with the code of this PR: #4354

no luck..
I tried a slightly older build i had lying around which works fine: (Build: ESP_Easy_mega_20230427_normal_ESP8266_4M1M Apr 27 2023)

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

I know there was a bug in one of the builds, but the before last build was working fine again between new and old builds again.
And I do have it working here between nodes running builds I made today and builds with 80+ days uptime. (built on 20230405)

@chromoxdor
Copy link
Contributor Author

just tried also Build: ESP_Easy_mega_20230508_normal_ESP8266_4M1M May 8 2023
this already doesn´t work

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

And which build is on 'the other end' sending the p2p data?

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 26, 2023

@chromoxdor
Copy link
Contributor Author

this works too:
ESP_Easy_mega_20201023_normal_ESP8266_1M
🙂

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 26, 2023

it seems that the issue started somewhere between 11.2021 and 10.2022

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

Very strange....
The reason of the other bug was when I added the code for those additional integer types for Dummy tasks (e.g. to store 4x 32bit int, 2x 64 bit int, 2x double, etc)
And that was added this year.

Are you using anything specific? Like different nr of tasks?

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 26, 2023

Are you using anything specific? Like different nr of tasks?

No, nothing i can think of…

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

Double check you have not used unit ID 0, nor 255
Also make sure UDP port 8266 (or whatever port you're using) is the same
Maybe perform an extra save on the sending and/or receiving task

What specific plugins are you using?
Maybe some are using NaN values and I think I've seen some checks for valid values...

@chromoxdor
Copy link
Contributor Author

ids and udp port are correct.
i up and downgraded one existing node and used a new one for verification. as soon as i upgraded to the newest build values became 0 and when downgrading again it worked

What specific plugins are you using?

Plugins for: dht22, bme280 and just for testing i used the internal analog plugin.

You could try to recreate it by installing one of the above mentioned builds on one device…

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

I have units sharing the sysinfo data among them as this seemed a nice test since it doesn't need any hardware to be connected.

@chromoxdor
Copy link
Contributor Author

didn’t think of the sysinfo plugin but the analog plugin reading noise worked also for me.

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

image

Proof to show it is working.

@chromoxdor
Copy link
Contributor Author

I am happy to see that it is working with recent builds. But i also never questioned that. I guess i have to do an updateweekend (easier would actually be to downgrade the one new device 😁) to make things work again…

@TD-er
Copy link
Member

TD-er commented Jun 26, 2023

It is with the same code as from which you concluded it wasn't working.... as it is the same code as on the PR I linked.

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 27, 2023

Ok i give up.. i tested today different older releases as sender and the latest build as receiver. It works fine!
It seems my custom builds are the issue. I don´t understand why. They are able to send and receive data to each other and to releases before 05.2023 but not to a recent build. Thats really weird since the only addition i made was the color code to rules...

I will close this as it seems to be a very individual issue which hopefully can be solved by updating my devices.

Edit:
Here some pictures to prove it :)
Bildschirmfoto 2023-06-27 um 11 08 29

Bildschirmfoto 2023-06-27 um 11 01 50

@TD-er
Copy link
Member

TD-er commented Jun 27, 2023

Like I said, for plugins other than sysinfo there might be a recently added check to see if the data is "valid" and it might not be for all plugins (e.g. when a NaN is set, maybe other reasons too...)
So if the sysinfo cannot be sent between old and new builds, then it probably is some weird build issue.
Otherwise it may still be a bug.

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 27, 2023

So if the sysinfo cannot be sent between old and new builds,

Unfortunately all my customs builds have in common that they lack the sysinfo plugin. :)
But i unsuccessfully tried a dummy device instead and fed it with values through taskvalueset.

then it probably is some weird build issue.

jup...

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 27, 2023

@TD-er
Sometimes the most obvious thing to do it the easiest to forget...

There is this thing called "log":
Bildschirmfoto 2023-06-27 um 14 22 45

🤦‍♂️
What does it mean?

Edit: This is what i am getting on the node with the new build trying to get data from one with an "old" release

@TD-er
Copy link
Member

TD-er commented Jun 28, 2023

Ah maybe that's too strict...
I have added a check to see what plugin was sending the data, so the received data would make sense.
Thus having local and remote use the same plugin ID (e.g. P004 for Dallas 1-wire)

But when it has "255" as value (or 0 ???) it is clearly not set at the remote end, thus older firmware
I will make this check a bit more relaxed.

@TD-er
Copy link
Member

TD-er commented Jun 28, 2023

In src/Globals/Plugins.cpp
Change this function into:

Edit2: Removed suggested code change, see next post.

Edit:
This is just a quick 'fix' to test this, not sure if it will be the final fix as this restricts the use of plugin ID 255.
So I might want to 'patch' the received data from older p2p nodes as we do know the build they are running.

@TD-er
Copy link
Member

TD-er commented Jun 28, 2023

OK, better fix: (thus undo the previous changes if already done)

bool C013_SensorDataStruct::matchesPluginID(pluginID_t pluginID) const
{
  if (deviceNumber == 255 || !validPluginID(deviceNumber) || !validPluginID(pluginID)) {
    // Was never set, so probably received data from older node.
    return true;
  }
  return pluginID == deviceNumber;
}

@chromoxdor
Copy link
Contributor Author

chromoxdor commented Jun 28, 2023

Change this function into:
OK, better fix:

Where am i supposed to put this or what should i change?

@TD-er
Copy link
Member

TD-er commented Jun 28, 2023

This is in src/DataStructs/C013_p2p_dataStruct.cpp
Around line 34 there is already this function, so I just added deviceNumer == 255 ||

@chromoxdor
Copy link
Contributor Author

not quite the fix..

Bildschirmfoto 2023-06-28 um 11 34 26

@chromoxdor
Copy link
Contributor Author

ich also changed this:

bool C013_SensorDataStruct::matchesSensorType(Sensor_VType sensor_type) const
{
  if (deviceNumber == 255 || sensorType == Sensor_VType::SENSOR_TYPE_NONE) {
    // Was never set, so probably received data from older node.
    return true;
  }
  return sensorType == sensor_type;
}

now it works

@TD-er
Copy link
Member

TD-er commented Jun 28, 2023

Check!
Will make a commit for it to fix this.

TD-er added a commit that referenced this issue Jun 29, 2023
[P2P] unit with latest build not receiving values of shared plugins #4718
erstec pushed a commit to ly1bwb/ESPEasy that referenced this issue Aug 23, 2023
Release mega-20230822

Changes in mega-20230822 (since mega-20230623):

TD-er (103):
      [Eastron] Fix merge issues
      [Eastron] Uncrustify + remove 500 bytes of  duplicate strings
      [Eastron] Fix merge issues
      [Cleanup] Reduce bin size by stripping unused code
      [Cleanup] Reduce bin size by deduplicate factory reset code
      [Factory Reset] Fix keeping settings (letscontrolit#4263)
      [RFcal] Fix typo in command "clearwifirfcal"
      [SendToHTTP] Only include "Authorization" header  when credentials set
      [ESP32 2.0.5.x] Fix crashes calling noInterrupt() in ISR callback
      [Adafruit NeoPixel] Update lib to fix noInterrupts
      [Quad Enc] Fix ESP32 noInterrupts()
      [Build] Fix merge issues, missing #endif
      [ESP32-solo1] Add support for single-core ESP32
      [Web flasher] Split "4M collections builds" in a separate group
      [release build] Copy ESP32solo1 binaries during release
      [Crash Analysis] Show controller/plugin name on crash at system event
      [Build] Fix merge issue with duplicate PlatformIO envs
      [Build] Fix missing #ifdef check
      [Devices] Fix showing all included plugins in devices pull-down list
      [Cleanup] Move plugin function pointer array to PROGMEM
      [Cleanup] Simplify weblog buffer
      [Cleanup] Fix build error due to missing defines
      [Cleanup] Improve web serve speed + value formatting speed
      [Cleanup] Reduce resouces used on Dallas, Sysinfo and Dummy tasks
      [Build] Fix missing #ifdef check
      [Cleanup] Reduce CPU usage only calculate valuecount once
      [ESPEasy p2p] Remove delay() calls from p2p messaging
      [Cleanup] Remove unused PLUGIN_EVENT_OUT
      [Cleanup] Get timing stats for controller function calls
      [cleanup] Simplify parse controller variables
      [P2P] unit with latest build not receiving values of shared plugins (letscontrolit#4718)
      [P2P] Check for unset DeviceNumber in matchesSensorType
      [Cleanup] Speedup frequent calls + reduce build size
      [Cleanup] Reduce calls to saveToRTC()
      [Cleanup] Simplify mustConsiderAsJSONString()
      [Cleanup] Remove long webserver debug log
      [Cleanup] Reduce backgroundtasks() frequency
      [Cleanup] Reduce mem allocs for ADC logs
      [HLW8012] Improve stability and accuracy
      [Cleanup] Allow a task to set initial PLUGIN_READ on init
      [Cleanup] Reduce memory usage when timingstats is disabled (or read)
      [PlatformIO] Force PIO to 6.1.7
      [HLW8012] Fix instability due to IRAM
      [Build] Move "hard_" builds to ESP8266 SDK 3.1.2
      [Cleanup] Reduce items being logged in TimingStats
      [Build] Revert switching 'hard_' builds to ESP8266 SDK 2.7.x
      [HLW8012] Remove IIR filter from interrupt callback function
      [PlatformIO] Test build with PIO 6.1.9rc1
      [PlatformIO] Test build PIO 6.1.9rc1 (2nd attempt)
      [PIO] Set minimal PIO version to 6.1.9
      [Cleanup] Simplify allocation of ControllerSettings allocation, memleak?
      [PVS Studio] Fix issues found by PVS Studio
      [PVS Studio] Fix build error & unsafe macro
      [PVS Studio] Fix some issues found by PVS Studio
      [Cleanup] Attempt to fix memory leak in PluginTaskData
      [ArduinoJSON] Update to ArduinoJson 6.21.2
      [Cleanup] Simplify PluginStats::getSample
      [PVS Studio] Fix some issues reported by PVS Studio
      [MDNS] Disable MDNS for ESP8266 due to memory leak
      [CSE7766] Improve reliability + fix issue with console port selection
      [CSE7766] Fix parsing packets with lost bytes
      [CSE7766] Add output selection of units of measure
      [WiFi] Fix ESP8266 instability  compiler optimization flag -O2
      [mDNS] Call MDNS.announce() every second on ESP8266 (mem leak?)
      [Cleanup] Add strformat function to simplify log line creation
      [Build] Fix build errors incorrect use of strformat()
      [Cleanup] Move duplicate oversampling code to helper class
      [ESP32 Serial] Fix serial tasks not working after save or restart task
      [ESP32 Serial] Do not cache baud rate between end() and begin()
      [Cleanup] Replace oversampling code duplication P145
      [Cleanup] Replace oversampling code duplication P060
      [Cleanup] Replace oversampling code duplication P067
      [CSE7766] Update task value with intermediate oversampling value
      [Oversampling] Add reset + weighted keep last
      [ESP32 Serial] Fix serial tasks not working after save or restart task
      [Eastron] Update SDM library to v2.2.2
      [Eastron] Re-apply ESPEasy specific changes to SDM library
      [Eastron] Generate all SDM register strings
      [Eastron] Cleanup showing default taskvalue names + SDM72_V2add
      [Eastron] Cleanup code duplicates & uncrustify
      [Eastron] Fix compiler warnings
      [Eastron] Add support for SDM320C
      [Eastron] Split 'readVal' function to allow async reads
      [Eastron] Add "demand" in register description
      [Eastron] Implement complete async read of Eastron registers
      [Eastron] Add hardware RTS support for ESP32 (for ~RE/DE pin on RS485)
      [Eastron] Fix ESP32 hardware RTS flow control
      [Eastron] improve speed & add commands to set modbusID & baudrate
      [Eastron] Start documentation
      [Eastron] Document RS485 specific wiring
      [Eastron] Complete documentation on Eastron plugin
      [Eastron] Add support for RS485 collision detection on ESP32
      [Serial Console] Fix HW Serial0 unusable on ESP8266 limited builds
      [Eastron] Add delay on first PLUGIN_READ to allow reading registers
      [Build] Set ESP8266 Energy build to LIMIT_BUILD_SIZE
      [Build] Change LIMIT_BUILD_SIZE on ESP8266 Energy to BUILD_NO_DEBUG
      [Eastron] Improve read stability on ESP8266
      [CSE7766] Fix crash when not all stats are enabled
      [Settings] Fix setting task var names & load default names
      [Settings] Fix task value names not being updated on ESP32 (cache)
      [Settings] Fix setting custom task value name on Eastron task
      [ADS1115] Retry on error reads within timeout
      automatically updated release notes for mega-20230822

Ton Huisman (68):
      [Release][Bugfix] Script should deploy an archive per CPU Architecture
      [Docs] Describe `Wrk` and `Wkd` indicators for `Clock#Time` event
      [P153] Remove unused #include
      [Build] Add support for ESP32-S3 8MB units
      [Build] Rearrange 8M2M to 8M1M configuration
      [P118] Fix compilation when suppressing only DEBUG logging
      [Build] Check all ESP32 builds correctly for factory.bin filesize
      [Docs] Fix typo on Plugin page
      [Build] Add `boards` definitions for ESP32-S3 8MB and ESP32-S3 16MB
      [ETH] Allow Ethernet Power GPIO pin unset (letscontrolit#4724)
      [P014] Set default output values to Temp/Hum
      [P118] Fix compilation when included in Custom builds
      [Build] Update board_name for custom_ESP32_4M2M_NO_OTA_LittleFS
      [Build] Add missing #undef FEATURE_TIMING_STATS
      [Rules] Add minHexDigits to toHex function
      [Docs] Improvements and various fixes
      [Build] Add some missing defines to MAX build to make them explicitly available
      [P128] Revert changes after library update (not working with ESP8266)
      [UI] Add option for disabling Rules auto-completion
      [P035] Add Inverted output option, as supported by the IRsend library
      [P035] Code improvements
      [P035] Add documentation
      [P009] Small documentation typo
      [P014] Fix invalid cast
      [Build] Move ESP32_4M2M settings to own board file
      [Build] Remove unused board definition files
      [P118] Allow enable/disable of ORCON feature via Custom.h
      [P141] Allow enable/disable of Cursor X/Y values feature via Custom.h
      [Console] Add newline after console command to avoid mixing in with logging
      [P046] Fix documentation, address some code issues, WiP
      [P046] Fix compilation issue and include in MAX build
      [Build] Include IR plugins in MAX builds
      [P035] Small size reductions, remove some unused commented code
      [Build] Enable I2C device scan for ESP32 builds
      [P098] Add documentation
      [EasyColorCode] Fix typo `7ddt` (was 7dtt)
      [EasyColorCode] Add some missing commands
      [P013] Move to category Distance (from position), add missing [P134] to Distance too
      [Controllers] Fix `validCPluginID` check (initialization and actual check)
      [P098] Add precaution to documentation for PWM usage
      [Build] Reduce ENERGY build size by removing DEBUG logging
      [Build] Fix compiler warnings
      [EasyColorCode] Fix some auto-complete issues and typos
      [EasyColorCode] Add few missing commands for P036
      [Lib] Update NeoPixelBus to current latest
      [P128] Use customized NeoPixelBus library, upgrading to latest features won't properly work
      [P113] Fix for shifted I2C address in library, code improvements
      [P113] Apply the fix at the original crime-scene ;-)
      [Libs] Update VL53L1X library to current latest v1.2.12 (with fix)
      [P113] Disable I2C address selection as that seems to mess up the sensor
      [P113] Update documentation
      [P122] Uncrustify and move initialization from constructor to .h file
      [P153] Remove unused variable
      [P020] Sourcecode formatting for clarification
      [Docs] Correct some layout issues, causing parts of documentation to disappear
      [P113] Add a few extra nullptr checks
      [Core] Fix some string/message inconsistencies
      [P113] Add a few extra nullptr checks
      [Converters] Add toStringNoZero() function
      [Generic] Improvements and fixes part 2, Aug 2023
      [P113] Fix typo in log statement
      [AdaGFX] Additional error information for unsupported bmp files, improve docs
      [UI] Widen `textarea` fields for easier input of larger/wider text
      [Docs] P002 / P047 Add YT link for selecting a proper (analog) Soil-moisture sensor
      [Converters] Use the correct conversion
      [Tools] Remove ESP Easy Flasher, update Espressif Download tool
      [Tools] Revert esptool to last version
      [Tools] Add `blank_8MB.bin` for erasing 8MB units, exclude from ESP8266 archive

tomcatxx (3):
      Update SDM.h
      Update _P078_Eastron.ino
      Update SDM.h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants