duplicate 114 SmartREST messages sent to Cumulocity IoT in quick succession

[reubenmiller]

Describe the bug

Duplicate Cumulocity IoT SmartREST messages with message id 114 are sent in very short succession, e.g. 10 messages are sent within 170ms.

Below shows the MQTT log snippet pulled from a system test run.

2024-01-09T16:03:33.039402       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039427       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039445       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039461       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039477       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039493       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039509       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039525       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039541       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039556       c8y/s/us                            114,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
2024-01-09T16:03:33.039572       tedge/commands/req/software/list    {"id":"c8y-mapper-2024-01-09T16:03:32.979985356Z"}

This issue is very similar to the following issues:

• Updating /etc/tedge/plugins/tedge-log-plugin.toml results in multiple log_upload metadata messages being sent #2451

To Reproduce

The error is visible by running the system tests, and checking the MQTT log history (which is included in the test report).

Expected behavior

Screenshots

Environment (please complete the following information):

• OS [incl. version]: Ubuntu 20.04
• Hardware [incl. revision]: docker
• System-Architecture [e.g. result of "uname -a"]: Github Action Ubuntu 20.04 Runner
• thin-edge.io version [e.g. 0.1.0]: 1.0.0~rc.2~42+g47f393e

Additional context

[Bravo555]

The duplicates are created because:

1. we read the operations directory when we construct an internal topic filter in crates/extensions/c8y_mapper_ext/src/config.rs:205, constructing a full list of operations
2. But we also send 114 when we get "operation registration" MQTT messages, e.g. on topic te/device/main///cmd/restart

The best solution would be most likely to make a current list of supported operation a part of digital twin, which should send messages only when there are actually any changes send.

[gligorisaev]

QA has thoroughly checked the bug and here are the results:

• Test for ticket exists in the test suite.
• tests/RobotFramework/tests/cumulocity/supported_operations/mapper-publishing-agent-supported-ops.robot
• QA has tested the bug and it's not reproducable anymore