[POC] netdata collector of thin-edge measurements

[didier-wenzek]

Proposed changes

Exploring the idea of a Netdata collector for tedge measurements.

Netdata provides out-of-the box many collector for device-level metrics (cpu, mem, network, disk, ...). Thin-edge can add application-specific insight by injecting netdata the measurements published by thin-edge devices.

Run it

A stream of thin-edge measurements:

$ tedge mqtt sub 'te/+/+/+/+/m/#`'
[te/device/main///m/environment] { "temperature": 1 , "time": 1709315843.322205791, "pressure": 11 }
[te/device/main///m/environment] { "temperature": 2 , "time": 1709315844.330121504, "pressure": 22 }
[te/device/main///m/environment] { "temperature": 3 , "time": 1709315845.336501517, "pressure": 33 }

is transformed into a stream of netdata CHART | DIMENSION | SET events consumed by Netdata plugin.d:

$ target/debug/netdata-collector
CHART "tedge.main/environment" "tedge.main/environment" "tedge.main/environment" "units"
DIMENSION "temperature" "temperature"
DIMENSION "pressure" "pressure"
BEGIN "tedge.main/environment"
SET "temperature" = 1
SET "pressure" = 11
END
BEGIN "tedge.main/environment" 1012796
SET "temperature" = 2
SET "pressure" = 22
END
BEGIN "tedge.main/environment" 1011501
SET "temperature" = 3
SET "pressure" = 33
END

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s)
• I ran cargo fmt as mentioned in CODING_GUIDELINES
• I used cargo clippy as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 157 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/extensions/netdata_collector/src/actor.rs	0.00%	53 Missing ⚠️
crates/extensions/netdata_collector/src/message.rs	0.00%	48 Missing ⚠️
plugins/netdata-collector/src/main.rs	0.00%	36 Missing ⚠️
crates/extensions/netdata_collector/src/builder.rs	0.00%	20 Missing ⚠️

Additional details and impacted files

📢 Thoughts on this report? Let us know!

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
510	0	2	510	100	1h25m18.596043999s

[reubenmiller]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

[didier-wenzek]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

Yes, this won't impact build times as the netdata interface is really thin. However, I wonder if the proper place for this plugin shouldn't be under https://github.com/thin-edge/thin-edge.io_examples. And yes, more testing is required.

[reubenmiller]

@didier-wenzek I'm assuming this won't impact our build times much right? If so then I think we can merge it soon. It might be worth while someone compiling it and testing it out once at least before merging though

Yes, this won't impact build times as the netdata interface is really thin. However, I wonder if the proper place for this plugin shouldn't be under https://github.com/thin-edge/thin-edge.io_examples. And yes, more testing is required.

I'm reluctant to add it to thin-edge.io_examples as it makes it harder to improve. Maybe this could be the first rust based plugin which re-uses some of the thin-edge.io crates, if so then we can create a separate repository for this (under the thin-edge gh organization).