-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Benchmark section to our documentation page #3125
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
name: Download SVGs | ||
|
||
on: | ||
workflow_dispatch: # You can trigger the workflow manually, or set it on a schedule. | ||
|
||
jobs: | ||
download-svgs: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.x' # Use Python 3.x | ||
|
||
- name: Install dependencies | ||
run: | | ||
pip install requests | ||
|
||
- name: Run download_svgs.py | ||
env: | ||
MUNIN_USERNAME: ${{ secrets.MUNIN_USERNAME }} | ||
MUNIN_PASSWORD: ${{ secrets.MUNIN_PASSWORD }} | ||
run: | | ||
python3 /scripts/download_svgs.py |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
--- | ||
title: Benchmarks | ||
sidebar_position: 10 | ||
--- | ||
|
||
# Benchmarks | ||
|
||
This section contains performance benchmarks for ThinEdge.io, including detailed CPU and memory usage for key processes, measured on a device running in the OSADL QA Farm. | ||
|
||
## Hardware Information | ||
|
||
The following hardware and software specifications describe the environment in which these benchmarks are measured. The device runs ThinEdge.io and its associated plugins for data collection. | ||
|
||
| Component | Specification | | ||
|----------------|---------------------------------------------------------------| | ||
| **Processor** | Quad-core ARM Cortex-A72 (ARM v8) | | ||
| **Memory** | 1.8 GB RAM | | ||
| **Operating System** | Debian GNU/Linux 12 (bookworm) | | ||
| **MQTT Broker**| Mosquitto (v2.0.18) | | ||
| **Firmware** | ThinEdge.io (v1.1.2) | | ||
| **Plugins Used** | `tedgecpuprocent`, `memory`, `tedge-agent`, `tedge-mapper` | | ||
|
||
> The device is a Raspberry Pi 4 Model B Rev 1.5, a member of the OSADL QA Farm, which continuously collects data on performance and resource consumption. | ||
|
||
## CPU Run-time Consumption of Key ThinEdge.io Tasks | ||
|
||
In this section, we monitor the CPU usage of the most critical ThinEdge.io tasks: `mosquitto`, `tedge-agent`, and `tedge-mapper`. These tasks are essential for device-to-cloud communication, and their performance is crucial for the overall efficiency of the system. | ||
|
||
### Graph of CPU Usage | ||
|
||
Below is a visual representation of the CPU consumption by the main ThinEdge.io processes over the past month. | ||
|
||
![CPU Usage](./tedgecpuprocent-month.svg) | ||
|
||
For each process, the graph shows: | ||
- **Command Name (comm)**: The name of the task running on the device. | ||
- **CPU Usage (cuc)**: The percentage of CPU time utilized by each process. | ||
|
||
|
||
The graph allows you to visually track CPU performance trends and identify potential bottlenecks in the system. | ||
|
||
### Detailed Metrics | ||
|
||
The following table provides a detailed description of the CPU consumption of measured tasks: | ||
|
||
| Task | Description | | ||
|----------------------|----------------------------------------------------------------| | ||
| `mosquitto` | Handles MQTT communications between devices and cloud | | ||
| `tedge-agent` | Coordinates device management, software updates, and telemetry | | ||
| `tedge-mapper_c8y` | Converts telemetry data into the format required by cloud | | ||
| `tedge-mapper-colle` | Converts telemetry data into the format required | | ||
|
||
> Note: The values in this table are updated dynamically once per month. | ||
|
||
|
||
## Memory Usage | ||
|
||
In this section, we monitor the memory consumption of key ThinEdge.io components to ensure efficient performance and to view how the system memory is being utilized. | ||
|
||
### Graph of Memory Usage | ||
|
||
Below is a visual representation of the memory consumption of ThinEdge.io tasks over the past month. | ||
|
||
![Memory Usage](./tedgemem-month.svg) | ||
|
||
### Monitored Metrics: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see these metrics as useful for giving an idea of the memory footprint of thin-edge, since there are not directly related. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, there are other metrics which just look at the memory for each of the thin-edge.io related memory usage, e.g. https://munin.osadl.org/munin/osadl.org/rackfslot1.osadl.org/tedgemem.html |
||
- **Mosquitto Memory Usage**: Tracks the memory consumed by the mosquitto process, which manages MQTT messages and connections. | ||
- **Tedge-Agent Memory Usage**: Tracks the memory consumed by the tedge-agent process, which handles software updates, configurations, and telemetry operations. | ||
- **Tedge-Mapper Memory Usage**: Tracks the memory consumed by the tedge-mapper process, which converts telemetry data between MQTT and other protocols (e.g., Cumulocity, AWS). For multi-instance tedge-mapper, each instance is tracked individually. | ||
|
||
For each metric, the graph shows the current and historical values, allowing you to monitor how memory is utilized by the system. | ||
|
||
### Detailed Metrics | ||
|
||
The following table provides a description of memory usage metrics: | ||
|
||
| Metric | Description | | ||
|--------------------------|-----------------------------------------------------------------------------| | ||
| **Mosquitto Memory** | Memory used by the mosquitto process, which manages MQTT messages and connections. | | ||
| **Tedge-Agent Memory** | Memory used by the tedge-agent process, responsible for software updates, configurations, and telemetry operations | | ||
| **Tedge-Mapper Memory** | Memory used by the tedge-mapper process, which converts telemetry data between MQTT and other protocols (e.g., Cumulocity, AWS). Each instance of tedge-mapper is monitored individually if multiple instances exist | | ||
|
||
|
||
> The values in this table are updated dynamically once per month. | ||
|
||
By monitoring these metrics, we can ensure that the system is using memory efficiently, and we can detect potential memory leaks or bottlenecks in the system's operation. | ||
|
||
--- | ||
|
||
This section now contains all relevant memory metrics for ThinEdge tasks, helping you track and analyze how resources are allocated and used by mosquitto, tedge-agent, and tedge-mapper over time. These benchmarks are crucial for maintaining optimal performance of ThinEdge.io components and ensuring efficient memory usage across the system. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A key point is missing: what is the workload used for these benchmarks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree fully, I put it in the ToDo section as decision needed what we want to be the workload