Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

grafana(dm): fix grafana expr #4405

Merged
merged 10 commits into from
Jan 25, 2022
Merged

grafana(dm): fix grafana expr #4405

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
baa476e
grafana(dm): fix grafana expr
Ehco1996 Jan 19, 2022
c34975c
Merge branch 'master' into fix-grafana
Ehco1996 Jan 19, 2022
2d19d10
Merge branch 'master' into fix-grafana
Ehco1996 Jan 24, 2022
685d047
address commnet
Ehco1996 Jan 24, 2022
28f5994
add group right for syncer binlog file metrci
Ehco1996 Jan 24, 2022
fa5c966
fix whitespace
Ehco1996 Jan 24, 2022
4e8500c
Merge branch 'master' into fix-grafana
ti-chi-bot Jan 24, 2022
25d7456
Merge branch 'master' into fix-grafana
ti-chi-bot Jan 24, 2022
1003b2f
Merge branch 'master' into fix-grafana
ti-chi-bot Jan 24, 2022
6509f70
Merge branch 'master' into fix-grafana
ti-chi-bot Jan 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions dm/dm/dm-ansible/scripts/DM-Monitor-Professional.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3059,7 +3059,7 @@
"steppedLine": false,
"targets": [
{
"expr": "dm_syncer_binlog_file{source_id=~\"$source\", task=~\"$task\", node=\"master\"} - ON( task,source_id) dm_syncer_binlog_file{source_id=~\"$source\", task=~\"$task\", node=\"syncer\"}",
"expr": "dm_syncer_binlog_file{source_id=~\"$source\", task=~\"$task\", node=\"master\"} - ON(task,source_id) dm_syncer_binlog_file{source_id=~\"$source\", task=~\"$task\", node=\"syncer\"}",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preview link

"format": "time_series",
"hide": false,
"interval": "$interval",
Expand Down Expand Up @@ -3716,12 +3716,12 @@
"steppedLine": false,
"targets": [
{
"expr": "dm_relay_binlog_file{instance=~\"$instance\", node=\"relay\"} - ON(instance, task) dm_syncer_binlog_file{instance=~\"$instance\", task=~\"$task\", node=\"syncer\"}",
"expr": "dm_relay_binlog_file{instance=~\"$instance\", node=\"relay\"} - ON(instance) group_right dm_syncer_binlog_file{instance=~\"$instance\", task=~\"$task\", node=\"syncer\"}",
"format": "time_series",
"hide": false,
"interval": "$interval",
"intervalFactor": 2,
"legendFormat": "{{task}} - {{instance}}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remain {{task}} - {{instance}} as each task will produce a different value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this metric: dm_relay_binlog_file don't have label task so this legend looks like this - 127.0.0.1:8621



dm_relay_binlog_file{instance="127.0.0.1:8262", job="dm-worker-1", node="master"} | 1
-- | --
dm_relay_binlog_file{instance="127.0.0.1:8262", job="dm-worker-1", node="relay"}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @glorv ptal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe you can use group_right

dm_relay_binlog_file{instance="172.16.5.177:8101", node="relay"} - ON(instance) group_right dm_syncer_binlog_file{instance="172.16.5.177:8101", node="syncer"}

please find a good solution from https://prometheus.io/docs/prometheus/latest/querying/operators/#many-to-one-and-one-to-many-vector-matches

Copy link
Contributor Author

@Ehco1996 Ehco1996 Jan 24, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

due to only syncer_binlog_fie have many to on relationship with relay_binlog_file, only one dashboard need to be adjusted. added in 28f5994 and fa5c966, preview link is here

"legendFormat": "{{instance}} - {{task}}",
"refId": "B"
}
],
Expand Down Expand Up @@ -5088,7 +5088,7 @@
"steppedLine": false,
"targets": [
{
"expr": "dm_relay_binlog_file{instance=~\"$instance\", node=\"master\"} - ON(instance, job) dm_relay_binlog_file{instance=~\"$instance\", node=\"relay\"}",
"expr": "dm_relay_binlog_file{instance=~\"$instance\", node=\"master\"} - ON(instance) dm_relay_binlog_file{instance=~\"$instance\", node=\"relay\"}",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preview link

"format": "time_series",
"hide": false,
"interval": "$interval",
Expand Down
6 changes: 3 additions & 3 deletions dm/dm/dm-ansible/scripts/DM-Monitor-Standard.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,23 +450,23 @@
"targets": [
{
"exemplar": true,
"expr": "histogram_quantile(0.90, sum(rate(dm_syncer_replication_lag_bucket{instance=~\"$instance\",task=~\"$task\"}[$interval])) by (le,instance,task))",
"expr": "histogram_quantile(0.90, sum(rate(dm_syncer_replication_lag_bucket{source_id=~\"$source\",task=~\"$task\"}[$interval])) by (le,task,source_id))",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preview link

"hide": false,
"interval": "",
"legendFormat": "{{task}} - {{source_id}} - 90",
"refId": "B"
},
{
"exemplar": true,
"expr": "histogram_quantile(0.95,\nsum(rate(dm_syncer_replication_lag_bucket{instance=~\"$instance\",task=~\"$task\"}[$interval])) by (le,instance,task))",
"expr": "histogram_quantile(0.95,\nsum(rate(dm_syncer_replication_lag_bucket{source_id=~\"$source\",task=~\"$task\"}[$interval])) by (le,task,source_id))",
"hide": false,
"interval": "",
"legendFormat": "{{task}} - {{source_id}} - 95",
"refId": "C"
},
{
"exemplar": true,
"expr": "histogram_quantile(0.99,\nsum(rate(dm_syncer_replication_lag_bucket{instance=~\"$instance\",task=~\"$task\"}[$interval])) by (le,instance,task))",
"expr": "histogram_quantile(0.99,\nsum(rate(dm_syncer_replication_lag_bucket{source_id=~\"$source\",task=~\"$task\"}[$interval])) by (le,task,source_id))",
"hide": false,
"interval": "",
"legendFormat": "{{task}} - {{source_id}} - 99",
Expand Down