Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

document, or get rid of, the custom prometheus rules #7917

Closed
richvdh opened this issue Jul 20, 2020 · 5 comments
Closed

document, or get rid of, the custom prometheus rules #7917

richvdh opened this issue Jul 20, 2020 · 5 comments
Labels
A-Docs things relating to the documentation

Comments

@richvdh
Copy link
Member

richvdh commented Jul 20, 2020 •
edited
Loading

our grafana dashboard (that we publish in contrib) relies on a few custom prometheus rules (particularly for the "Event Persist Rates" section). We should really either document them, or update the dashboard not to depend on them.
@richvdh richvdh changed the title document, or get rid of, the custom prometheus rules our grafana dashboard (that we publish in contrib) relies on a few custom prometheus rules (particularly for the Event Persist Rates section). We should really either document them, or update the dashboard not to depend on them. document, or get rid of, the custom prometheus rules Jul 20, 2020
@richvdh
Copy link
Member Author

richvdh commented Jul 20, 2020

the custom rules in question are:

groups:
- name: synapse
  rules:
  - record: synapse_http_server_requests:method
    expr: sum without(servlet, tag) (synapse_http_server_requests)
    labels:
      servlet: ""
  - record: synapse_http_server_requests:servlet
    expr: sum without(method) (synapse_http_server_requests)
    labels:
      method: ""
  - record: synapse_http_server_response_ru_time_seconds
    expr: synapse_http_server_response_ru_utime_seconds + synapse_http_server_response_ru_stime_seconds
  - record: synapse_federation_client_sent
    expr: synapse_federation_client_sent_edus + 0
    labels:
      type: EDU
  - record: synapse_federation_client_sent
    expr: synapse_federation_client_sent_pdu_destinations:count + 0
    labels:
      type: PDU
  - record: synapse_federation_client_sent
    expr: sum without(type) (synapse_federation_client_sent_queries)
    labels:
      type: Query
  - record: synapse_federation_server_received
    expr: synapse_federation_server_received_edus + 0
    labels:
      type: EDU
  - record: synapse_federation_server_received
    expr: synapse_federation_server_received_pdus + 0
    labels:
      type: PDU
  - record: synapse_federation_server_received
    expr: sum without(type) (synapse_federation_server_received_queries)
    labels:
      type: Query
  - record: synapse_federation_transaction_queue_pending
    expr: synapse_federation_transaction_queue_pending_edus + 0
    labels:
      type: EDU
  - record: synapse_federation_transaction_queue_pending
    expr: synapse_federation_transaction_queue_pending_pdus + 0
    labels:
      type: PDU
  - record: synapse_storage_events_persisted_by_source_type
    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_type="remote"})
    labels:
      type: remote
  - record: synapse_storage_events_persisted_by_source_type
    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_entity="*client*",origin_type="local"})
    labels:
      type: local
  - record: synapse_storage_events_persisted_by_source_type
    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_entity!="*client*",origin_type="local"})
    labels:
      type: bridges
  - record: synapse_storage_events_persisted_by_event_type
    expr: sum without(origin_entity, origin_type) (synapse_storage_events_persisted_events_sep)
  - record: synapse_storage_events_persisted_by_origin
    expr: sum without(type) (synapse_storage_events_persisted_events_sep)

I've not gone through to check which of those are used by the dashboard.

@anoadragon453 anoadragon453 added the A-Docs things relating to the documentation label Jul 20, 2020
@richvdh
Copy link
Member Author

richvdh commented Jul 29, 2020

turns out they are documented, in https://github.com/matrix-org/synapse/blob/master/contrib/prometheus/synapse-v2.rules, but that could do with moving.

@richvdh richvdh mentioned this issue Oct 27, 2020
Merged
@jdreichmann
Copy link
Contributor

If i see it correctly, the last 5 rules in the list are not in the synapse-v2.rules, is that possible?

jdreichmann added a commit to jdreichmann/synapse that referenced this issue Nov 23, 2020
@jdreichmann
Add missing prometheus rules for persisted events
93f19b5 
The official dashboard uses data from these rules, but they were never
added to the synapse-v2.rules. They are mentioned in
[this issue](matrix-org#7917 (comment)),
but never got added to the rules.

Adding them results in all graphs in the "Event persist rate" section
to function as intended.
@jdreichmann jdreichmann mentioned this issue Nov 23, 2020
Merged
4 tasks
anoadragon453 pushed a commit that referenced this issue Dec 2, 2020
@jdreichmann
Add missing prometheus rules for persisted events (#8802)
0fed46e 
The official dashboard uses data from these rules, but they were never added to the synapse-v2.rules. They are mentioned in this issue: #7917 (comment), but never got added to the rules.

Adding them results in all graphs in the "Event persist rate" section to function as intended.

Signed-off-by: Johanna Dorothea Reichmann <[email protected]>
@jdreichmann
Copy link
Contributor

They should all be documented now - so this issue is resolved?

@richvdh
Copy link
Member Author

richvdh commented Dec 7, 2020

yeah I think it is. Thanks!

@richvdh richvdh closed this as completed Dec 7, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
A-Docs things relating to the documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@anoadragon453 @richvdh @jdreichmann