Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[yugabyted] connection management metrics integration with yugabyted UI #18805

Closed
1 task done
nchandrappa opened this issue Aug 22, 2023 · 0 comments
Closed
1 task done
Assignees
Labels
area/ecosystem Label for all ecosystem related projects kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@nchandrappa
Copy link
Contributor

nchandrappa commented Aug 22, 2023

Jira Link: DB-7685

Description

Introduction

The connection management project enables the server-side pooling in the YugabyteDB cluster to improve client connection utilization in low-resource environments and handle large number of connections when the YugabyteDB is stressed with a burst of incoming client connections. Refer to #17599 for more details.

Connection Management Metrics

Connection management feature provides a host of metrics to show the connection utilization by reporting the active physical and logical connections, number of connections queued and avg time spent by the connections in the queue. Below is a sample metrics response reported in tserver-ip:13000/connections endpoint -

{
    "pools": [
        {
            "pool": "global",
            "active_logical_connections": 0,
            "queued_logical_connections": 0,
            "pending_logical_connections": 0,
            "active_physical_connections": 0,
            "idle_physical_connections": 0,
            "avg_wait_time": 0,
            "count_query": 0,
            "count_transactions": 0
        },
        {
            "pool": "control",
            "active_logical_connections": 0,
            "queued_logical_connections": 0,
            "pending_logical_connections": 0,
            "active_physical_connections": 0,
            "idle_physical_connections": 0,
            "avg_wait_time": 0,
            "count_query": 0,
            "count_transactions": 0
        }
    ]
  • Each server reports metrics related to the current client connections load
  • :13000/connections endpoint reports load stats for each database in the cluster

UI Integration Tasks

Status UI Page Description
Overview page Add a badge to the cluster overview page to display connection management is enabled on the cluster
Nodes Page When connection management is enabled, connection sections should display the active physical connections and active logical connections for each node
Performance page Add a connections tab, display a stacked graph to show the currently active connections and total connections
Alerts page TBD

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@nchandrappa nchandrappa added area/ecosystem Label for all ecosystem related projects status/awaiting-triage Issue awaiting triage area/ybd yugabyted project related Github tickets. labels Aug 22, 2023
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue and removed status/awaiting-triage Issue awaiting triage area/ybd yugabyted project related Github tickets. labels Aug 22, 2023
djiang9001 added a commit that referenced this issue Nov 2, 2023
…shotter

Summary:
Added some ysql connection manager stats to the tserver metrics snapshotter.
The metrics are enabled when `ysql_conn_mgr` is added to the `metrics_snapshotter_tserver_metrics_whitelist` gflag.
The recorded metrics are the total number of logical connections and total number of physical connections.

In order to access the shared memory where the connection manager stats are stored from the tserver, a shared memory key is needed.
This key is passed in to the tserver and stored in a new member variable.
Jira: DB-7685

Test Plan: no test plan

Reviewers: nikhil, hsunder

Reviewed By: hsunder

Subscribers: hsunder, janand, ybase, yugabyted-dev, bogdan, djiang

Differential Revision: https://phorge.dev.yugabyte.com/D29618
djiang9001 added a commit that referenced this issue Nov 7, 2023
…if connection manager is enabled in tserver flags

Summary:
Currently connection manager is not enabled by default in yugabyted, so we should only add the ysql_conn_mgr flag to the metrics whitelist if it is enabled.
Jira: DB-7685

Test Plan: no test plan

Reviewers: nikhil

Reviewed By: nikhil

Subscribers: yugabyted-dev, djiang

Differential Revision: https://phorge.dev.yugabyte.com/D30025
djiang9001 added a commit that referenced this issue Nov 14, 2023
…gabyted UI

Summary:
- Implemented /connections api endpoint for go API.
- Add total logical/physical connections metric to the api server, taken from metrics snapshotter table system.metrics.

If connection manager is enabled
- Display clickable banner that brings you to the connections page.
- Show connections page that shows total logical/physical connections graph.
- Show active logical/physical connections columns in nodes page.
Jira: DB-7685

Test Plan: no test plan

Reviewers: nikhil

Reviewed By: nikhil

Subscribers: yugabyted-dev, djiang

Differential Revision: https://phorge.dev.yugabyte.com/D28225
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ecosystem Label for all ecosystem related projects kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Development

No branches or pull requests

3 participants