Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Server side connection management #17599

Open
kneeraj opened this issue May 30, 2023 · 0 comments
Open

[YSQL] Server side connection management #17599

kneeraj opened this issue May 30, 2023 · 0 comments
Assignees
Labels
area/ecosystem Label for all ecosystem related projects area/ysql Yugabyte SQL (YSQL)

Comments

@kneeraj
Copy link
Contributor

kneeraj commented May 30, 2023

Description

Jira Link: DB-5180

Design Doc: WIP. To be shared.

The server-side connection management feature greatly enhances the capacity of a YugabyteDB cluster to handle a significantly larger number of connections compared to the current limitations. Moreover, this feature enables efficient routing of queries to the appropriate nodes, eliminating unnecessary node hopping and resulting in reduced latency during query execution. It needs to be ensured that addition of such values does, neither adversely affect the performance of the database nor introduces new sql limitations in YugabyteDB.

The key functional and related non functional goals are:

  • Scaling large number of database connections
  • Database aware query routing
  • No regression in terms of sql limitations
  • Non functional performance goals
    • Marginal performance drop in single connection latency
    • Idle connections should not adversely affect the active workload performances
    • Increasing connections should not lead to exponential degradation in performance

Phase 1 - Scaling and non functional performance requirements

Feature Status Comments
Source code and build Integration of Odyssey with YugabyteDB ⬜️ Odyssey being used as a base product which will be modified as per YugabyteDB's need
Start cluster with connection management enabled ⬜️ Startup flags to opt-in or opt-out. Default is to not start the connection manager for now
Using single server connection pool for multiple database/user combination ⬜️ Ensuring error free context switching on the pooled server connection when database/user combination changes
Security passthrough ⬜️ Modify Odyssey to get authentication done on the database side instead of on the Odyssey
Ensure zero sql limitation ⬜️ Upstream Odyssey has certain sql feature limitation in transaction mode. YugabyteDB enhances Odyssey to eliminate such limitations

Phase 2 - Support for query routing and Metrics

Feature Status Comments
Database aware query routing ⬜️ For colocated databases redirect the queries to the node hosting the leader tablet for that database
Metrics ⬜️ Connection manager's usage, performance and health related stats

Phase 3 - Memory tracking

Feature Status Comments
Memory tracking ⬜️ Track memory usage of the connection manager and enforce memory usage limits
@kneeraj kneeraj added area/ysql Yugabyte SQL (YSQL) area/ecosystem Label for all ecosystem related projects status/awaiting-triage Issue awaiting triage labels May 30, 2023
@kneeraj kneeraj self-assigned this May 30, 2023
@kneeraj kneeraj changed the title [YSQL] Server side connection management <WIP> [YSQL] Server side connection management May 30, 2023
@kneeraj kneeraj removed the status/awaiting-triage Issue awaiting triage label May 31, 2023
@premkumr premkumr assigned suranjan and unassigned kneeraj Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ecosystem Label for all ecosystem related projects area/ysql Yugabyte SQL (YSQL)
Projects
None yet
Development

No branches or pull requests

2 participants