Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cdc: provide core changefeed data stream using notifications rather than results #59390

Open
ajwerner opened this issue Jan 25, 2021 · 1 comment
Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@ajwerner
Copy link
Contributor

ajwerner commented Jan 25, 2021

Is your feature request related to a problem? Please describe.

Today the EXPERIMENTAL CHANGEFEED works by effectively creating a query in the postgres protocol with an infinite result set. This works in a relatively ergonomic way for clients but it has its downsides. One major downside is that closing the changefeed requires closing the connection. This is primarily true because the postgres protocol does not permit sending or processing messages until all of the results have been consumed. This is also a problem because cockroach does not implement the postgres cancelation protocol (#41335).

At the time when changefeeds were implemented, cockroach had no support for asynchronous notification. Using async notifications would be especially handy in order to permit multiplexing multiple changefeeds over a single connection. Such a thing would become much more handy if changefeeds supported watching portions of tables.

Jira issue: CRDB-3295

@ajwerner ajwerner added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-cdc Change Data Capture labels Jan 25, 2021
@miretskiy
Copy link
Contributor

A bit of a stale issue; changefeeds now do support watching portions of tables;
@ajwerner do you still feel such a change would be useful?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

No branches or pull requests

2 participants