Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate read and write path for leveldb sorter #4631

Closed
Tracked by #3227
overvenus opened this issue Feb 19, 2022 · 0 comments · Fixed by #4712
Closed
Tracked by #3227

Separate read and write path for leveldb sorter #4631

overvenus opened this issue Feb 19, 2022 · 0 comments · Fixed by #4712
Labels
area/ticdc Issues or PRs related to TiCDC. component/sorter Sorter component. subject/performance Denotes an issue or pull request is related to replication performance. type/enhancement The issue or PR belongs to an enhancement.

Comments

@overvenus
Copy link
Member

overvenus commented Feb 19, 2022 •
edited
Loading

Separate read and write path to improve read and write throughput.

We plan to add two actors reader and writer, and let table sorter be a thin shim, which delegates AddEntry to writer and Output to reader.
image image
@overvenus overvenus added subject/performance Denotes an issue or pull request is related to replication performance. component/sorter Sorter component. area/ticdc Issues or PRs related to TiCDC. labels Feb 19, 2022
@overvenus overvenus mentioned this issue Feb 19, 2022
Merged
@overvenus overvenus added the type/enhancement The issue or PR belongs to an enhancement. label Feb 23, 2022
ti-chi-bot pushed a commit that referenced this issue Feb 24, 2022
@overvenus
sorter/leveldb(ticdc): replace cleaner with delete range (#4632)
c396df1 
ref #4631
@overvenus overvenus mentioned this issue Feb 24, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Feb 26, 2022
@overvenus
sorter/leveldb(ticdc): separate write from table sorter (#4686)
fa006db 
ref #4631
@overvenus overvenus mentioned this issue Feb 28, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Mar 3, 2022
@overvenus
sorter/leveldb(ticdc): separate read from table sorter (#4712)
fe2ea36 
close #4631
This was referenced Mar 3, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Mar 7, 2022
@overvenus
sorter/leveldb(ticdc): fix duplicate events (#4783)
49f12dc 
close #3381, ref #4631
ti-chi-bot pushed a commit that referenced this issue Mar 7, 2022
@overvenus
sorter(ticdc): rename files for better readability (#4766)
e60c8f0 
ref #4631
@overvenus overvenus mentioned this issue Mar 15, 2022
Closed
16 tasks
overvenus added a commit to overvenus/actor that referenced this issue Jan 11, 2023
@overvenus
sorter/leveldb(ticdc): replace cleaner with delete range (#4632)
89d839c 
ref pingcap/tiflow#4631
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/ticdc Issues or PRs related to TiCDC. component/sorter Sorter component. subject/performance Denotes an issue or pull request is related to replication performance. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sorter/leveldb(ticdc): separate read from table sorter overvenus/ticdc
1 participant
@overvenus