Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking issues for cloud storage sink #6797

Closed
1 task done
zhaoxinyu opened this issue Aug 18, 2022 · 4 comments
Closed
1 task done

Tracking issues for cloud storage sink #6797

zhaoxinyu opened this issue Aug 18, 2022 · 4 comments
Labels
affects-6.5 area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. type/enhancement The issue or PR belongs to an enhancement.

Comments

@zhaoxinyu
Copy link
Contributor

zhaoxinyu commented Aug 18, 2022 •
edited by CharlesCheung96
Loading

Add a cloud storage sink to connect TiCDC with different cloud storage systems such as AWS S3, GCS or Azure Blob Storage, etc.

related issues
@zhaoxinyu zhaoxinyu mentioned this issue Aug 18, 2022
Merged
@ti-chi-bot ti-chi-bot added area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. labels Aug 18, 2022
@zhaoxinyu
Copy link
Contributor Author

/type feature

@ti-chi-bot ti-chi-bot added the type/feature Issues about a new feature label Aug 18, 2022
ti-chi-bot pushed a commit that referenced this issue Aug 22, 2022
@zhaoxinyu
sink/(ticdc): Restructure codec files (#6796)
f8f1df2 
ref #6797
@zhaoxinyu zhaoxinyu mentioned this issue Aug 31, 2022
Merged
@zhaoxinyu zhaoxinyu mentioned this issue Sep 13, 2022
Merged
@zhaoxinyu zhaoxinyu mentioned this issue Sep 26, 2022
Merged
@zhaoxinyu zhaoxinyu mentioned this issue Oct 9, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 10, 2022
@zhaoxinyu
sink/(ticdc): Add csv encoder into codec (#6942)
13e6f48 
ref #6797
3AceShowHand pushed a commit to 3AceShowHand/tiflow that referenced this issue Oct 10, 2022
@zhaoxinyu @3AceShowHand
sink/(ticdc): Add csv encoder into codec (pingcap#6942)
40be734 
ref pingcap#6797
ti-chi-bot pushed a commit that referenced this issue Oct 10, 2022
@zhaoxinyu
sink(ticdc): Use TableInfo instead of SimpleTableInfo to hold mor…
eb03362 
…e information in `DDLEvent` (#7221)

ref #6797
@zhaoxinyu zhaoxinyu mentioned this issue Oct 13, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 18, 2022
@zhaoxinyu
sink(ticdc): Add csv decoder into codec (#7352)
b3d1782 
ref #6797
This was referenced Oct 20, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 20, 2022
@zhaoxinyu
sink/(ticdc): Use TableInfo instead of TableName when emitting ch…
8fbe75f 
…eckpoint ts (#7296)

ref #6797
ti-chi-bot pushed a commit that referenced this issue Oct 21, 2022
@zhaoxinyu
sink(ticdc): Dispatch DDL events to cloud storage sink (#7416)
438e042 
ref #6797
@zhaoxinyu zhaoxinyu mentioned this issue Oct 28, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Oct 31, 2022
@zhaoxinyu
sink/(ticdc): Support building TableInfo from schema file in cloud …
a9c5dca 
…storage (#7415)

ref #6797
ti-chi-bot pushed a commit that referenced this issue Oct 31, 2022
@zhaoxinyu
sink/(ticdc): Dispatch DML events to cloud storage sink (#7048)
5fed59a 
ref #6797
This was referenced Nov 2, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Nov 2, 2022
@zhaoxinyu
sink(ticdc): Fix several issues in cloud storage sink (#7517)
915ad30 
ref #6797
ti-chi-bot pushed a commit that referenced this issue Nov 3, 2022
@zhaoxinyu
sink(ticdc): Add integration test for cloud storage sink (#7518)
ad8ed62 
ref #6797
ti-chi-bot pushed a commit that referenced this issue Nov 3, 2022
@zhaoxinyu
consumer/(ticdc): Add a consumer for cloud storage sink (#7481)
13c2fb9 
ref #6797
This was referenced Nov 8, 2022
Merged
Merged
ti-chi-bot pushed a commit that referenced this issue Nov 15, 2022
@zhaoxinyu
sink(ticdc): Support canal-json protocol in cloud storage sink (#7563)
f841ae6 
ref #6797
ti-chi-bot pushed a commit that referenced this issue Nov 28, 2022
@zhaoxinyu
sink/(ticdc): Simplify and optimize some logic in cloud storage sink (#…
7f27730 
…7547)

ref #6797, close #7724
@zhaoxinyu zhaoxinyu mentioned this issue Nov 28, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Nov 29, 2022
@zhaoxinyu
sink(ticdc): Add metrics for cloud storage sink (#7735)
5cbc97f 
ref #6797
@IANTHEREAL IANTHEREAL mentioned this issue Nov 30, 2022
Merged
15 tasks
@zhaoxinyu zhaoxinyu mentioned this issue Dec 1, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 1, 2022
@zhaoxinyu
sink(ticdc): Fix a slice overwriting problem (#7759)
9ef9702 
ref #6797
@zhaoxinyu zhaoxinyu mentioned this issue Dec 7, 2022
Merged
ti-chi-bot pushed a commit that referenced this issue Dec 8, 2022
@zhaoxinyu
sink(ticdc): Add DDL query into schema.json file (#7834)
215bf26 
ref #6797
@ti-chi-bot ti-chi-bot mentioned this issue Dec 8, 2022
Merged
ti-chi-bot added a commit that referenced this issue Dec 8, 2022
@ti-chi-bot
sink(ticdc): Add DDL query into schema.json file (#7834) (#7848)
3693dc1 
ref #6797, close #7847
@Frank945946 Frank945946 mentioned this issue Mar 13, 2023
Merged
15 tasks
@pingcap pingcap deleted a comment from zhaoxinyu Mar 14, 2023
@CharlesCheung96 CharlesCheung96 mentioned this issue Mar 15, 2023
Merged
ti-chi-bot pushed a commit that referenced this issue Mar 31, 2023
@CharlesCheung96
doc(ticdc): add design document for storage sink (#8540)
96ac281 
ref #6797
@ti-chi-bot ti-chi-bot mentioned this issue Mar 31, 2023
Closed
3AceShowHand pushed a commit to 3AceShowHand/tiflow that referenced this issue Apr 3, 2023
@CharlesCheung96 @3AceShowHand
doc(ticdc): add design document for storage sink (pingcap#8540)
23127bf 
ref pingcap#6797
@ti-chi-bot ti-chi-bot mentioned this issue Apr 3, 2023
Merged
@nongfushanquan
Copy link
Contributor

/label affects-6.5

@CharlesCheung96 CharlesCheung96 added type/enhancement The issue or PR belongs to an enhancement. and removed type/feature Issues about a new feature labels Apr 4, 2023
@nongfushanquan
Copy link
Contributor

/close

@ti-chi-bot
Copy link
Member

@nongfushanquan: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-6.5 area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zhaoxinyu @CharlesCheung96 @ti-chi-bot @nongfushanquan