-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BACKPORT 2.12] [CDCSDK] [#9019] CDC SDK Client API and Java Console …
…Subscriber Summary: Original commits: - d294abf / D13836 - 6b15b16 / D15860 - cf5fead / D16057 - 5408e30 / D15989 - 53afee99e7cfe912fa7ec579ce470e999701a074 / D16176 Github Master Ticket: #9019 Design DocumentL https://docs.google.com/document/d/1_xZqU5UgzCu1W--kci3ajU7_iYXXHMQvudmybDI-Xsk/edit Functional Spec: https://docs.google.com/document/u/2/d/1nHuzHQ-qYVPbKi2dqo_drzSXMq00h7w5oi0JDf0GD1U/edit#heading=h.jmqfs7jgvvg8 This is the client-side change that exposes some APIs to be consumed by CDC consumers. Currently, these APIs are not public and are to be consumed by our Debezium connector. For testing purposes, we have written a console subscriber for testing purposes. [CDCSDK][#11679] Add missing license headers to Java files The following files have missing license headers: 1. `CreateCDCStreamRequest.java` 2. `GetCheckpointRequest.java` 3. `GetCheckpointResponse.java` 4. `GetDBStreamInfoRequest.java` 5. `GetDBStreamInfoResponse.java` 6. `SetCheckpointRequest.java` 7. `SetCheckpointResponse.java` This diff adds these missing headers. [#11779][CDCSDK] Add option to send a DDL record based on a flag value in GetChangesRequest Before this, the issue was that if for a stream ID, some data was consumed and a client comes up with the same stream ID and requests for changes, it will only receive the changes. Now the issue with this was with `Debezium` that when the connector was restarted, it directly received the changes without any DDL record, this DDL record was essential for Debezium since it was used to process the schema info for the columns in Debezium and in case it was not there, it lead to a `NullPointerException` on the client side, thus causing a connector crash effectively. [#11729][DocDB][xCluster] Fix for replication not working if user upgrades to a branch with CDCSDK code changes With the changes for CDCSDK, we have separate `source_type` values i.e. `XCLUSTER` for xCluster replication and `CDCSDK` for the new changes. Similarly there is another option i.e. `checkpoint_type` which can have `IMPLICIT` and `EXPLICIT` values. If a stream for replication has been created before upgrading, it was unable to continue replication after upgrading to the latest version since the `source_type` and `checkpoint_type` options were missing from it as it has only been introduced with the CDCSDK changes only. Test Fixes for 2.12 Test Plan: Jenkins: skip Unit tests in java for APIs and CDC behavior. We have done some long-running testing with applications. We have also run the YB-sample apps and enabled CDC on the table. Verified that all the events are received. Tested the complete CDC with Debezium pipeline with the specified change. Command to run test: `ybd --cxx-test integration-tests_cdcsdk_ysql-test --gtest_filter CDCSDKYsqlTest.TestNeedSchemaInfoFlag` * Manually tested with a custom build on dev portal Reviewers: nicolas, bogdan, ybase, rahuldesirazu, sdash, iamoncar, zyu, jhe, mkantimath, sergei Reviewed By: sergei Differential Revision: https://phabricator.dev.yugabyte.com/D16251
- Loading branch information
Showing
85 changed files
with
8,713 additions
and
451 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.