Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalize schema retriever (#291) #294

Merged
merged 1 commit into from
Aug 25, 2020
Merged

Conversation

stoynov96
Copy link
Collaborator

@stoynov96 stoynov96 commented Aug 25, 2020

Already reviewed when merging into generalize-feature

#291

* Change Bucket Clearer

* Add RegexRouter Transform

* Add regexrouter transform

* Change topicToTableResolver Methods

* Change EnsureExistingTables Method

* Change SchemaRegistrySchemaRetriever

* Add RecordNameStrategy

* change standalone properties

* Delete TopicToTableResolver and TopicToTableResolverTest class

* Generalise schemaRetriever interface

* Change schemaRegistrySchemaRetriever

* Change TableClearer Class to clear all tables in a Dataset

* Add IdentitySchemaRetriever Class

* Change BigQuerySinkTask

* Remove TOPIC_TO_TABLES config from BigQuerySinkConfig class

* Modify BigQuerySinkConnectorTest and BigQuerySinkTaskTest

* Delete SchemaRegistrySchemaRetriever Class

* Delete MemorySchemaRetriever Class

* Change Writer Classes

* Remove extra space

* Modify bigQuerySinkTaskTest

* Change SchemaManagerTest Class

* Fix unit tests, make changes acc to code review

* Minor fixes

* Fixes according to code review

* Add javadoc

* Modify SMT
@codecov-commenter
Copy link

codecov-commenter commented Aug 25, 2020

Codecov Report

Merging #294 into master will decrease coverage by 4.76%.
The diff coverage is 45.68%.

@@             Coverage Diff              @@
##             master     #294      +/-   ##
============================================
- Coverage     70.87%   66.10%   -4.77%     
+ Complexity      301      267      -34     
============================================
  Files            32       32              
  Lines          1538     1484      -54     
  Branches        164      152      -12     
============================================
- Hits           1090      981     -109     
- Misses          390      450      +60     
+ Partials         58       53       -5     
Impacted Files Coverage Δ Complexity Δ
.../kafka/connect/bigquery/BigQuerySinkConnector.java 78.37% <ø> (+10.45%) 11.00 <0.00> (-3.00) ⬆️
...ect/bigquery/retrieve/IdentitySchemaRetriever.java 0.00% <0.00%> (ø) 0.00 <0.00> (?)
...om/wepay/kafka/connect/bigquery/SchemaManager.java 28.28% <4.28%> (-48.19%) 6.00 <0.00> (-3.00)
...afka/connect/bigquery/write/row/GCSToBQWriter.java 80.00% <25.00%> (ø) 13.00 <1.00> (ø)
...ect/bigquery/write/row/AdaptiveBigQueryWriter.java 39.68% <40.00%> (-0.95%) 6.00 <1.00> (ø)
...wepay/kafka/connect/bigquery/BigQuerySinkTask.java 57.21% <48.57%> (-1.74%) 27.00 <7.00> (ø)
...ka/connect/bigquery/utils/SinkRecordConverter.java 66.66% <66.66%> (ø) 3.00 <3.00> (?)
...onnect/bigquery/config/BigQuerySinkTaskConfig.java 95.38% <75.00%> (-0.21%) 14.00 <0.00> (ø)
...ka/connect/bigquery/config/BigQuerySinkConfig.java 78.94% <90.47%> (-2.83%) 16.00 <1.00> (-13.00)
...nect/bigquery/write/batch/GCSBatchTableWriter.java 86.20% <100.00%> (-0.46%) 3.00 <2.00> (ø)
... and 5 more

@stoynov96 stoynov96 merged commit ae30437 into master Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants