Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix empty airbyte source check #14509

Merged
merged 2 commits into from
Jul 7, 2022
Merged

Conversation

alovew
Copy link
Contributor

@alovew alovew commented Jul 7, 2022

We are doing a check for resets with LEGACY state to ensure that all streams in a Catalog are being reset. The existing check just checked that streams_to_reset == streams_in_catalog. But since it's ok for streams that are NOT in the catalog to be reset, what we actually want is streams_to_reset.containsAll(streams_in_catalog).

This PR updates the logic, renames a few things, and adds an extra test for this case.

@github-actions github-actions bot added area/platform issues related to the platform area/worker Related to worker labels Jul 7, 2022
@@ -302,8 +302,10 @@ public void testPerStreamWithMissingState() throws Exception {
Assertions.assertThat(emptyAirbyteSource.isFinished()).isTrue();
}

// In the LEGACY state, if the list of streams passed in to be reset does not include every stream
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me know if these comments are confusingly written. just thought the tests should have some explanation.

@alovew alovew requested review from lmossman and gosusnp July 7, 2022 21:14
Copy link
Contributor

@lmossman lmossman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@alovew alovew temporarily deployed to more-secrets July 7, 2022 21:46 Inactive
@alovew alovew merged commit 19c33b0 into master Jul 7, 2022
@alovew alovew deleted the anne/fix-empty-airbyte-source-check branch July 7, 2022 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/platform issues related to the platform area/worker Related to worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants