Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Oracle source: emit state messages more frequently #16238

Merged
merged 8 commits into from
Sep 1, 2022
Merged

🎉 Oracle source: emit state messages more frequently #16238

merged 8 commits into from
Sep 1, 2022

Conversation

tuliren
Copy link
Contributor

@tuliren tuliren commented Sep 1, 2022

What

🚨 User Impact 🚨

  • No negative impact.
  • When there is a large amount of data to sync and a failure in the middle, the next sync attempt won't need to start from scratch.

@github-actions github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Sep 1, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-oracle-strict-encrypt
  • source-postgres-strict-encrypt

@tuliren
Copy link
Contributor Author

tuliren commented Sep 1, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-oracle

🕑 connectors/source-oracle https://github.com/airbytehq/airbyte/actions/runs/2974006649
✅ connectors/source-oracle https://github.com/airbytehq/airbyte/actions/runs/2974006649
No Python unittests run

Build Passed

Test summary info:

All Passed

@tuliren
Copy link
Contributor Author

tuliren commented Sep 1, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-oracle-strict-encrypt

🕑 connectors/source-oracle-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2974008299
✅ connectors/source-oracle-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/2974008299
No Python unittests run

Build Passed

Test summary info:

All Passed

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-oracle-strict-encrypt
  • source-postgres-strict-encrypt

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-postgres-strict-encrypt
  • source-oracle-strict-encrypt

@tuliren tuliren temporarily deployed to more-secrets September 1, 2022 18:21 Inactive
tuliren
tuliren commented Sep 1, 2022
View reviewed changes
.github/workflows/slash-commands.yml Outdated
@@ -18,7 +18,7 @@ jobs:
id: scd
uses: peter-evans/slash-command-dispatch@v2
with:
token: ${{ secrets.DAVINCHIA_PAT }}
token: ${{ secrets.SUPERTOPHER_PAT }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to revert this before merging.

@tuliren tuliren marked this pull request as ready for review September 1, 2022 19:17
@tuliren tuliren requested a review from a team as a code owner September 1, 2022 19:17
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-postgres-strict-encrypt
  • source-oracle-strict-encrypt

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-oracle-strict-encrypt
  • source-postgres-strict-encrypt

@tuliren tuliren temporarily deployed to more-secrets September 1, 2022 19:20 Inactive
@tuliren
Copy link
Contributor Author

tuliren commented Sep 1, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-oracle

🕑 Publishing the following connectors:
connectors/source-oracle
https://github.com/airbytehq/airbyte/actions/runs/2974365064


Connector Did it publish? Were definitions generated?
connectors/source-oracle ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@tuliren
Copy link
Contributor Author

tuliren commented Sep 1, 2022 •
edited by github-actions bot
Loading

/publish connector=connectors/source-oracle-strict-encrypt

🕑 Publishing the following connectors:
connectors/source-oracle-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/2974365877


Connector Did it publish? Were definitions generated?
connectors/source-oracle-strict-encrypt ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

ryankfu
ryankfu approved these changes Sep 1, 2022
View reviewed changes
Copy link
Contributor

@ryankfu ryankfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@tuliren tuliren merged commit fbac1d3 into master Sep 1, 2022
@tuliren tuliren deleted the liren/state-emission-oracle branch September 1, 2022 21:17
@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

  • source-oracle-strict-encrypt
  • source-postgres-strict-encrypt

@tuliren tuliren temporarily deployed to more-secrets September 1, 2022 21:20 Inactive
@tuliren tuliren mentioned this pull request Sep 2, 2022
Closed
@tuliren tuliren linked an issue Sep 2, 2022 that may be closed by this pull request
Add checkpointing to JDBC Source databases #10909
Closed
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Sep 2, 2022
Merged
robbinhan pushed a commit to robbinhan/airbyte that referenced this pull request Sep 29, 2022
@tuliren @octavia-squidington-iii @robbinhan
🎉 Oracle source: emit state messages more frequently (airbytehq#16238)
30768d5 
* Activate intermediate state emission for oracle

* Bump version

* Use a different pat to avoid api rate limit

* Revert "use a different pat to avoid api rate limit"

This reverts commit aaf4502.

* Use a different pat to avoid api rate limit

This reverts commit d9d741e.

* auto-bump connector version [ci skip]

* Revert "Use a different pat to avoid api rate limit"

This reverts commit bb3430b.

Co-authored-by: Octavia Squidington III <[email protected]>
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022
@tuliren @octavia-squidington-iii @jhammarstedt
🎉 Oracle source: emit state messages more frequently (airbytehq#16238)
e4c29bd 
* Activate intermediate state emission for oracle

* Bump version

* Use a different pat to avoid api rate limit

* Revert "use a different pat to avoid api rate limit"

This reverts commit aaf4502.

* Use a different pat to avoid api rate limit

This reverts commit d9d741e.

* auto-bump connector version [ci skip]

* Revert "Use a different pat to avoid api rate limit"

This reverts commit bb3430b.

Co-authored-by: Octavia Squidington III <[email protected]>
@philippeboyd philippeboyd mentioned this pull request May 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ryankfu ryankfu ryankfu approved these changes

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/oracle connectors/source/oracle-strict-encrypt connectors/source/postgres
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add checkpointing to JDBC Source databases
3 participants
@tuliren @ryankfu @octavia-squidington-iii