Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cloud availability updater: automatic PR creation #22568

Merged
merged 6 commits into from
Feb 8, 2023
Merged

cloud availability updater: automatic PR creation #22568

merged 6 commits into from
Feb 8, 2023

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Feb 8, 2023 •
edited
Loading

What

Closes https://github.com/airbytehq/airbyte-internal-issues/issues/3777
We want to automatically open PR on our internal platform repo for each connector eligible for Cloud.

How

  • Implement calls to GitHub API to create a PR and check if one already exists for the same branch.
  • Implement deploy_eligible_connectors_to_cloud_repo which:
    1. Clones the internal platform repo to a tmp dir
    2. Iterate on eligible connectors to deploy these to cloud (the final step of the deployment is actually a PR creation on the platform internal repo)
  • Provide the run-qa-engine github action workflow with the required env var to perform PR creation.
  • Disable the automatic scheduling of this github action workflow. We'll turn it on again once all QA checks are implemented.

An example of the automatically created PR is available here.

Recommended reading order

  1. airbyte/tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py

    Line 145 in 97b34ef

    def deploy_eligible_connectors_to_cloud_repo(eligible_connectors: Iterable):

  2. airbyte/tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py

    Line 121 in 97b34ef

    def deploy_new_connector_to_cloud_repo(airbyte_cloud_repo_path: Path, airbyte_cloud_repo: git.Repo, connector: ConnectorQAReport):

  3. airbyte/tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py

    Line 97 in 97b34ef

    def create_pr(connector: ConnectorQAReport, branch: str) -> Optional[requests.Response]:

  4. airbyte/tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py

    Line 87 in 97b34ef

    def pr_already_created_for_branch(head_branch: str) -> bool:

  5. airbyte/.github/workflows/run-qa-engine.yml

    Line 1 in 97b34ef

    name: Run QA Engine

🚨 User Impact 🚨

None as the Run QA Engine workflow scheduling is disabled.

@alafanechere alafanechere force-pushed the augustin/cloud-availability-updater-pr-creation branch from e59fbde to b9910e2 Compare February 8, 2023 16:06
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:08 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:08 — with GitHub Actions Inactive
@github-actions
Copy link
Contributor

github-actions bot commented Feb 8, 2023

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage 24.66% ❌

@alafanechere alafanechere force-pushed the augustin/cloud-availability-updater-pr-creation branch from b9910e2 to 97b34ef Compare February 8, 2023 16:13
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:15 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:16 — with GitHub Actions Inactive
@alafanechere alafanechere marked this pull request as ready for review February 8, 2023 16:23
@alafanechere alafanechere requested a review from a team February 8, 2023 16:23
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:24 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 16:24 — with GitHub Actions Inactive
@evantahler
Copy link
Contributor

@alafanechere is there an example PR you have made that demonstrates this? That's the unit test for this PR :D

@alafanechere
Copy link
Contributor Author

@evantahler yep, this PR https://github.com/airbytehq/airbyte-platform-internal/pull/4388

evantahler
evantahler reviewed Feb 8, 2023
View reviewed changes
.github/workflows/run-qa-engine.yml
Comment on lines +5 to +8
# schedule:
## 1pm UTC is 6am PDT.
## same time as Generate Build Report
# - cron: "0 13 * * *"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 disabled for now

tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py Show resolved Hide resolved
tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py
Comment on lines +30 to +32
return git.Repo.clone_from(
AIRBYTE_PLATFORM_INTERNAL_GITHUB_REPO_URL, local_repo_path, branch=AIRBYTE_PLATFORM_INTERNAL_MAIN_BRANCH_NAME
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess we have to clone the platform repo to open a PR

tools/ci_connector_ops/ci_connector_ops/qa_engine/cloud_availability_updater.py Outdated Show resolved Hide resolved
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:15 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:15 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:34 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:34 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:42 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 17:42 — with GitHub Actions Inactive
@alafanechere alafanechere enabled auto-merge (squash) February 8, 2023 17:54
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 21:30 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets February 8, 2023 21:30 — with GitHub Actions Inactive
@alafanechere alafanechere merged commit d918385 into master Feb 8, 2023
@alafanechere alafanechere deleted the augustin/cloud-availability-updater-pr-creation branch February 8, 2023 22:13
@github-actions github-actions bot mentioned this pull request Feb 9, 2023
Closed
danidelvalle pushed a commit to danidelvalle/airbyte that referenced this pull request Feb 9, 2023
@alafanechere @danidelvalle-frontiers
cloud availability updater: automatic PR creation (airbytehq#22568)
dc835e3
This was referenced Feb 10, 2023
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@evantahler evantahler evantahler approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alafanechere @evantahler