Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bechurch/21720 versioned connector build status #22029

Merged
merged 15 commits into from
Jan 31, 2023

Conversation

bnchrch
Copy link
Contributor

@bnchrch bnchrch commented Jan 27, 2023

What

Closes #21720

This issue needs the connector version alongside the build status for each connector.

This is so that we can proactively mark connectors as "Ready for cloud"

How

This accomplishes it with the following changes

  1. Retrieve the version from the dockerfile in the report.sh script
  2. Use the version to output the test artifacts at $CONNECTOR/$VERSION
  3. Consume those artifacts in the qa_engine to determine build status and version

What else

We've made an additional effort in this PR to ensure that

  1. All build logs for connectors are recorded to tests/history/connectors not tests/history. (Currently they are being recorded to both)
  2. The relevant build statuses point to tests/history/connectors

Recommended reading order

  1. tools/status/report.sh:write_job_log
  2. tools/status/report.sh:generate_job_log_json
  3. tools/ci_connector_ops/ci_connector_ops/qa_engine/validations.py:latest_build_is_successful
  4. tools/ci_connector_ops/ci_connector_ops/qa_engine/inputs.py:fetch_latest_build_status_for_connector_version
  5. tools/bin/build_report.py

🚨 User Impact 🚨

No breaking change if successful.

However, there is a risk here that a part of the system is made stale. This is because we've made an effort to standardize the build report path, which means that theres a chance we missed a system that still references the old path.

Though thats low risk low impact

Pre-merge Checklist

Expand the relevant checklist and delete the others.

  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • PR name follows PR naming conventions
  • Build is successful
  • /test connector=connectors/<name> command is passing
  • Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Post merge

  • run build_report workflow
  • run qa-engine workflow

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 20:59 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 20:59 — with GitHub Actions Inactive
@github-actions
Copy link
Contributor

github-actions bot commented Jan 27, 2023

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage 24.47%

@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 22:01 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 22:01 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 23:23 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 23:25 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 23:25 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 27, 2023 23:27 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 00:10 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 00:10 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 00:11 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 00:37 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 00:37 — with GitHub Actions Inactive
@octavia-squidington-iv octavia-squidington-iv added the area/connectors Connector related issues label Jan 28, 2023
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 01:49 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 01:49 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 01:50 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 02:09 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 02:09 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 02:10 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 03:01 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 28, 2023 03:01 — with GitHub Actions Inactive
@bnchrch bnchrch force-pushed the bechurch/21720-versioned-connector-build-status branch from 5ffecaf to 3e1547f Compare January 30, 2023 15:29
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 15:30 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 15:31 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 15:37 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 15:37 — with GitHub Actions Inactive
@bnchrch bnchrch marked this pull request as ready for review January 30, 2023 15:45
@evantahler
Copy link
Contributor

Screenshot 2023-01-30 at 1 23 04 PM

Github hack - if you use the phrase "closes #ticket" it will link up the PR and Issue, and close the issue when the PR is merged.

https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue

Copy link
Contributor

@evantahler evantahler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work Ben!

airbyte-integrations/builds.md Show resolved Hide resolved
@@ -42,5 +42,23 @@ full_path_to_gradle_path() {
python -c "print(':airbyte-' + ':'.join(p for p in '${path}'.split('airbyte-')[-1].replace('/', ':').split(':') if p))"
}

get_connector_path_from_name() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

tools/status/report.sh Show resolved Hide resolved
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 23:30 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 23:31 — with GitHub Actions Inactive
@bnchrch bnchrch enabled auto-merge (squash) January 30, 2023 23:32
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 23:33 — with GitHub Actions Inactive
@bnchrch bnchrch temporarily deployed to more-secrets January 30, 2023 23:33 — with GitHub Actions Inactive
@bnchrch bnchrch disabled auto-merge January 30, 2023 23:36
@bnchrch
Copy link
Contributor Author

bnchrch commented Jan 30, 2023

/test connector=connectors/source-faker

🕑 connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/4049185710
❌ connectors/source-faker https://github.com/airbytehq/airbyte/actions/runs/4049185710
🐛 https://gradle.com/s/7sb7wa3tx6rvo

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/unit_test.py::test_read_small_random_data - AttributeError:...
	 FAILED unit_tests/unit_test.py::test_no_read_limit_hit - AttributeError: 'Moc...
	 FAILED unit_tests/unit_test.py::test_read_big_random_data - AttributeError: '...
	 FAILED unit_tests/unit_test.py::test_with_purchases - AttributeError: 'MockLo...
	 FAILED unit_tests/unit_test.py::test_sync_ends_with_limit - AttributeError: '...
	 FAILED unit_tests/unit_test.py::test_read_with_seed - AttributeError: 'MockLo...
	 �[31m========================= �[31m�[1m6 failed�[0m, �[32m2 passed�[0m�[31m in 0.95s�[0m�[31m ==========================�[0m

@bnchrch
Copy link
Contributor Author

bnchrch commented Jan 30, 2023

/test connector=connectors/source-pokeapi

🕑 connectors/source-pokeapi https://github.com/airbytehq/airbyte/actions/runs/4049294944
❌ connectors/source-pokeapi https://github.com/airbytehq/airbyte/actions/runs/4049294944
🐛

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestSpec.test_config_match_spec: The spec is currently invalid: it has additionalProperties set to false
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: This connector does not support incremental syncs.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
======================== 12 passed, 3 skipped in 8.82s =========================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

QA Engine: retrieve connector build status
3 participants