Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial PR /publish workflow into pre-merge /build-connector and post-merge /publish-connector workflows #12405

Merged
merged 17 commits into from
Apr 29, 2022

Conversation

noahkawasaki-airbyte
Copy link
Contributor

@noahkawasaki-airbyte noahkawasaki-airbyte commented Apr 27, 2022

What

Experiment doing this #12108 in order to actually iterate on this I need to merge in the initial workflow so Github Actions creates it and then I can invoke it on a second PR and finish iterating and making sure it works.

Note this does not actually touch or change any pre-existing /publish behavior, its all just adding new workflows and functions into manage.sh

TODO:

Add this in next PR in DockerHelpers.groovy

    static String getDevTaggedImage(projectDir, dockerfileName) {
        def imageName = extractImageName(Paths.get(projectDir.absolutePath, dockerfileName).toString())
        def imageVersion = extractImageVersion(Paths.get(projectDir.absolutePath, dockerfileName).toString())

        if (System.getenv("BUILD_CANDIDATE_IMAGE") == "true" && System.getenv("PR_NUMBER") != null) {
            return "${imageName}:${imageVersion}-candidate-${System.getenv("PR_NUMBER")}"
        } else {
            return "${imageName}:dev"
        }
    }

How

Describe the solution

Recommended reading order

  1. x.java
  2. y.python

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 16:23 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 16:23 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/bump-build-test-connector connector=fake

@noahkawasaki-airbyte
Copy link
Contributor Author

/bump-build-test-connector connector=fake

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 16:45 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 16:45 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte marked this pull request as ready for review April 27, 2022 16:47
@noahkawasaki-airbyte
Copy link
Contributor Author

/bump-build-test-connector connector=fake

@noahkawasaki-airbyte
Copy link
Contributor Author

/bump-build-test-connector

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 23:15 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 27, 2022 23:15 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/bump-connector connector=connectors/snowflake

@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/snowflake

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 15:41 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 15:41 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/snowflake

@noahkawasaki-airbyte
Copy link
Contributor Author

noahkawasaki-airbyte commented Apr 28, 2022

/test connector=connectors/destination-snowflake

🕑 connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/2240378979
✅ connectors/destination-snowflake https://github.com/airbytehq/airbyte/actions/runs/2240378979
Python tests coverage:

Name                                                                                                                            Stmts   Miss  Cover
---------------------------------------------------------------------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                                                                                          2      0   100%
normalization/transform_catalog/reserved_keywords.py                                                                               13      0   100%
normalization/transform_catalog/__init__.py                                                                                         2      0   100%
normalization/destination_type.py                                                                                                  13      0   100%
normalization/__init__.py                                                                                                           4      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/airbyte_protocol.py     124      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/__init__.py               1      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/__init__.py                      2      0   100%
normalization/transform_catalog/destination_name_transformer.py                                                                   155      8    95%
normalization/transform_config/transform.py                                                                                       159     31    81%
normalization/transform_catalog/table_name_registry.py                                                                            174     34    80%
normalization/transform_catalog/utils.py                                                                                           34      7    79%
normalization/transform_catalog/dbt_macro.py                                                                                       22      7    68%
normalization/transform_catalog/catalog_processor.py                                                                              147     80    46%
normalization/transform_catalog/transform.py                                                                                       61     38    38%
normalization/transform_catalog/stream_processor.py                                                                               534    345    35%
---------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                                            1447    550    62%

@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/snowflake

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 15:46 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/test-x connector=connectors/destination-snowflake

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 15:48 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 15:48 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/source-nowflake

@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/source-nowflake

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 19:00 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 19:00 Inactive
@noahkawasaki-airbyte
Copy link
Contributor Author

/build-connector connector=connectors/source-nowflake

@noahkawasaki-airbyte
Copy link
Contributor Author

noahkawasaki-airbyte commented Apr 28, 2022

/publish-connector connector=connectors/source-snowflake

Error: Workflow does not have 'workflow_dispatch' trigger

@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 19:07 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 19:07 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 20:33 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 20:33 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte changed the title Separate /publish workflow into pre-merge /build-connector and post-merge /publish-connector workflows Initial PR /publish workflow into pre-merge /build-connector and post-merge /publish-connector workflows Apr 28, 2022
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 20:37 Inactive
@noahkawasaki-airbyte noahkawasaki-airbyte temporarily deployed to more-secrets April 28, 2022 20:37 Inactive
Copy link
Contributor

@Phlair Phlair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ so you can test

@noahkawasaki-airbyte noahkawasaki-airbyte merged commit e8406a2 into master Apr 29, 2022
@noahkawasaki-airbyte noahkawasaki-airbyte deleted the noah/connectors-publish-pipeline branch April 29, 2022 14:03
suhomud pushed a commit that referenced this pull request May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants