Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] Completely deprecate and remove base-python and base-singer packages #6164

Closed
sherifnada opened this issue Sep 17, 2021 · 0 comments · Fixed by #12598
Closed

[EPIC] Completely deprecate and remove base-python and base-singer packages #6164

sherifnada opened this issue Sep 17, 2021 · 0 comments · Fixed by #12598
Assignees

Comments

@sherifnada
Copy link
Contributor

sherifnada commented Sep 17, 2021

Tell us about the problem you're trying to solve

Before releasing the CDK, we relied on the base-python and base-singer packages. However, now that we have the CDK (which still supports singer) there is no reason to keep these packages around. Keeping them around is clutter and increases the build time.

  1. find all packages which depend on base-python (an easy way is to grep for implementation files(project(':airbyte-integrations:bases:base-python').airbyteDocker.outputs) in gradle build files)
  2. Remove the dependency from the gradle file. There are three cases here:
    1. The connector doesn't actually depend on base-python (i.e: it relies on the CDK from PyPi and its Docker image inherits from a python image directly, not an airbyte special image e.g: FROM python:3.7-slim not FROM airbyte/base-integration-python. In this case, removing the line in the gradle build file is sufficient.
    2. The connector actually still relies on the old base python package. You can tell this pretty immediately if the Dockerfile extends one of the special airbyte images. If this is the case, there are one of two possibilities:
      1. The connector basically relies on the CDK but since before it was released to Pypi. In this case migration should be doable by following the steps here.
      2. The connector significantly relies on dead CDK code. In this case, you should still be able to find all the dead CDK code in the deprecated package of the CDK published on Pypi. You'll need to migrate the connector's usages to that subpackage.
  3. PR, publish, checklist, etc...

In every case, make sure to remove the line in the gradle file which depends on the base- package.

For base-singer, do the same but ignore the following modules:

  1. Google Adwords, Google Analytics Singer, Google Search Console singer,
  2. Zoom
  3. Quickbooks
  4. Marketo
    All of these will soon be replaced with non-singer modules and removed soon.

I recommend that a single person lead this effort at first to get a sense of how to perform this task, then parallelize it with someone else on the team as needed.

Important: please create a subticket for each connector you work on to help us keep visibility.

@sherifnada sherifnada added type/enhancement New feature or request area/connectors Connector related issues labels Sep 17, 2021
@sherifnada sherifnada changed the title Completely deprecate and remove base-python and base-singer packages [EPIC] Completely deprecate and remove base-python and base-singer packages Sep 17, 2021
@sherifnada sherifnada added this to the Connectors 2021-10-29 milestone Oct 14, 2021
@yevhenii-ldv yevhenii-ldv self-assigned this Oct 21, 2021
@VasylLazebnyk VasylLazebnyk removed this from the Connectors Dec 10 2021 milestone Dec 7, 2021
@sherifnada sherifnada linked a pull request May 5, 2022 that will close this issue
@sherifnada sherifnada self-assigned this May 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants