Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source File-Secure: source-file-secure should depend on source-file python package instead of source-file docker image #26823

Closed
alafanechere opened this issue May 31, 2023 · 1 comment

Comments

@alafanechere
Copy link
Contributor

alafanechere commented May 31, 2023

source-file-secure is a variant of source-file that enforces SSL.

source-file-secure currently overrides source-file by installing source-file via the use of the FROM airbyte/source-file:0.3.9 instruction in its Dockerfile.

It leads to the following clumsy imports to make sure the python execution can work inside a docker container or inside a classic python virtual env:

try:
    import source_file.source
except ModuleNotFoundError:
    current_dir = os.path.dirname(os.path.abspath(__file__))
    parent_source_local = os.path.join(current_dir, "../../source-file")
    if os.path.isdir(parent_source_local):
        sys.path.append(parent_source_local)
    else:
        raise RuntimeError("not found parent source folder")
    import source_file.source

I believe we should rather install source-file in source-file-secure via local pip dependencies declared in setup.py like here.

It will simplify the dependency expression outside of the Docker context and allow us to build this connector with a standard dockerfile / dagger pipeline.

This will become more pressing when we'd want to get rid of Python connector's dockerfile and replace them by Dagger build pipeline with airbyte-ci.

@igrankova igrankova changed the title source-file-secure should depend on source-file python package instead of source-file docker image Source File-Secure: source-file-secure should depend on source-file python package instead of source-file docker image Jun 2, 2023
@alafanechere
Copy link
Contributor Author

Done in #27107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants