Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build Encryption logic for files being generated in destinations #10009

Closed
subodh1810 opened this issue Feb 2, 2022 · 2 comments
Closed

Build Encryption logic for files being generated in destinations #10009

subodh1810 opened this issue Feb 2, 2022 · 2 comments
Assignees
@edgao
Labels
type/enhancement New feature or request
Milestone
ConnCore Feb 23, ...

Comments

@subodh1810
Copy link
Contributor

subodh1810 commented Feb 2, 2022 •
edited by sherifnada
Loading

Part of airbytehq/airbyte-internal-issues#310
Tech spec : https://docs.google.com/document/d/18_Ezbl7Z1aSpHBdFFUy6BHMlDfYTktI4Yp8h2DbX4RE/edit#heading=h.fa72b39y2m99
As part of this issue we should introduce logic to encrypt files that are being generated in destinations using external loading method. We should make sure we use ephemeral keys.

As part of this ticket we need to also decide how encryption interacts with the option in the Redshift destination which allows the user to keep the staging data.
@subodh1810 subodh1810 added the type/enhancement New feature or request label Feb 2, 2022
@subodh1810 subodh1810 added this to the ConnCore Feb 9 milestone Feb 2, 2022
@subodh1810 subodh1810 changed the title Implementation of common S3 support for destinations Build Encryption logic for files being generated in destinations Feb 16, 2022
@subodh1810 subodh1810 self-assigned this Feb 16, 2022
@ChristopheDuong ChristopheDuong mentioned this issue Mar 11, 2022
Merged
@subodh1810
Copy link
Contributor Author

subodh1810 commented Mar 23, 2022 •
edited
Loading

This should not be done for BigQuery, and for now only be done for Redshift destination

@sherifnada sherifnada assigned edgao and unassigned tuliren Mar 30, 2022
@edgao
Copy link
Contributor

edgao commented May 4, 2022

Done in #12452, also added to destination-snowflake

@edgao edgao closed this as completed May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@edgao edgao

Labels
type/enhancement New feature or request
Projects
None yet
Milestone
ConnCore Feb 23, 2022
Development

No branches or pull requests
5 participants
@bleonard @tuliren @edgao @sherifnada @subodh1810