Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bigquery Row-Level Deletes + Erase After on Database Collections #5293

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Bigquery Row-Level Deletes + Erase After on Database Collections #5293

wants to merge 11 commits into from
+440 −17

Commits on Sep 10, 2024

  1. POC - executing a delete request on a row in a postgres db.

    @pattisdr
    pattisdr committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    65dfd23 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. Merge main

    @pattisdr
    pattisdr committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d8d1cb1 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Merge branch 'main' into PROD-2744-row-deletes

    @pattisdr
    pattisdr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    4aeefe5 View commit details
    Browse the repository at this point in the history

  2. Generate a delete statement for bigquery -

    @pattisdr
    pattisdr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e07c481 View commit details
    Browse the repository at this point in the history

  3. Revert sql-connector code for now to focus on bigquery-

    @pattisdr
    pattisdr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e358379 View commit details
    Browse the repository at this point in the history

  4. Remove masking strategy override on postgres dataset

    @pattisdr
    pattisdr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e7f22d7 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Add end-to-end testing of row-level deletes as a masking strategy for… 

    … bigquery

- Seed existing bigquery table with employee record
- Updated employee dataset reference direction
- Confirm employee record exists before running DSR and confirm it's been removed afterwards
    @pattisdr
    pattisdr committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    9fa25f2 View commit details
    Browse the repository at this point in the history

  2. Add query config tests for bigquery around the update or delete state… 

    …ment built.
    @pattisdr
    pattisdr committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    0f60402 View commit details
    Browse the repository at this point in the history

  3. Improve docstrings

    @pattisdr
    pattisdr committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    4973996 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2024

  1. Extend "erase_after" concept so it can be used by databases not just … 

    …saas connectors.
    @pattisdr
    pattisdr committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    c67acc4 View commit details
    Browse the repository at this point in the history

  2. Mark tests using bigquery

    @pattisdr
    pattisdr committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    2bc6a90 View commit details
    Browse the repository at this point in the history