Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Destination snowflake: mostly done implementations for sqlgenerator+destinationhandler #28677

Merged
merged 108 commits into from
Aug 7, 2023
Merged

Destination snowflake: mostly done implementations for sqlgenerator+destinationhandler #28677

merged 108 commits into from
Aug 7, 2023

Commits on Jul 24, 2023

  1. csv sheet generator supports 1s1t

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    b7d9844 View commit details
    Browse the repository at this point in the history

  2. create+insert raw tables 1s1t

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f18f06f View commit details
    Browse the repository at this point in the history

  3. add skeletons

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    06c764c View commit details
    Browse the repository at this point in the history

  4. start writing tests

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    8ae93f4 View commit details
    Browse the repository at this point in the history

  5. progress in creating raw tables

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    0d9c8e0 View commit details
    Browse the repository at this point in the history

  6. fix tests

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    1c2d7e7 View commit details
    Browse the repository at this point in the history

  7. add s3 test; better csv generation

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f7e0f26 View commit details
    Browse the repository at this point in the history

  8. handle case-sensitive column names

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    512f9aa View commit details
    Browse the repository at this point in the history

  9. also add gcs test

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    b1dce5b View commit details
    Browse the repository at this point in the history

  10. hook T+D into the destination

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    ebaf1de View commit details
    Browse the repository at this point in the history

  11. fix redshift; simplify

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    5a5a61e View commit details
    Browse the repository at this point in the history

  12. Delete unused files?

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    d72cb67 View commit details
    Browse the repository at this point in the history

  13. disable test; enable cleanup

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    5308d59 View commit details
    Browse the repository at this point in the history

  14. initialize config singleton in tests

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    0a74e24 View commit details
    Browse the repository at this point in the history

  15. logistics

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    46af829 View commit details
    Browse the repository at this point in the history

  16. header

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    7afaa99 View commit details
    Browse the repository at this point in the history

  17. simplify

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    3a6834b View commit details
    Browse the repository at this point in the history

  18. fix unit tests

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    47afc4f View commit details
    Browse the repository at this point in the history

  19. correctly disable tests

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    db7382d View commit details
    Browse the repository at this point in the history

  20. use default null for loaded_at

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    6bd17cb View commit details
    Browse the repository at this point in the history

  21. fix test

    @edgao
    edgao committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    1da26d2 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. autoformat

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    f58abda View commit details
    Browse the repository at this point in the history

  2. cython >.>

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    fb1409e View commit details
    Browse the repository at this point in the history

  3. more singleton init

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    bd57003 View commit details
    Browse the repository at this point in the history

  4. literally how?

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    8c585c5 View commit details
    Browse the repository at this point in the history

  5. basic destinationhandler impl

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    7188340 View commit details
    Browse the repository at this point in the history

  6. use raw string for type >.>

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    ba7e62b View commit details
    Browse the repository at this point in the history

  7. add toDialectType

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    bce00b9 View commit details
    Browse the repository at this point in the history

  8. basic createTable impl

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    ad30bae View commit details
    Browse the repository at this point in the history

  9. better sql query

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    4de24d0 View commit details
    Browse the repository at this point in the history

  10. comment

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    4cfd89d View commit details
    Browse the repository at this point in the history

  11. unused variables

    @edgao
    edgao committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    4c2a460 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. recorddiffer can be case-sensitive

    @edgao
    edgao committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    968ccef View commit details
    Browse the repository at this point in the history

  2. Merge branch 'edgao/1s1t/snowflake_skeleton' into edgao/1s1t/snowflak… 

    …e_impl
    @edgao
    edgao committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    5967428 View commit details
    Browse the repository at this point in the history

  3. misc fixes

    @edgao
    edgao committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    c64aa3e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into edgao/1s1t/snowflake_skeleton

    @edgao
    edgao authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    b17b30b View commit details
    Browse the repository at this point in the history

  5. Merge branch 'edgao/1s1t/snowflake_skeleton' into edgao/1s1t/snowflak… 

    …e_impl
    @edgao
    edgao authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    1e08377 View commit details
    Browse the repository at this point in the history

  6. add expected_records

    @edgao
    edgao committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    18a651d View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. move constants to base-java

    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    ff3ce7e View commit details
    Browse the repository at this point in the history

  2. use ternary

    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    04d3080 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into edgao/1s1t/snowflake_skeleton

    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    d757fdb View commit details
    Browse the repository at this point in the history

  4. Merge branch 'edgao/1s1t/snowflake_skeleton' into edgao/1s1t/snowflak… 

    …e_impl
    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    309d473 View commit details
    Browse the repository at this point in the history

  5. fix tests

    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    5e2afdb View commit details
    Browse the repository at this point in the history

  6. resolve todo

    @edgao
    edgao committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    0ad3de9 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'master' into edgao/1s1t/snowflake_impl

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    6a85ceb View commit details
    Browse the repository at this point in the history

  2. T+D can trigger on first commit

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    68ae5f0 View commit details
    Browse the repository at this point in the history

  3. fix test teardown

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    bd8e0a2 View commit details
    Browse the repository at this point in the history

  4. implement softReset

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    2747c7e View commit details
    Browse the repository at this point in the history

  5. implement overwriteFinalTable

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    9dc16d3 View commit details
    Browse the repository at this point in the history

  6. better type stuff; check table schema

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    18a9994 View commit details
    Browse the repository at this point in the history

  7. fix

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    e61fad0 View commit details
    Browse the repository at this point in the history

  8. derp

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    1143ecf View commit details
    Browse the repository at this point in the history

  9. implement updateTable?

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    9e1ed56 View commit details
    Browse the repository at this point in the history

  10. derp

    @edgao
    edgao committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    57947d0 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. random wip stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5a5c8a0 View commit details
    Browse the repository at this point in the history

  2. fix insertRaw

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    d710402 View commit details
    Browse the repository at this point in the history

  3. theoretically implement stuff?

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    fa16292 View commit details
    Browse the repository at this point in the history

  4. stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    fbb2950 View commit details
    Browse the repository at this point in the history

  5. put suffix at the end

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    75925d8 View commit details
    Browse the repository at this point in the history

  6. different uuids

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    0de22e8 View commit details
    Browse the repository at this point in the history

  7. fix expected records

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    3dc920d View commit details
    Browse the repository at this point in the history

  8. move tdtest resources into dat folder

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    4035dd5 View commit details
    Browse the repository at this point in the history

  9. use resource files

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    304a807 View commit details
    Browse the repository at this point in the history

  10. stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    d427a35 View commit details
    Browse the repository at this point in the history

  11. move code around

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    52a0828 View commit details
    Browse the repository at this point in the history

  12. more stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    f6c2546 View commit details
    Browse the repository at this point in the history

  13. rename final table

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    8173901 View commit details
    Browse the repository at this point in the history

  14. stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    b4110b0 View commit details
    Browse the repository at this point in the history

  15. cdc immediate deletion

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    e9de901 View commit details
    Browse the repository at this point in the history

  16. cdcComplexUpdate

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    8120393 View commit details
    Browse the repository at this point in the history

  17. cleanup

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    2def75f View commit details
    Browse the repository at this point in the history

  18. botched rebase

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    ff26cb2 View commit details
    Browse the repository at this point in the history

  19. more tests

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    2c6e46d View commit details
    Browse the repository at this point in the history

  20. move back to old file

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    50b1709 View commit details
    Browse the repository at this point in the history

  21. Automated Commit - Format and Process Resources Changes

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    0b10da9 View commit details
    Browse the repository at this point in the history

  22. add comments

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5227fa2 View commit details
    Browse the repository at this point in the history

  23. Automated Commit - Format and Process Resources Changes

    @edgao @octavia-squidington-iii
    edgao authored and octavia-squidington-iii committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    583a341 View commit details
    Browse the repository at this point in the history

  24. Merge branch 'master' into edgao/1s1t/snowflake_impl

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    db4ef7b View commit details
    Browse the repository at this point in the history

  25. Merge branch 'edgao/1s1t/sqlgeneratorintegrationtest' into edgao/1s1t… 

    …/snowflake_impl
    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    2f84de4 View commit details
    Browse the repository at this point in the history

  26. fix merge

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    59c1ce0 View commit details
    Browse the repository at this point in the history

  27. move expected_records into dat folder

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    9785913 View commit details
    Browse the repository at this point in the history

  28. wip implement sqlgenerator test

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    824cdc5 View commit details
    Browse the repository at this point in the history

  29. basic implementation

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    3c4c40b View commit details
    Browse the repository at this point in the history

  30. tons of fixes, still tons more to go

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    23ced7a View commit details
    Browse the repository at this point in the history

  31. more stuff

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    a8ea566 View commit details
    Browse the repository at this point in the history

  32. fix more things

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5dcf87b View commit details
    Browse the repository at this point in the history

  33. Merge branch 'master' into edgao/1s1t/snowflake_impl

    @edgao
    edgao committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    58f3718 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. hacky convert temporal types to varchar

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    ae31c80 View commit details
    Browse the repository at this point in the history

  2. test data fix

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    a816bff View commit details
    Browse the repository at this point in the history

  3. fix variant parsing

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    119f162 View commit details
    Browse the repository at this point in the history

  4. fix number

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    00f554b View commit details
    Browse the repository at this point in the history

  5. fix time parsing; fix test data

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    3495b14 View commit details
    Browse the repository at this point in the history

  6. typo

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    281a1dd View commit details
    Browse the repository at this point in the history

  7. fix input data

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    798fd91 View commit details
    Browse the repository at this point in the history

  8. progress

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    2aa14e0 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'master' into edgao/1s1t/snowflake_impl

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    7526008 View commit details
    Browse the repository at this point in the history

  10. switch back to float

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    b1470ae View commit details
    Browse the repository at this point in the history

  11. add more test files

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    846399e View commit details
    Browse the repository at this point in the history

  12. swap int -> number

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    3e2e85f View commit details
    Browse the repository at this point in the history

  13. fix PK null check

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    752d5fd View commit details
    Browse the repository at this point in the history

  14. fix overwriteTable

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    dc4f192 View commit details
    Browse the repository at this point in the history

  15. better test

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    84ca2ed View commit details
    Browse the repository at this point in the history

  16. Automated Commit - Format and Process Resources Changes

    @edgao @octavia-squidington-iii
    edgao authored and octavia-squidington-iii committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    3c687eb View commit details
    Browse the repository at this point in the history

  17. type aliases, one more test

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8df71a9 View commit details
    Browse the repository at this point in the history

  18. also verify numeric precision/scale

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    e7bbca2 View commit details
    Browse the repository at this point in the history

  19. Merge branch 'master' into edgao/1s1t/snowflake_impl

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    5ca505e View commit details
    Browse the repository at this point in the history

  20. Merge branch 'edgao/1s1t/snowflake_impl' of github.com:airbytehq/airb… 

    …yte into edgao/1s1t/snowflake_impl
    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    18d895e View commit details
    Browse the repository at this point in the history

  21. logistics

    @edgao
    edgao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    95cb310 View commit details
    Browse the repository at this point in the history