Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alex/configurable retrier #14330

Merged
merged 126 commits into from
Jul 14, 2022
Merged

Alex/configurable retrier #14330

merged 126 commits into from
Jul 14, 2022

Commits on Jun 29, 2022

  1. Configuration menu
    Copy the full SHA
    114daea View commit details
    Browse the repository at this point in the history
  2. read_incremental works

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    6d47a6d View commit details
    Browse the repository at this point in the history
  3. reset to master

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    dbff403 View commit details
    Browse the repository at this point in the history
  4. remove dead code

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    24f54db View commit details
    Browse the repository at this point in the history
  5. comment

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    e3f8d50 View commit details
    Browse the repository at this point in the history
  6. fix

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    cbe4e57 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    ca44a97 View commit details
    Browse the repository at this point in the history
  8. Add test

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    3d3f321 View commit details
    Browse the repository at this point in the history
  9. comments

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    4ea9a2b View commit details
    Browse the repository at this point in the history
  10. utc

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    2f09e79 View commit details
    Browse the repository at this point in the history
  11. format

    girarda committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    67f174d View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2022

  1. small fix

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    096b54e View commit details
    Browse the repository at this point in the history
  2. Add test with rfc3339

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    db01257 View commit details
    Browse the repository at this point in the history
  3. remove unused param

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    f097ee2 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b40eb69 View commit details
    Browse the repository at this point in the history
  5. fix test

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    eae5bb5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e1a59aa View commit details
    Browse the repository at this point in the history
  7. update test

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    f534b20 View commit details
    Browse the repository at this point in the history
  8. start working on retrier

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    22a2296 View commit details
    Browse the repository at this point in the history
  9. retry predicate

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    5ade34d View commit details
    Browse the repository at this point in the history
  10. return response status

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    eef3a2b View commit details
    Browse the repository at this point in the history
  11. look in error message

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    16c33a5 View commit details
    Browse the repository at this point in the history
  12. cleanup test

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    355fdc4 View commit details
    Browse the repository at this point in the history
  13. constant backoff strategy

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    2b4da45 View commit details
    Browse the repository at this point in the history
  14. chain backoff strategy

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    e58b977 View commit details
    Browse the repository at this point in the history
  15. chain retrier

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    94b0ab1 View commit details
    Browse the repository at this point in the history
  16. Add to class types registry

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    d9d9004 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    6296403 View commit details
    Browse the repository at this point in the history
  18. wait until

    girarda committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    a97d730 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2022

  1. update

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    814088a View commit details
    Browse the repository at this point in the history
  2. split file

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    7972c20 View commit details
    Browse the repository at this point in the history
  3. parse_records

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    f94be2c View commit details
    Browse the repository at this point in the history
  4. merge master

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    d50f832 View commit details
    Browse the repository at this point in the history
  5. classmethod

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    892270c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    57ed90e View commit details
    Browse the repository at this point in the history
  7. delete dead code

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    fb88553 View commit details
    Browse the repository at this point in the history
  8. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    51fc8eb View commit details
    Browse the repository at this point in the history
  9. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    efc40b5 View commit details
    Browse the repository at this point in the history
  10. comments

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    e33e004 View commit details
    Browse the repository at this point in the history
  11. fix

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    fc41d54 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    e2ef317 View commit details
    Browse the repository at this point in the history
  13. fix parsing

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    b8faa8d View commit details
    Browse the repository at this point in the history
  14. cleanup

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    6f96b26 View commit details
    Browse the repository at this point in the history
  15. fix

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    d864b01 View commit details
    Browse the repository at this point in the history
  16. reset

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    d209dd6 View commit details
    Browse the repository at this point in the history
  17. never raise on http error

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    04fca0d View commit details
    Browse the repository at this point in the history
  18. remove print

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    0301e9a View commit details
    Browse the repository at this point in the history
  19. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    0367f0f View commit details
    Browse the repository at this point in the history
  20. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    994d125 View commit details
    Browse the repository at this point in the history
  21. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    f3a0af4 View commit details
    Browse the repository at this point in the history
  22. comment

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    2140c28 View commit details
    Browse the repository at this point in the history
  23. remove prints

    girarda committed Jul 1, 2022
    Configuration menu
    Copy the full SHA
    77a0325 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    ecbfcf3 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2022

  1. Configuration menu
    Copy the full SHA
    f507f91 View commit details
    Browse the repository at this point in the history
  2. Delete dead code

    girarda committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    b510b17 View commit details
    Browse the repository at this point in the history
  3. Add docstrings

    girarda committed Jul 6, 2022
    Configuration menu
    Copy the full SHA
    d03c809 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. Configuration menu
    Copy the full SHA
    6471b15 View commit details
    Browse the repository at this point in the history
  2. quick fix

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    b8b3c3a View commit details
    Browse the repository at this point in the history
  3. exponential backoff

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    71fa551 View commit details
    Browse the repository at this point in the history
  4. fix test

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    e0da35d View commit details
    Browse the repository at this point in the history
  5. fix

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    f7bc427 View commit details
    Browse the repository at this point in the history
  6. delete unused properties

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    0fca96c View commit details
    Browse the repository at this point in the history
  7. fix

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    04fc03e View commit details
    Browse the repository at this point in the history
  8. missing unit tests

    girarda committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    dda7b9c View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. merge master

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    a35c9bc View commit details
    Browse the repository at this point in the history
  2. uppercase

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    9862b4b View commit details
    Browse the repository at this point in the history
  3. docstrings

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    ad1cce6 View commit details
    Browse the repository at this point in the history
  4. rename to success

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    15c3f4e View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    eca2f45 View commit details
    Browse the repository at this point in the history
  6. renmae module

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    5a756bb View commit details
    Browse the repository at this point in the history
  7. rename test file

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    9b289cb View commit details
    Browse the repository at this point in the history
  8. rename interface

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    fff16ed View commit details
    Browse the repository at this point in the history
  9. rename default retrier

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    918e31a View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    82fb6e8 View commit details
    Browse the repository at this point in the history
  11. fix missing renames

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    98e17eb View commit details
    Browse the repository at this point in the history
  12. move action to filter

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    9ab1008 View commit details
    Browse the repository at this point in the history
  13. str -> minmaxdatetime

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    2a6c9d2 View commit details
    Browse the repository at this point in the history
  14. small fixes

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    8a72ac0 View commit details
    Browse the repository at this point in the history
  15. plural

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    73279d7 View commit details
    Browse the repository at this point in the history
  16. add example

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    181fb74 View commit details
    Browse the repository at this point in the history
  17. handle header variations

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    104ec1b View commit details
    Browse the repository at this point in the history
  18. also fix wait time from

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    cf17cae View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    08a3663 View commit details
    Browse the repository at this point in the history
  20. group()

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    764eca0 View commit details
    Browse the repository at this point in the history
  21. docstring

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    34f3a77 View commit details
    Browse the repository at this point in the history
  22. add docs

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    5cde974 View commit details
    Browse the repository at this point in the history
  23. update comment

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    573dd90 View commit details
    Browse the repository at this point in the history
  24. docstrings

    girarda committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    efdca4a View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    45df5d9 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. Configuration menu
    Copy the full SHA
    9bf3808 View commit details
    Browse the repository at this point in the history
  2. update comment

    girarda committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    db47529 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9e5fddc View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. Update airbyte-cdk/python/airbyte_cdk/sources/declarative/requesters/…

    …http_requester.py
    
    Co-authored-by: Sherif A. Nada <[email protected]>
    girarda and sherifnada authored Jul 12, 2022
    Configuration menu
    Copy the full SHA
    7aae31d View commit details
    Browse the repository at this point in the history
  2. version: Update Parquet library to latest release (#14502)

    The upstream Parquet library that is currently pinned for use in the S3 destination plugin is over a year old. The current version is generating invalid schemas for date-time with time-zone fields which appears to be addressed in the `1.12.3` release of the library in commit apache/parquet-java@c72862b
    blarghmatey authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    6656af3 View commit details
    Browse the repository at this point in the history
  3. merge

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    1a49c95 View commit details
    Browse the repository at this point in the history
  4. 🎉 Source Github: improve schema for stream pull_request_commits add…

    …ed "null" (#14613)
    
    Signed-off-by: Sergey Chvalyuk <[email protected]>
    grubberr authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    d30aa5b View commit details
    Browse the repository at this point in the history
  5. Docs: Fixed broken links (#14622)

    * fixing broken links
    
    * more broken links
    Amruta-Ranade authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    a525afe View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    64750ec View commit details
    Browse the repository at this point in the history
  7. Helm Chart: Add external temporal option (#14597)

    * conflict env configmap and chart lock
    
    * reverting lock
    
    * add eof lines and documentation on values yaml
    
    * conflict json file
    
    * rollback json
    
    * solve conflict
    
    * correct minio with new version
    
    Co-authored-by: Guy Feldman <[email protected]>
    2 people authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    f3fc604 View commit details
    Browse the repository at this point in the history
  8. 🎉 Add YAML format to source-file reader (#14588)

    * Add yaml reader
    
    * Update docs
    
    * Bumpversion of connector
    
    * bump docs
    
    * Update pyarrow dependency
    
    * Upgrade pandas dependency
    
    * auto-bump connector version
    
    Co-authored-by: Octavia Squidington III <[email protected]>
    2 people authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    2d2ef71 View commit details
    Browse the repository at this point in the history
  9. 🎉 Source Okta: add GroupMembers stream (#14380)

    * add Group_Members stream to okta source
    
    - Group_Members return a list of users, the same schema of Users stream.
    - Create a shared schema users, and both group_members and users sechema use it as a reference.
    - Add Group_Members stream to source connector
    
    * add tests and fix logs schema
    
    - fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator
    https://github.com/python-jsonschema/jsonschema/blob/ddb87afad8f5d5c40600b5ede0ab96e4d4bdf7d3/jsonschema/_validators.py#L279-L285
    - change grouop_members to use id as the cursor field since `filter` is not supported in the query string
    - fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value
    - remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api.
    
    * last polish before submit the PR
    
    - bump docker version
    - update changelog
    - add the right abnormal value for logs stream
    - correct the sample catalog
    
    * address comments::
    
    - improve comments for until parameter under the logs stream
    - add use_cache on groupMembers
    
    * add use_cache to Group_Members
    
    * change configured_catalog to test
    
    * auto-bump connector version
    
    Co-authored-by: marcosmarxm <[email protected]>
    Co-authored-by: Octavia Squidington III <[email protected]>
    3 people authored and girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    8b422c7 View commit details
    Browse the repository at this point in the history
  10. split test files

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    73b47b1 View commit details
    Browse the repository at this point in the history
  11. renames

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    f97cfee View commit details
    Browse the repository at this point in the history
  12. missing unit test

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    18e562a View commit details
    Browse the repository at this point in the history
  13. add missing unit tests

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    2d1312e View commit details
    Browse the repository at this point in the history
  14. rename

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    a0ab86b View commit details
    Browse the repository at this point in the history
  15. merge

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    d4c7db1 View commit details
    Browse the repository at this point in the history
  16. assert isinstance

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    32e34dc View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    187858f View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    bbc8cf5 View commit details
    Browse the repository at this point in the history
  19. assert we retry 429 errors

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    4647033 View commit details
    Browse the repository at this point in the history
  20. Add log

    girarda committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    a201c3f View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    198c1d2 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    0dbe4a9 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    d996baa View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. get class_name from type

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    47a56d8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    22521bf View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6b77952 View commit details
    Browse the repository at this point in the history
  4. sort

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    598c36a View commit details
    Browse the repository at this point in the history
  5. delete print statement

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    72dc105 View commit details
    Browse the repository at this point in the history
  6. comment

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    2d291b6 View commit details
    Browse the repository at this point in the history
  7. comment

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    7673b52 View commit details
    Browse the repository at this point in the history
  8. merge master

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    7bdc243 View commit details
    Browse the repository at this point in the history
  9. format

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    ee7de21 View commit details
    Browse the repository at this point in the history
  10. delete unused file

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    58d154a View commit details
    Browse the repository at this point in the history
  11. merge master

    girarda committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    a00c6a8 View commit details
    Browse the repository at this point in the history