Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake v2 bug fixes and refactoring Convert action #3670

Merged
merged 30 commits into from
Oct 3, 2024

Commits on Aug 8, 2024

  1. - First pass at getting all partitions. Still needs a few parameters …

    …setup so the code is currently broken. WIP.
    TobinWritesCode committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    4fe3265 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. Update script.csx

    Convert all partitions, not just the first one.
    TobinWritesCode committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    fd85b8f View commit details
    Browse the repository at this point in the history
  2. Update script.csx

    Fix placeholders of querystring params that need gotten/set for fetching subsequent partitions.
    TobinWritesCode committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    0387027 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. Update script.csx

    Fix syntax errors to ensure that isn't why connector upload is failing.
    TobinWritesCode committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    7db5142 View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Update script.csx

    Fix more compilation errors.
    TobinWritesCode committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    e1ca0a8 View commit details
    Browse the repository at this point in the history
  2. Update script.csx

    Wanted a record of the fact that these changes are still returning subsequent partitions in array format, despite literally every response being converted.
    
    Is there maybe some sort of caching of the connector behavior, because it seems like I can't ever get the behavior of my data flow to change at all...
    TobinWritesCode committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    4d4febb View commit details
    Browse the repository at this point in the history
  3. Update script.csx

    This version of the connector is the most complete example that can successfully be uploaded as a custom connector.
    
    Yet I still can't get the behavior to change no matter what code changes I make.
    TobinWritesCode committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    3eaefe8 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. Issue #5 - Null detection and Type Conversion Error (#1)

    * Update script.csx
    
    - Fix null detection.
    
    * Fix issue with null handling in Snowflake connector
    
    ---------
    
    Co-authored-by: jbrinkman <[email protected]>
    TobinWritesCode and jbrinkman authored Aug 22, 2024
    Configuration menu
    Copy the full SHA
    aab7e25 View commit details
    Browse the repository at this point in the history
  2. Update apiDefinition.swagger.json

    - This version of the swaggerjson SHOULD be working, but we are seeing the DataSchema object being flattened out once uploaded to customer connector.
    TobinWritesCode committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    f4f5107 View commit details
    Browse the repository at this point in the history
  3. array data for DataSchema

    Got the swagger right (was really the code was had checked in before with just a little cleanup.
    
    The custom connector is now failing due to internal server error so we need to find a way to use the test page in powerapps online, despite the fact that it doesn't really handle array data very well. Possibly specifying the raw body data might be a workaround.
    TobinWritesCode committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    ed22c98 View commit details
    Browse the repository at this point in the history
  4. Update script.csx

    - Last few tweaks to get the customer connector to return subsequent partitions in pre-converted format.
    TobinWritesCode committed Aug 22, 2024
    Configuration menu
    Copy the full SHA
    c738294 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Configuration menu
    Copy the full SHA
    38377bc View commit details
    Browse the repository at this point in the history
  2. cleanup endpoints

    - extra body element is required. Caused a whole mess of issues.
    - change DataSchema to required and deprecate or delete unused endpoints as needed.
    - Remove fetchAllPages feature and separate into its own branch.
    TobinWritesCode committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    74c7ad6 View commit details
    Browse the repository at this point in the history
  3. Code cleanup.

    - Make log messages more accurate.
    - Remove last remnant of fetchAllPartitions.
    TobinWritesCode committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7081327 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #2 from Bit-Quill/jbrinkman_VersionInfo

    Add version information into readme documentation
    jbrinkman authored Aug 26, 2024
    Configuration menu
    Copy the full SHA
    9e6b2a9 View commit details
    Browse the repository at this point in the history
  5. code cleanup

    TobinWritesCode committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    6cd349f View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    df01092 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    1601c27 View commit details
    Browse the repository at this point in the history
  8. Merge pull request #3 from Bit-Quill/tchee_Partitions

    SPC-12/SPC-32 - All partitions should be returned in json class notation.
    jbrinkman authored Aug 26, 2024
    Configuration menu
    Copy the full SHA
    31fde79 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. intermediate check in

    - The code is acting absolutely insane and returning GetResults method as just a single property "Data" formatted as array. Since this is the 0 partition it should include metadata.
    - The interface is also not showing the partition parameter for the execsql method, so something is borked.
    TobinWritesCode committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    5840936 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. GetResults partition zero fixed

    - This was a very subtle issue related to the fact that when you call GetResults operation for partition zero you have no request body, so it cannot be parsed as json.
    - Change the response of execstmt async to match the schema of the sync version b/c the powerapps ui does not seem to be able to deal with the fact that async/sync have different response formats respectively.
    TobinWritesCode committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    78402e2 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Merge pull request #4 from Bit-Quill/tche_AsyncTimeout

    Fix Async always erroring out
    jbrinkman authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    8eeef21 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. SPC-36: Handle unexpected async responses better (#5)

    * Update script.csx
    
    - Fix async detection based on response code instead of request params, b/c apparently snowflake API can decide to return an async response if a synchronous response takes too long to return.
    
    * Fix typo in script.csx
    
    "BeginFetch" misspelled
    
    ---------
    
    Co-authored-by: Joseph Brinkman <[email protected]>
    TobinWritesCode and jbrinkman authored Sep 10, 2024
    Configuration menu
    Copy the full SHA
    3f2502e View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. SPC-39: MULTI_STATEMENT_COUNT parameter was being ignored (#6)

    * Update apiDefinition.swagger.json
    
    - change parameter name case to match snowflake docs exactly
    
    * add StatementHandles
    
    Map new response property for multi-statement handling.
    
    * Apply mappings to GetResults
    
    Same statementHandles mapping that was previously added to ExecSql was applied to GetResults to support Async
    
    * Remove async fixes
    
    These changes are already in dev branch, it was just a temporary change for debugging.
    TobinWritesCode authored Sep 11, 2024
    Configuration menu
    Copy the full SHA
    7a33e8f View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. GetResults schema (#7)

    Innacurate schema was causing compilation issues in the power apps. Better to leave it as a untyped object since the schema is dynamic.
    TobinWritesCode authored Sep 17, 2024
    Configuration menu
    Copy the full SHA
    45c6720 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. - Parse Object/Array types (#8)

    They were represented as a string before.
    TobinWritesCode authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    0146734 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. Document limitations per my experience. (#9)

    * Document limitations per my experience.
    
    * - Tweak readme
    
    * Update language limitations in the Readme documentation.
    
    ---------
    
    Co-authored-by: jbrinkman <[email protected]>
    TobinWritesCode and jbrinkman authored Sep 24, 2024
    Configuration menu
    Copy the full SHA
    2856664 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. openapi spec validation errors (#10)

    I was able to type the untyped objects, but a lot of those openapi spec validation errors are inherent to the fact that the snowflake api routes are technically all partial matches for eachother since the exec stmt path is "/"
    TobinWritesCode authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    db3b2db View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. Updated version history

    jbrinkman committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    1d170f4 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. Fix typo

    jbrinkman committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    75e550f View commit details
    Browse the repository at this point in the history