Snowflake v2 bug fixes and refactoring Convert action #3670
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…setup so the code is currently broken. WIP.
Convert all partitions, not just the first one.
Fix placeholders of querystring params that need gotten/set for fetching subsequent partitions.
Fix syntax errors to ensure that isn't why connector upload is failing.
Fix more compilation errors.
Wanted a record of the fact that these changes are still returning subsequent partitions in array format, despite literally every response being converted. Is there maybe some sort of caching of the connector behavior, because it seems like I can't ever get the behavior of my data flow to change at all...
This version of the connector is the most complete example that can successfully be uploaded as a custom connector. Yet I still can't get the behavior to change no matter what code changes I make.
* Update script.csx - Fix null detection. * Fix issue with null handling in Snowflake connector --------- Co-authored-by: jbrinkman <[email protected]>
- This version of the swaggerjson SHOULD be working, but we are seeing the DataSchema object being flattened out once uploaded to customer connector.
Got the swagger right (was really the code was had checked in before with just a little cleanup. The custom connector is now failing due to internal server error so we need to find a way to use the test page in powerapps online, despite the fact that it doesn't really handle array data very well. Possibly specifying the raw body data might be a workaround.
- Last few tweaks to get the customer connector to return subsequent partitions in pre-converted format.
- extra body element is required. Caused a whole mess of issues. - change DataSchema to required and deprecate or delete unused endpoints as needed. - Remove fetchAllPages feature and separate into its own branch.
- Make log messages more accurate. - Remove last remnant of fetchAllPartitions.
Add version information into readme documentation
SPC-12/SPC-32 - All partitions should be returned in json class notation.
- The code is acting absolutely insane and returning GetResults method as just a single property "Data" formatted as array. Since this is the 0 partition it should include metadata. - The interface is also not showing the partition parameter for the execsql method, so something is borked.
- This was a very subtle issue related to the fact that when you call GetResults operation for partition zero you have no request body, so it cannot be parsed as json. - Change the response of execstmt async to match the schema of the sync version b/c the powerapps ui does not seem to be able to deal with the fact that async/sync have different response formats respectively.
Fix Async always erroring out
* Update script.csx - Fix async detection based on response code instead of request params, b/c apparently snowflake API can decide to return an async response if a synchronous response takes too long to return. * Fix typo in script.csx "BeginFetch" misspelled --------- Co-authored-by: Joseph Brinkman <[email protected]>
* Update apiDefinition.swagger.json - change parameter name case to match snowflake docs exactly * add StatementHandles Map new response property for multi-statement handling. * Apply mappings to GetResults Same statementHandles mapping that was previously added to ExecSql was applied to GetResults to support Async * Remove async fixes These changes are already in dev branch, it was just a temporary change for debugging.
Innacurate schema was causing compilation issues in the power apps. Better to leave it as a untyped object since the schema is dynamic.
They were represented as a string before.
* Document limitations per my experience. * - Tweak readme * Update language limitations in the Readme documentation. --------- Co-authored-by: jbrinkman <[email protected]>
I was able to type the untyped objects, but a lot of those openapi spec validation errors are inherent to the fact that the snowflake api routes are technically all partial matches for eachother since the exec stmt path is "/"
@microsoft-github-policy-service agree [company="Snowflake"] |
@microsoft-github-policy-service agree company="Snowflake" |
vmanoharas
reviewed
Sep 30, 2024
vmanoharas
approved these changes
Oct 3, 2024
There was a problem hiding this comment.
Dear Partner,
Congratulations your PR is approved !! We are proud to announce the Brand-New Certification experience for you to certify Power Platform Copilot Connectors & Plugins. Read the blog here- Announcing Partner Center to certify and publish Power Platform Copilot Connectors and Plugins - Microsoft Power Platform Blog
As next steps:
- Learn about the new certification experience- Get your Power Platform connector and plugin certified - Overview
- Package your connector and plugin files- Prepare Power Platform connector and plugin files for certification
- Validate the package for structure before submitting the package for certification to Partner Center - Run Package Validator tool
- Initiate a certification request in Partner Center- Verified publisher certification process
- Ensure your connector & plugin files comply to the Marketplace policies here- 5000 Power Platform Connector Policies for Marketplace and also 1000 Marketplace policies(as applicable)
- Test your connector post certification to provide go-live for deployment- Testing Guidelines
- If you wish to Update your connector or plugin in future- Updating Guidelines
For any queries or concerns please get in touch with us on [email protected]
Many thanks,
Power Platform
Copilot connector & plugin certification team
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When submitting a connector, please make sure that you follow the requirements below, otherwise your PR might be rejected. We want to make you have a well-built connector, a smooth certification experience, and your users are happy :)
If this is your first time submitting to GitHub and you need some help, please sign up for this session.
apiDefinition.swagger.json, by runningpaconn validatecommand.apiProperties.jsonhas a valid brand color and doesn't use an invalid brand color,#007ee5or#ffffff. If this is an independent publisher connector, I confirm that I am not submitting a connector icon.If you are an Independent Publisher, you must also attest to the following to ensure a smooth publishing process:
Fixes #3562, #2947, #2892, #2887
Deprecate the Convert action and incorporate conversion directly into the Submit SQL Statement for Execution and Check the Status and Get Results actions.