Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake Loader: fix loading path used with temp creds auth method #1002

Closed
spenes opened this issue Jul 24, 2022 · 0 comments
Closed

Snowflake Loader: fix loading path used with temp creds auth method #1002

spenes opened this issue Jul 24, 2022 · 0 comments

Comments

@spenes
Copy link
Contributor

spenes commented Jul 24, 2022

When temporary credentials method is enabled on Snowflake Loader, temporary table is used in the interim step while loading the data to events table.

In the interim step, events are loaded to temp table initially then they are copied from temp table to events table.

In here, loader is copying all the data in the given path. However, this is not correct. Loader needs to copy only successfully transformed data and this data reside on output=good sub folder. Therefore, we should add output=good to this path.

@spenes spenes closed this as completed in 93f00da Jul 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant