Support GCS #3905
Merged
Support GCS #3905
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Ashwin Ramaswami <[email protected]>
This reverts commit eeaccd7.
epicfaace
commented
Dec 8, 2021
epicfaace
commented
Dec 12, 2021
|
|
||
| def upgrade(): | ||
| # ### commands auto generated by Alembic - please adjust! ### | ||
| op.alter_column('bundle_store', 'storage_type', type_=sa.String(255), existing_type=sa.Enum("disk", "azure_blob"), nullable=True) |
There was a problem hiding this comment.
This is a one-time change. Realized that it's difficult to modify enums so I changed the columns to just be Text types instead, so we don't need migrations for future expansions of supporting new storage types.
|
@percyliang FYI this PR is about the unit of work required to support a new storage type that Apache Beam supports. Relatively straightforward and minimal -- a similar PR could add support for S3. |
jzwang43
approved these changes
Dec 14, 2021
epicfaace
commented
Dec 14, 2021
| First, make sure the `CODALAB_GOOGLE_APPLICATION_CREDENTIALS` env var is set on the server that runs the REST server. It should be set to a path to a credentials JSON file. | ||
|
|
||
| ``` | ||
| export CODALAB_GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json |
There was a problem hiding this comment.
@teetone Flagging this for you -- you'll need to set up this environment variable when setting up GCP on codalab.stanford.edu.
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3904. Depends on #3828 to be merged first.