Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C4-89 Always use S3BlobStorage #138

Merged
merged 4 commits into from
Mar 24, 2020
Merged

C4-89 Always use S3BlobStorage #138

merged 4 commits into from
Mar 24, 2020

Conversation

willronchetti
Copy link
Member

  • At some point registry.settings[BLOBS] became set and we stopped using S3BlobStorage. We never use Postgres for blobs so remove code that does this from register_storage

Copy link
Collaborator

@netsettler netsettler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this matters a lot, I'm going to make my comment a blocker. I'll look at the fix in place before approving.

Comment on lines 101 to 104
try:
registry[BLOBS] = S3BlobStorage(registry.settings['blob_bucket'])
else:
except KeyError: # only use this if blob_bucket is not set
registry[BLOBS] = RDBBlobStorage(registry[DBSESSION])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worked through this by cases and believe this is almost right, but I don't like the use of try to avoid doing a simple conditional, plus it doesn't do the right thing in the case of a null string in registry.settings['blob_bucket'], which won't give a KeyError nor even an error at all, it will end up just making a S3BlobStorage(""), which I'd rather not leave to chance because it might err immediately or it might err later.

blob_bucket = registry.settings['blob_bucket']
if blob_bucket:
    registry[BLOBS] = S3BlobStorage(blob_bucket)
else:
    registry[BLOBS] = RDBBlobStorage(registry[DBSESSION])

or even the following, which could be done on one line, but isn't as easy to read. The main reason to do this version would be to emphasize the fact that the assignment is unconditional and only the value to be assigned is conditional.

blob_bucket = registry.settings['blob_bucket']
registry[BLOBS] = (S3BlobStorage(blob_bucket)
                   if blob_bucket 
                   else RDBBlobStorage(registry[DBSESSION]))

By the way, one thing I puzzled about in reviewing this code is why we previously tested the value of something we were going to unconditionally assign, so that's why unconditional assignment is on my mind. I'm glad that test went away, as it didn't seem to be helping.

There is, by the way, an odd case I noticed working through this where the previous value is a blob storage of the other kind and you just suddenly change because the blob_bucket has been set or unset. I think that case can never happen unless you make a mechanism to dynamically assign blob_bucket (which we might do), and in that case, I think it's proper to go ahead with such a change of type. I just wanted to note here that I thought about that case and am OK with the basic flow proposed here even if we later make such a change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will accept this, but shouldn't registry.settings['blob_bucket'] use .get()?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Good catch. Please consider that fix part of my proposed rewrite.

Copy link
Collaborator

@netsettler netsettler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Thanks for integrating your fix to mine. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants