Cloud storage is missing 'compose' functionality

[theacodes]

No description provided.

[daspecster]

https://cloud.google.com/storage/docs/composite-objects

[theacodes]

Thanks, @daspecster. This is important so we can write usage samples for gcloud-python that have feature parity with the google-api-python-client samples.

[tseaver]

@jonparrott Can you clarify whether the compose request is intended only to create a new object, or whether it might replace the content of an existing one? For instance, the docs show two optional query parameters, ifGenerationMatch and ifMetagenerationMatch, which don't make sense for new objects, but the description of the API says, "Concatenates a list of existing objects into a new object."

Also, the request body example has a top-level contentType key which is not documented below it, and which would appear to duplicate the same field from the destination resource.

[theacodes]

@tseaver our current sample creates a new object from two existing objects.

[tseaver]

@jonparrott Thanks for the example. I imagine it would look something like this gist, which assumes that compose is an operation on our Bucket class, taking the destination name, a list of our Blob objects as the sources, and a content-type.

is there somebody we should loop in to answer my questions about the API docs?

[theacodes]

is there somebody we should loop in to answer my questions about the API docs?

@jgeewax do we have a particular eng contact for GCS?

which assumes that compose is an operation on our Bucket class, taking the destination name, a list of our Blob objects as the sources, and a content-type.

That could work, it could also be an operation on a blob, which would make it on parity with the API (it's an operation on the object resource):

blob = bucket.blob('composed-object')
blob.compose(part_one, part_two)

It's up to you, I think either way is fine.

[tseaver]

@jonparrott I considered that option: if the API is actually supposed to allow replacing the contents of an existing Object with the composed media, that would definitely be a better spelling. If it can only be used to create a new Object, then making it an operation on the Bucket seems cleaner (I just updated my gist to show bucket.compose() returning the new instance).

[jgeewax]

@Capstan is our GCS go-to person :)

On Thu, Jul 14, 2016 at 2:23 PM, Tres Seaver [email protected]
wrote:

@jonparrott https://github.com/jonparrott I considered that option: if
the API is actually supposed to allow replacing the contents of an existing
Object with the composed media, that would definitely be a better
spelling. If it can only be used to create a new Object, then making it
an operation on the Bucket seems cleaner (I just updated my gist to show
bucket.compose() returning the new instance).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1982 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAG5IElgavVoAqXrtvEbQgVk5zaXBnsdks5qVn6VgaJpZM4JLrp8
.

[theacodes]

@tseaver makes sense. I think you can append to an existing blob, but @Capstan will have to verify.

[tseaver]

@jonparrott speaking of samples, WDYT of keeping them in this repository, so that we can use them within the Sphinx docs / docstrings, like the ones we have for pubsub and bigquery. Then we can arrange to test / fix them as they break.

[theacodes]

@tseaver it's be exhaustively discussed. The samples in python-docs-samples are samples included from documentation on cloud.google.com. This sometimes overlaps with samples here, especially for the really basic use cases, but often is more illustrative on how to accomplish a task with the service rather than just use the API.

[tseaver]

@jonparrott OK. How do you deal with drift?

[theacodes]

Tests, tests, and more tests. We also optimistically update dependencies and fix as needed.

[tseaver]

@jonparrott Do you have a webhook we should be poking from this repo to trigger those tests for the Python examples?

[theacodes]

We have nightly tests and an automated dependency checker (which right now we're behind on for reasons).

[tseaver]

@Capstan can you verify whether the compose operation always creates a new object, or might be used to append to an existing one?

[Capstan]

It logically always creates a new object. That said, creating a new object on top of an old object overwrites the old object, so you can do an extremely limited mimic of append by writing a separate object and composing the original and the separate object with a destination of the original object. This matters more if you have versioning turned on, as the old version will stick around.

[tseaver]

@Capstan thanks for the clarification.