Object metadata should be optional in resumable uploads #332 #699
Conversation
examples/cpp/list_bucket.cpp
Outdated
| @@ -0,0 +1,15 @@ | |||
| #include "google/cloud/storage/client.h" | |||
There was a problem hiding this comment.
Can you add this to CI?
For that we need:
- A script in
./ci - A docker image where that script can run (probably something with cmake preinstalled?)
- Then add it to the matrix here:
fake-gcs-server/.github/workflows/main.yml
Lines 120 to 139 in cf3fcb0
There was a problem hiding this comment.
Didn't realize that the examples are run by the CI, but that's awesome reworked the example so it is now a test.
c5f55f7
Unfortunately I didn't find any docker image that has the google-cloud-cpp already built (that could change as the fixes are now included the latest v1.37.0 release), so I had to build it from source that makes the cpp test a bit longer than the other languages.
There was a problem hiding this comment.
Oh 3 minutes isn't too bad given that CI runs things in parallel and running the tests on Windows takes 6 minutes anyway. We could definitely optimize it later though (would probably require maintaining a custom image, or leverage some caching mechanism in GitHub Actions).
| @@ -327,10 +327,7 @@ func (s *Server) multipartUpload(bucketName string, r *http.Request) jsonRespons | |||
| func (s *Server) resumableUpload(bucketName string, r *http.Request) jsonResponse { | |||
| predefinedACL := r.URL.Query().Get("predefinedAcl") | |||
| contentEncoding := r.URL.Query().Get("contentEncoding") | |||
| metadata, err := loadMetadata(r.Body) | |||
| if err != nil { | |||
There was a problem hiding this comment.
Can you add a test that demonstrates this behavior?
Also is it possible to retain this check if we narrow it to ignore specific kinds of errors, e.g., missing metadata?
There was a problem hiding this comment.
Sure thing. I checked the problem only occurs when there is no body in the initial resumable upload request, so I've changed that we only load the metadata if there is a body, and we still throw if the load fails.
I also refactored the TestServerUndocumentedResumableUploadAPI so the test is performed with and without the body of the initial request.
8fe5502
added cpp examples
Co-authored-by: fsouza <[email protected]>
resumableUpload tests without initial body
We are working on google-cloud integration for our project Apache Nifi Minifi CPP and fake-gcs-server seems to be an awesome way to create integration tests.
However I encountered some issues, which made impossible to integrate fake-gcs-server with a C++ app
I already fixed the issues from google-cloud-cpp's side
googleapis/google-cloud-cpp#8446
googleapis/google-cloud-cpp#8477
The last issue that blocks using fake-gcs-server with a cpp application seems to be #332
As described in the issue fake-gcs-server is stricter than google cloud storage when it comes to resumable upload requests.
With this PR the object metadata is optional when creating a resumable upload request.
I also added cpp examples so it can be verified that the resumable uploads are working now with fake-gcs-server.