Skip to content

Commit

Permalink
S3ng include md5 checksum on put (#4100)
Browse files Browse the repository at this point in the history
* Do not swallow errors from PutObject

We wrongly passed err to errors.Wrapf instead of err1, leading to a
nil error to be returned.

* Als log upload errors in the non-async uploads case

* Add changelog

* include md5 checksum on put operatorions

---------

Co-authored-by: André Duffeck <[email protected]>
  • Loading branch information
wkloucek and aduffeck authored Aug 8, 2023
1 parent 5856b2e commit a944f8f
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 2 deletions.
5 changes: 5 additions & 0 deletions changelog/unreleased/fix-logging-upload-errors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Bugfix: Fix logging upload errors

We fixed a problem where problems with uploading blobs to the blobstore weren't logged.

https://github.com/cs3org/reva/pull/4099
7 changes: 7 additions & 0 deletions changelog/unreleased/fix-s3-put-md5.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Bugfix: S3ng include md5 checksum on put

We've fixed the S3 put operation of the S3ng storage to include a md5 checksum.

This md5 checksum is needed when a bucket has a retention period configured (see https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html).

https://github.com/cs3org/reva/pull/4100
4 changes: 2 additions & 2 deletions pkg/storage/fs/s3ng/blobstore/blobstore.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ func (bs *Blobstore) Upload(node *node.Node, source string) error {
}
defer reader.Close()

_, err1 := bs.client.PutObject(context.Background(), bs.bucket, bs.path(node), reader, node.Blobsize, minio.PutObjectOptions{ContentType: "application/octet-stream"})
_, err = bs.client.PutObject(context.Background(), bs.bucket, bs.path(node), reader, node.Blobsize, minio.PutObjectOptions{ContentType: "application/octet-stream", SendContentMd5: true})

if err1 != nil {
if err != nil {
return errors.Wrapf(err, "could not store object '%s' into bucket '%s'", bs.path(node), bs.bucket)
}
return nil
Expand Down
1 change: 1 addition & 0 deletions pkg/storage/utils/decomposedfs/upload/upload.go
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,7 @@ func (upload *Upload) FinishUpload(_ context.Context) error {
err = upload.Finalize()
Cleanup(upload, err != nil, false)
if err != nil {
log.Error().Err(err).Msg("failed to upload")
return err
}
}
Expand Down

0 comments on commit a944f8f

Please sign in to comment.