Skip multipart if body size is less than chunk.

[brettcave]

Fixes fog/fog#3899

@geemus did not write a test for this, spent a bit of time dabbling with shindo(nt) and have a real (non mock) test but couldn't get AWS auth params passed to run the test. Have tested manually with 0b, 5242879b and 10m files and all upload correctly with multipart being set on file size >= multipart_chunk.

[brettcave]

@geemus thinking about also adding in a minimum chunk size check in this PR too. According to http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html the minimum chunk size is 5242880. This means that if I set the chunk size to 5242879 and upload a 5242880 file, the upload will fail with something similar to:

Excon::Error::BadRequest: Expected(200) <=> Actual(400 Bad Request)
excon.error.response
  :body          => "<Error><Code>EntityTooSmall</Code><Message>Your proposed upload is smaller than the minimum allowed size</Message><ProposedSize>5242879</ProposedSize><MinSizeAllowed>5242880</MinSizeAllowed><PartNumber>1</PartNumber>....

Something like this would fail fast on the client before attempting the upload (instead of uploading 5242879 bytes first and then failing), but not sure if service-side checks should fall in the scope of the library. Your thoughts?

--- a/lib/fog/aws/models/storage/file.rb
+++ b/lib/fog/aws/models/storage/file.rb
@@ -205,6 +205,8 @@ module Fog
           options['x-amz-storage-class'] = storage_class if storage_class
           options.merge!(encryption_headers)

+          raise "multipart_chunk_size too small, minimum 5242880b" if multipart_chunk_size && multipart_chunk_size<5242880
+
           if multipart_chunk_size && Fog::Storage.get_body_size(body) >= multipart_chunk_size && body.respond_to?(:read)

[geemus]

Ah, didn't know about that minimum. Yeah, maybe we should just do if body < 5242879 so that it would "just work" instead of erroring. Maybe fail fast would be better though, as you suggest. What do you think?

[brettcave]

It might be better to validate the parameter on the setter, so have overridden it in the same way that acl does (for consistency). Calling .multipart_chunk_size=1 will raise an exception and not set as MP upload, but the .save will still work (less likely to break any implementations with chunks too small).

[brettcave]

http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html

Upload objects in a single operation—With a single PUT operation you can upload objects up to 5 GB in size.

Also automatically set multipart for file sizes > 5gb.

[geemus]

Definitely sounds like a nicer/clearer approach. Thanks!