Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow gcloud storage upload for small files? #1078

Closed
srlowe opened this issue Jan 20, 2016 · 15 comments
Closed

Slow gcloud storage upload for small files? #1078

srlowe opened this issue Jan 20, 2016 · 15 comments
Assignees
Labels
api: storage Issues related to the Cloud Storage API.

Comments

@srlowe
Copy link

srlowe commented Jan 20, 2016

I'm seeing upload times of over 6000ms for small 10kB files. Please could someone confirm if this is expected for Google cloud uploads, or is it more likely there is some client problem happening?

I've tried disabling resumable and validation, and this improves things a bit, but not substantially. Our upload code looks like this:

    var options = {
        resumable : false,
        validation: false
    };

    var remoteWriteStream = bucket.file(fileName).createWriteStream(options);
    remoteWriteStream.on('finish', function(){ callback(); });
    remoteWriteStream.on('error',  function(err){ callback(err); });
    sourceStream.pipe(remoteWriteStream);
@stephenplusplus
Copy link
Contributor

Can you run another upload immediately after the first one completes and measure the time difference?

@stephenplusplus stephenplusplus added the api: storage Issues related to the Cloud Storage API. label Jan 20, 2016
@srlowe srlowe changed the title Extremely slow gcloud storage upload for small files? Slow gcloud storage upload for small files? Jan 20, 2016
@srlowe
Copy link
Author

srlowe commented Jan 20, 2016

Sure. It seems subsequent calls are faster. Here's the same upload repeated 5x in a row (~10kB with resume and validation off):

4935ms, 3336ms, 2336ms, 2551ms, 2695ms.

Still seems on the slow side to me - what do you think?

Maybe my expectations are wrong though. I suppose there are overheads like auth etc, which disadvantage small files.

@jgeewax
Copy link
Contributor

jgeewax commented Jan 20, 2016

This definitely seems wrong to me. Uploading 10k shouldn't be taking 1000
ms. That said, GCS is optimized for throughput (aka, large file overall
upload time) rather than latency (small file overall upload time), so it's
possible (just shitty...)

@srlowe
Copy link
Author

srlowe commented Jan 20, 2016

@jgeewax - Right, that was my feeling too.

Does anyone know if there's some kind of debug mode, which might give some visibility into what's happening, time-wise?

@stephenplusplus
Copy link
Contributor

With my own tests, I complete uploads consistently under 1s. It went up to around 3 if I uploaded to a bucket overseas. Any chance location could be playing a role?

@jgeewax
Copy link
Contributor

jgeewax commented Jan 20, 2016 via email

@srlowe
Copy link
Author

srlowe commented Jan 20, 2016

Thanks, I tried both US and EU storage locations, but times seemed similar (I'm in EU).

@jgeewax
Copy link
Contributor

jgeewax commented Jan 20, 2016 via email

@stephenplusplus
Copy link
Contributor

If you have gsutil CLI tool installed, you can see what times you get there, just to see if there are any huge differences:

time gsutil cp 10kfile.txt gs://your-bucket

@stephenplusplus
Copy link
Contributor

Here's the test I've used: https://gist.github.com/stephenplusplus/15d0d7e96a40887c74aa

@srlowe
Copy link
Author

srlowe commented Jan 20, 2016

@jgeewax , @stephenplusplus - Thanks, I'll test traceroute, gsutil and that test script and report back.

Worth noting that I'm doing this from an external machine, not google-compute, and am specifying credentials like so (which may be a factor):

var storage = gcloud.storage({
    projectId   : xxx,
    credentials : {
          client_email : yyy,
          private_key  : zzz
});

@stephenplusplus
Copy link
Contributor

The credentials will cause a delay, but will only affect the first request. Have you had a chance to run any more tests?

@mziccard any chance you can run the sample here and report your findings? https://gist.github.com/stephenplusplus/15d0d7e96a40887c74aa

@stephenplusplus
Copy link
Contributor

@bgSosh I'm going to close this out as this is likely a geographic issue, beyond what this library can help with. If you have a chance to run some more tests or find anything we might not have thought of that would help, please re-open.

@Eve-rine
Copy link

Eve-rine commented Nov 9, 2020

Hello guys my uploads are taking too long for small files of 500kbs upto 98 seconds how do I sort this issue please

@stephenplusplus
Copy link
Contributor

@Evrine19 please open a new issue and follow the issue template. That will help us get up to speed and see if we can reproduce the issue you're having. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API.
Projects
None yet
Development

No branches or pull requests

4 participants