Control number of threads for downloading

[AmandaRiu]

I'm writing a sync process on Android that grabs a list of items to download from one of our internal services, and then creates a download request for each file needed. Right now I have one project where I need to download 412 files (124.2 MB) total, and although I have this code working, I was wondering if there was a way to specify a Thread pool to use specifically for downloading so I can gain further control over simultaneous downloads?

Here is how I currently download a file:

BasicSessionCredentials awsCreds = new BasicSessionCredentials(
        creds.getAccessKeyId(),
        creds.getSecretAccessKey(),
        creds.getSessionToken());
AmazonS3Client client = new AmazonS3Client(awsCreds);

mAwsTransferUtility = new TransferUtility(client, context.getApplicationContext());

for (Object obj : downloadQueue) {
    mAwsTransferUtility.download(obj.getBucketName(), obj.getPath(), obj.getDestFile());
}

[fosterzhang]

Unfortunately you can't specify a thread pool. TransferUtility can do 2 downloads in parallel. Too many concurrent downloads doesn't necessarily speed things up. What's the reason you want to manage the thread pool?

[AmandaRiu]

I'm getting really bad download times. Right now I'm averaging 2 minutes and 27 seconds to download 412 files totaling 124 mb. Both our iOS and Windows developers rolled their own downloader and is averaging 30 seconds to complete the download over the same pipe with comparable devices. I'm wondering what I can do to speed this up.

[fosterzhang]

@AmandaRiu Thanks for bring this up. I see there is room for the Android SDK to improve. Stay tuned.

[fosterzhang]

The latest release v2.2.14 has addressed this issue. The number is still not configurable. However it is set to # of cores + 1 where modern Android phones may have 4-8 cores. And we remove an unnecessary network request during download. Overall, it should speed up massive smaller downloads by 50%. See http://aws.amazon.com/releasenotes/1790148718365679

[AmandaRiu]

Thanks!