-
Notifications
You must be signed in to change notification settings - Fork 389
Very Slow #27
Comments
I've run 5 tests so far with very small files and all my times are over 20 seconds. I guess this is expected?
|
I found that if you crank the lambda function settings all the way up to 3GB (as high as it will go, as of now), the timing on the scan goes down to about 13s. Still not great, but improved. |
Has anyone had any luck with improving speeds? When scanning 2 files (500 bytes each), it takes 15 seconds each file @ 3008 megabytes memory, and it's because of the actual clamscan (from print to print): Any way to speed this up? Would it be more valuable to put any of this inside of a lambda layer? I am unsure what 'local' scan means in the above message, how do I enable/perform that? |
same issue. No matter how much file weigh, 5KB or 300MB. Execution time the same |
I'm scanning the 68 byte EICAR test file and it's taking around 70 seconds. Haven't cranked the memory, but it sounds like that has diminishing returns anyway. The majority of time is spent scanning, not checking / downloading definitions, or even transferring the file to /tmp/. See below
|
Same here, we see average scan time on the north of 80 seconds per file. Most of our files are images and pdfs. |
The scanning has ballooned to almost 100 seconds per file. It's not practical to use Lamda any more. We've implemented a local scan solution for our document uploads using a clamav docker image from https://github.com/mko-x/docker-clamav. |
Increasing the AWS Lambda max memory helps with run time. We set to 2048 and get 40-45 second run times. |
FWIW I went with this implementation instead: https://github.com/widdix/aws-s3-virusscan Autoscaling EC2 cluster that does the scanning. Was pretty easy to set up and is near instant. |
I think the answers in this thread are correct. Update the memory allocated for the lambda and that will speed things up. Also, if AWS re-uses a lambda it can have a faster spin-up time but that's not guaranteed. If you need something faster then lambda may not be the best option for your workflow. |
It looks to me like this PR will address this issue: |
Thanks! |
I tested this PR out, it works really well in my testing. |
The time between the file uploaded and the scan process to the end, is taking more than 10 seconds. This is normal?
There is a faster way. Local scanning takes no more than 200ms
The text was updated successfully, but these errors were encountered: