increasing the GPU usage #798
acn-reginald-casela
started this conversation in
General
Replies: 2 comments
-
|
No, you can increase num_workers with GPU utilization but it will not increase the throughput (#100 (comment)) |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@acn-reginald-casela were you able to solve this? I'm also receiving bad results on Tesla T4 2CPU 15GB each. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Is there a way to fully utilize the GPU memory that I'm using? I've deployed an app in GKE that uses a NVIDIA Tesla T4 and it's only using roughly 3-4gb out of 16gb. I'm using the large-v2 model and the usual runtime for a 10-15s clip is around 2-5s.
I was wondering if using the num_workers parameter could also help with GPU utilization and decrease the runtime further to at least 0.5-1sec.
Beta Was this translation helpful? Give feedback.
All reactions