whisper.transcribe() Fails on Multi-GPU Systems: Incorrect Tensor Transfer to CUDA Device

[jaimebs2]

I’ve encountered a bug in the dtw_cuda() function located in timing.py. The issue arises when the cost tensor is sent to .cuda() without considering the device on which the input tensor x resides.

In the function dtw_cuda(), the cost tensor is being sent to .cuda() directly, which defaults to "cuda:0". This causes an issue if the tensor x is on a different device, such as "cuda:1", resulting in a device mismatch error. The cost tensor should be sent to the device where x is located (i.e., x.device) instead of assuming it is on "cuda:0".

If x is on "cuda:1" it will raise the following error:

ValueError: Pointer argument (at 2) cannot be accessed from Triton (cpu tensor?)

Solution:
Change dtw_cuda() function.

cost = cost.to(device=x.device)