Struggling to compile to work with Ubuntu 20.04 and CUDA 11.4

[DaveBaker]

Please excuse my request, but I would really like some assistance to get faster-whisper working with my Jetson Orin Nano.

I have been working on a voice activated robot for some time, and recently came across faster-whisper. Unfortunately though it seems that I am restricted to using CPU only, if i use the PIP method to install faster-whisper. Due to the fact that it informs me that it is not compiled for use with CUDA (thus a default version is installed).
I have completed the installation of CTranslate2 3.24.0 according to the directions given by the installation instructions. But I still can't seem to use either faster-whisper or the compiled CTranslate2 programs.
As a last resort, I had hoped that someone might be able to guide me (as best as possible) through the process. I am sorry for the request, i am a noob with installing these libraries but have been looking endlessly across the Github site for instructions to help.
If the following information can help with the system I am using, so you can confirm I have the appropriate additions.
Platform: NVIDIA Jetson Orin Nano with aarch64, Linux, Ubuntu 20.04 focal
Python 3.8.10
CUDA: 11.4.315
cuDNN 8.6.0.166
TensorRT: 8.5.2.2

Many thanks for any assistance anyone can give me
Dave B

[trungkienbkhn]

@DaveBaker , hello. It's better to use the latest version of ctranslate2 (4x). Have you tried to upgrade CUDA from 11 to 12 ? (refer to this link) ? And could you share you code logic ? You can refer to this example when transcribing with cuda.

[DaveBaker]

Hi trungkienbkhn
Firstly may I say a heartfelt thank you for your reply.
Although it may seems a little out of place (being the faster-whisper github page). I feel the matter seems to be with the install of CTranslate2, but again I have been struggling on how to install a GPU enabled version of CTranslate2 (v3.24.0). Even with any documentation, it is not easy or intuitive for someone outside a loop to do.
As a result, I had hoped someone may have been roughly through the process and they may help me in my issue.
It looks like I will have to continue my search, but that aside thank you it was and is appreciated.
Dave B

[DaveBaker]

OK, I think i seemed to have got it to work. Using many different sources of pages and manuals by:
1: compiling CTranslate2 (checkout 3.24.0) from source essentially using the manual metohd, but changing /build to /lib directories
2: using pip install ctranslate2/python/dist/ctranslate2-3.24.0-cp38-cp38-linux_aarch64.whl - within the environment I am working in
3: pip install faster-whisper==0.10.1

It does provide faster responses and seems to be using the GPU's rather than the CPU's. But hasn't been easy to deploy.

Many thanks for taking your time to reply

Dave B