RuntimeError: CUDA failed with error out of memory after fine-tuing

[kouohhashi]

Hi,

I fine-tuned large model which was originally OpenAI Whisper model.
I converted whisper model into hugging face model and faster-whisper model.

but afterwards, I start getting "CUDA failed with error out of memory" error at self.model.align.

This happens only when I set word_timestamps True.
Not all weights cause this error.
Some fine-tuned weights works fine.

Could someone help me to dig this issue deeper?
I have no idea where to start.

Thanks in advance.

[guillaumekln]

Hi,

How much memory does your GPU have?

[kouohhashi]

Thanks, my GPU memory size was 12GB. But I think I find the problem.
In a nutshell, what I did to convert whisper model into faster-whisper model was wrong.

What I dis was

1. convert openAI model into hugging face model with python convert_openai_to_hf.py.
2. copy hugging face json files because step 1 did not create all json files
3. convert hugging face mode into faster-whisper model with ct2-transformers-converter

the problem was hugging face's config.json was not equal to faster-whisper's config.json.
Or just I might have copied wrong one.

So I copied config.json from working example and it worked.

Thanks anyway,