CUDA out of memory

[AdrLfv]

I try to generate a simple audio with the web interface and I get the following error:

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB. GPU 0 has a total capacity of 5.69 GiB of which 420.69 MiB is free. Including non-PyTorch memory, this process has 4.70 GiB memory in use. Of the allocated memory 4.53 GiB is allocated by PyTorch, and 28.60 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

I use the model and pretrained model_config from https://huggingface.co/stabilityai/stable-audio-open-1.0/tree/main
I could generate audios by downgrading the sample_size and some channels size in the config but then the generated audio gets very bad.

My config

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03 Driver Version: 560.35.03 CUDA Version: 12.6 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================|
| 0 NVIDIA GeForce RTX 3060 ... Off | 00000000:01:00.0 On | N/A |
| N/A 59C P5 19W / 80W | 636MiB / 6144MiB | 37% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

[fletcherist]

set batch size to 1
how many gpu ram do u have?
try on A100 40gb it works fine
on google colab

[AdrLfv]

I have 6go VRAM as indicated in "My config" and the batch size is already on 1 by default.
I will try with google colab thx

[fletcherist]

I have 6go VRAM as indicated in "My config" and the batch size is already on 1 by default. I will try with google colab thx

yeah it's too low, it works fine on T4 with batch size 2