"Robotic noise" with automatic1111 and the extension "sd-webui-riffusion"

[zejacky]

Thank you for your work! And happy new year.

I have a "problem" with automatic1111 and the extension "sd-webui-riffusion". I installed it like described.
Also added the ENV Variable to boot System and User PATH. (D:\ffmpeg\bin)
The Extension appears booth in the webui Tab and under Scripts.

If I convert a picture to audio from the Tab or via script, there is no error in the CMD.
However, if I listen to the audio file, it sounds "corrupted" and like some strange noise (robotic), not like music.
(See attachment)

I think Im doing something wrong, or I don't understand how it works.
Maybe you have already an idea.
Sound Device: Realtek HD audio / Nvidia HD audio
Windows 11 x64

Thank you very much for helping me out.

Untitled.video.mp4

[xuuyz]

did you download the ckpt for riffusion

[zejacky]

Thank you very much for my missing puzzle piece :). I was downloading it now from
https://huggingface.co/riffusion/riffusion-model-v1/blob/main/riffusion-model-v1.ckpt
and it works !

[xuuyz]

np! glad i could help

[Diamondturtle99]

Same issue, but I have the model, and it says at the end of the command line output 'torch not compiled with cuda enabled'. Everything else works, and the image comes out fine, but when I use an online converter, I get these weird sounds. logs here:

Found 1 images in C:\Users\ryano\Music\stable-diffusion-webui-directml\outputs\txt2img-images\2023-08-09, pattern *.jpg, *.png
Traceback (most recent call last):
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\routes.py", line 422, in run_predict
output = await app.get_blocks().process_api(
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1323, in process_api
result = await self.call_function(
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1051, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\extensions\sd-webui-riffusion\scripts\riffusion.py", line 364, in convert_audio
output_files.append(convert_audio_image(image, image_file, image_dir, width))
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\extensions\sd-webui-riffusion\scripts\riffusion.py", line 333, in convert_audio_image
wav_bytes, duration_s = riffusion.wav_bytes_from_spectrogram_image(image_file)
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\extensions\sd-webui-riffusion\scripts\riffusion.py", line 186, in wav_bytes_from_spectrogram_image
samples = self.waveform_from_spectrogram(
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\extensions\sd-webui-riffusion\scripts\riffusion.py", line 291, in waveform_from_spectrogram
Sxx_torch = torch.from_numpy(Sxx).to(device)
File "C:\Users\ryano\Music\stable-diffusion-webui-directml\venv\lib\site-packages\torch\cuda_init_.py", line 239, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled