Fooocus 2.1.25: Supporting CUDA 12.1

[lllyasviel]

Fooocus 2.1.25 begins to support CUDA 12.1.

If you download Fooocus again (download "Fooocus_win64_2-1-25.7z"), your Fooocus will use CUDA 12.1, and it is a bit faster.

In my tests, for 30XX cards, it will be about 4% faster. For 40XX cards, it will be about 13% faster. For 20XX/10XX cards, no obvious speed difference are observed.

On 4090, generating one image with default parameters only take about 3.8 seconds in my tests. This is remarkable.

If you do not download Fooocus again, nothing will happen. You can still use it without any problems. The automatic updating launcher only updates scripts. It will not change your CUDA if you do not re-download Fooocus.

You can post CUDA-related problems here for the newer version.

[fatinghenji]

我需要额外的配置cuda环境吗？我有其他的软件在使用其他版本的cuda。是否会造成现在已有的cuda环境的损坏呢？

[lanyusan]

On unbutun 22.04, it seems xformers doesn't work with cuda 12.1.

  File "Fooocus/venv/lib/python3.10/site-packages/xformers/ops/fmha/dispatch.py", line 69, in _run_priority_list
    raise NotImplementedError(msg)

NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(20, 4032, 1, 64) (torch.float16)
     key         : shape=(20, 4032, 1, 64) (torch.float16)
     value       : shape=(20, 4032, 1, 64) (torch.float16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`flshattF` is not supported because:
    xFormers wasn't build with CUDA support
    Operator wasn't built - see `python -m xformers.info` for more info
`tritonflashattF` is not supported because:
    xFormers wasn't build with CUDA support
    requires A100 GPU
    Only work on pre-MLIR triton for now
`cutlassF` is not supported because:
    xFormers wasn't build with CUDA support
    Operator wasn't built - see `python -m xformers.info` for more info
`smallkF` is not supported because:
    xFormers wasn't build with CUDA support
    dtype=torch.float16 (supported: {torch.float32})
    max(query.shape[-1] != value.shape[-1]) > 32
    Operator wasn't built - see `python -m xformers.info` for more info
    unsupported embed per head: 6

[lllyasviel]

pip uninstall xformers

[lanyusan]

Uninstall works. Thanks.

[lanyusan]

but if i have several SD as webui or invoke , comfy, if i uninstall xformers they no more be usable by those other installations of SD ? or is it specific to fooocus installation Venv ?

• also lost of Xformers is some time lost too ...

Use virtual environment.

[AkoZ]

ah ! ?? yep for sure many SD ui have Venv .. but there for fooocus i don't see any Venv dir ? as yours !
( ""Fooocus/venv/lib/python3.10/site-packages/xfor...")

• under windows is fooocus/python_embeded but no activate.ps1 or anything to start a Venv ?
• ok may be to use the cuda and not interfere between other installed SD, i've to create the fooocus/Venv ? and install CUDA 12.1 ver
• today is "Cuda compilation tools, release 11.7, V11.7.64" but even the fooocus 2.1.54 doesn't indicate any CUDA version... so may be ?

[rockmanxyj]

Hello lllyasviel, are you using the 4090TI? I can't achieve this speed with my 4090 under CUDA 12.2.

[Alarukai]

On Cuda 12.3 i can't get the image generation to work. Running on 4070.

Getting:
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

Seems like a Pythorch error?

Full Error Log:
ErrorLog.txt

[lwrnckahiga88]

hello am having a hard time accessing the fooocus -2.1.865 repository .1. does it exist ? 2. am also having a hard time downloading the win64_2-1-831.7z version ..it stops downloading at 1.62 GB/1.83 GB ON MY CHROME ,MOZILLA BROWSERS...IS IT A PATCH ISSUE OR A CONFIG PROBLEM?