Request for support for CUDA 12.4 #67571
Assignees
Labels
Comments
|
Currently 12.3 is supported. This version can already support Flash Attention 2. You can give this version a try. There is no obvious difference between 12.4 and 12.3. |
The only problem is that PyTorch does not have wheels for CUDA 12.3... |
13 tasks
emilazy
added a commit
to emilazy/nixpkgs
that referenced
this issue
Sep 25, 2024
We currently package all CUDA versions from 10.0 onwards. In some cases, CUDA is the only thing preventing us from removing old versions of GCC. Since we currently don’t deprecate or remove CUDA versions, this will be an increasing drag on compiler maintenance in Nixpkgs going forward unless we establish a sensible policy. After discussing this with @SomeoneSerge in the context of old versions of GCC, I learned that there was already a desire to remove at least versions prior to 11.3, as those versions were only packaged in the old “runfile” format, but that it was blocked on someone doing the work to warn about the upcoming deprecation for a release cycle. This change adds a release note and warnings indicating that CUDA 10.x and 11.x will be removed in Nixpkgs 25.05, about 8 months from now. I chose this version cut‐off because these versions of CUDA require GCC < 12. GCC releases a major version every year, and seems to support about four releases at a time, releasing the last update to the oldest version and marking it as unsupported on their site around the time of the release of the next major version. Therefore, by the time of the 25.05 release, we should expect GCC 15 to be released and GCC 11 to become unsupported. Adding a warning and communicating the policy of only shipping CUDA versions that work with supported compilers in the release notes means that we should be able to clean up old versions as required without any issue or extensive deprecation period in future, without obligating us to do so if there is a strongly compelling reason to be more lenient. That should help solve both shipping an indefinitely‐growing list of CUDA versions and an indefinitely‐growing list of GCC and LLVM versions. As I’m not a user of CUDA myself, I can’t be sure of how sensible this version support policy is, but I think it’s fair to say that it’s reasonable for Nixpkgs to choose not to maintain compiler versions that are unsupported upstream just for the sake of versions of CUDA that are also unmaintained. CUDA 11.x has not received an update for two years already, and would only become unsupported in Nixpkgs in over half a year’s time. CUDA 10.x is currently unused in‐tree except for the unmaintained Caffe and NVIDIA DCGM, which depends on multiple CUDA versions solely so that it can provide plugins for those versions. The latest DCGM version has already removed support for CUDA 10.x and is just awaiting an update in Nixpkgs. They maintain a list of supported versions to build plugins for in their CMake build system, so it should be simple enough for us to only build support for the versions of CUDA that we support in Nixpkgs. From what I can tell, CUDA 11.x is currently used by the following packages other than DCGM: * `catboost`, because of <catboost/catboost#2540>. It looks like upstream has since redesigned this part of their build system, so perhaps the problem is no longer present, or would be easier to fix. * `magma_2_6_2`, an old version from before upstream added CUDA 12 support. This seems okay to break to me; that version is not maintained and will never be updated for new CUDA versions, and the CUDA support is optional. * `paddlepaddle`, which, uh, also requires OpenSSL 1.1 of all things. <PaddlePaddle/Paddle#67571> states that PaddlePaddle supports up to 12.3. * `python3Packages.cupy`, which is listed as “possibly incompatible with cutensor 2.0 that comes with `cudaPackages_12`”. I’m not sure what the “possibly” means here, but according to <https://github.com/cupy/cupy/tree/v13.3.0?tab=readme-ov-file#installation> they ship binary wheels using CUDA 12.x so I think this should be fine. * `python3Packages.tensorrt`, which supports CUDA 12.x going by <https://github.com/NVIDIA/TensorRT/blob/release/10.4/CMakeLists.txt#L111>. * TensorFlow, which has a link to <https://www.tensorflow.org/install/source#gpu> above the `python3Packages.tensorflow-bin` definition, but that page lists the versions we package as supporting CUDA 12.x. Given the years since CUDA 11.x received any update upstream, and the seemingly very limited set of packages that truly require it, I think the policy of being able to drop versions that require unsupported compilers starting from the next Nixpkgs release is a reasonable one, but of course I’m open to feedback from the CUDA maintainers about this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
需求描述 Feature Description
I would like to request support for CUDA 12.4, at least on Windows.
Currently, it is impossible to have the latest versions of PyTorch, Flash Attention 2 and PaddlePaddle installed at the same time.
The reason is that PyTorch requires CUDA 11.8, 12.1 or 12.4, Flash Attention 2 requires 12.XX and PaddlePaddle requires 11.2, 11.6, 11.7, 11.8, 12.0 or 12.3.
替代实现 Alternatives
No response
The text was updated successfully, but these errors were encountered: