Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not load CUTENSOR artifact dll on Windows 10 #355

Closed
stillyslalom opened this issue Aug 6, 2020 · 31 comments
Closed

Could not load CUTENSOR artifact dll on Windows 10 #355

stillyslalom opened this issue Aug 6, 2020 · 31 comments
Labels
bug Something isn't working installation CUDA is easy to install, right?

Comments

@stillyslalom
Copy link
Contributor

Describe the bug

When installing CUDA#master in Julia 1.5 on Windows 10 (after nuking .julia/ to start fresh), a (504) Gateway Timeout exception is thrown while downloading artifacts, followed by an error resulting from failure to load cutensor.dll, despite the existence of the library at the specified path.

To reproduce

The Minimal Working Example (MWE) for this bug:

PS C:\Users\alexa\AppData\Local\Programs\Julia 1.5.0\bin> Set-Variable -Name "JULIA_DEBUG" -Value "CUDA"
julia> using CUDA
[ Info: Precompiling CUDA [052768ef-5323-5732-b1bb-66c8b64840ba]

julia> CUDA.version()
Downloading artifact: CUDA110
Exception calling "DownloadFile" with "2" argument(s): "The remote server returned an error: (504) Gateway Timeout."
At line:12 char:1
+ $webclient.DownloadFile("https://pkg.julialang.org/artifact/4747fe73a ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException
ERROR: could not load library "C:\Users\alexa\.julia\artifacts\804435c7dbe9d002d6caa0a8b3596d11a6faa778\bin\cutensor.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] use_artifact_cutensor(::VersionNumber) at C:\Users\alexa\.julia\packages\CUDA\YU1DS\deps\bindeps.jl:315
 [4] use_artifact_cuda() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\deps\bindeps.jl:177
 [5] __init_dependencies__() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\deps\bindeps.jl:342
 [6] __runtime_init__() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:112
 [7] (::CUDA.var"#609#610"{Bool})() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:32
 [8] lock(::CUDA.var"#609#610"{Bool}, ::ReentrantLock) at .\lock.jl:161
 [9] _functional(::Bool) at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:26
 [10] functional(::Bool) at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:19
 [11] libcuda() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:50
 [12] (::CUDA.var"#676#cache_fptr!#7")() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\utils\call.jl:31
 [13] macro expansion at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\utils\call.jl:39 [inlined]
 [14] macro expansion at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\cudadrv\libcuda.jl:28 [inlined]
 [15] macro expansion at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\cudadrv\error.jl:108 [inlined]
 [16] cuDriverGetVersion(::Base.RefValue{Int32}) at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\utils\call.jl:93
 [17] version() at C:\Users\alexa\.julia\packages\CUDA\YU1DS\lib\cudadrv\version.jl:10
 [18] top-level scope at REPL[3]:1

julia> ispath(raw"C:\Users\alexa\.julia\artifacts\804435c7dbe9d002d6caa0a8b3596d11a6faa778\bin\cutensor.dll")
true
Manifest.toml

# This file is machine-generated - editing it directly is not advised

[[AbstractFFTs]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "051c95d6836228d120f5f4b984dd5aba1624f716"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"
version = "0.5.0"

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "0fac443759fa829ed8066db6cf1077d888bb6573"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "2.0.2"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[BinaryProvider]]
deps = ["Libdl", "Logging", "SHA"]
git-tree-sha1 = "ecdec412a9abc8db54c0efc5548c64dfce072058"
uuid = "b99e7846-7c00-51b0-8f62-c81ae34c0232"
version = "0.5.10"

[[CEnum]]
git-tree-sha1 = "215a9aa4a1f23fbd05b92769fdd62559488d70e9"
uuid = "fa961155-64e5-5f13-b03f-caf6b980ea82"
version = "0.4.1"

[[CUDA]]
deps = ["AbstractFFTs", "Adapt", "BinaryProvider", "CEnum", "DataStructures", "ExprTools", "GPUArrays", "GPUCompiler", "LLVM", "Libdl", "LinearAlgebra", "Logging", "MacroTools", "NNlib", "Pkg", "Printf", "Random", "Reexport", "Requires", "SparseArrays", "Statistics", "TimerOutputs"]
git-tree-sha1 = "33975e799350f944295c6ccfb481cd902a54aa20"
repo-rev = "master"
repo-url = "https://github.com/JuliaGPU/CUDA.jl.git"
uuid = "052768ef-5323-5732-b1bb-66c8b64840ba"
version = "1.2.1"

[[DataStructures]]
deps = ["InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "88d48e133e6d3dd68183309877eac74393daa7eb"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.17.20"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[ExprTools]]
git-tree-sha1 = "6f0517056812fd6aa3af23d4b70d5325a2ae4e95"
uuid = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
version = "0.1.1"

[[GPUArrays]]
deps = ["AbstractFFTs", "Adapt", "LinearAlgebra", "Printf", "Random", "Serialization"]
git-tree-sha1 = "9027d6128645d227fa9ba7d5b3aa00af696b9aaf"
uuid = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
version = "5.0.0"

[[GPUCompiler]]
deps = ["DataStructures", "InteractiveUtils", "LLVM", "Libdl", "TimerOutputs", "UUIDs"]
git-tree-sha1 = "d6db068d8373bb45ac17e14744cc1d3681d0dbc7"
uuid = "61eb1bfa-7361-4325-ad38-22787b887f55"
version = "0.5.5"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[LLVM]]
deps = ["CEnum", "Libdl", "Printf", "Unicode"]
git-tree-sha1 = "a662366a5d485dee882077e8da3e1a95a86d097f"
uuid = "929cbde3-209d-540e-8aea-75f648917ca0"
version = "2.0.0"

[[LibGit2]]
deps = ["Printf"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "f7d2e3f654af75f01ec49be82c231c382214223a"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.5"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[NNlib]]
deps = ["Libdl", "LinearAlgebra", "Pkg", "Requires", "Statistics"]
git-tree-sha1 = "8ec4693a5422f0b064ce324f59351f24aa474893"
uuid = "872c559c-99b0-510c-b3b7-b6c96a88d5cd"
version = "0.7.4"

[[OrderedCollections]]
git-tree-sha1 = "293b70ac1780f9584c89268a6e2a560d938a7065"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.3.0"

[[Pkg]]
deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[Reexport]]
deps = ["Pkg"]
git-tree-sha1 = "7b1d07f411bc8ddb7977ec7f377b97b158514fe0"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "0.2.0"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "d37400976e98018ee840e0ca4f9d20baa231dc6b"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.0.1"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[TimerOutputs]]
deps = ["Printf"]
git-tree-sha1 = "f458ca23ff80e46a630922c555d838303e4b9603"
uuid = "a759f4b9-e2f1-59dc-863e-4aeb61b1ea8f"
version = "0.5.6"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

Expected behavior

Automatic download/installation of binary artifacts

Version info

Details on Julia:

julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 9 4900HS with Radeon Graphics
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, znver1)
Environment:
  JULIA_NUM_THREADS = 8

Details on CUDA:

julia> CUDA.versioninfo()
ERROR: AssertionError: CUDA.jl did not successfully initialize, and is not usable.
Stacktrace:
 [1] macro expansion at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\initialization.jl:50 [inlined]
 [2] toolkit_version at C:\Users\alexa\.julia\packages\CUDA\YU1DS\deps\bindeps.jl:25 [inlined]
 [3] versioninfo(::Base.TTY) at C:\Users\alexa\.julia\packages\CUDA\YU1DS\src\utilities.jl:43 (repeats 2 times)
 [4] top-level scope at REPL[5]:1

Additional context

PS C:\Users\alexa> nvidia-smi
Thu Aug 06 12:06:19 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 451.67       Driver Version: 451.67       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 206... WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   27C    P8    12W /  N/A |    164MiB /  6144MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
@stillyslalom stillyslalom added the bug Something isn't working label Aug 6, 2020
@maleadt
Copy link
Member

maleadt commented Aug 6, 2020

Can you list the files in C:\Users\alexa\.julia\artifacts\804435c7dbe9d002d6caa0a8b3596d11a6faa778\bin\?

@maleadt maleadt added the installation CUDA is easy to install, right? label Aug 6, 2020
@stillyslalom
Copy link
Contributor Author

cutensor.dll is the only file in that folder. There are two other artifact folders containing various other CUDA dlls:
image
image

@maleadt
Copy link
Member

maleadt commented Aug 6, 2020

I'm not sure why the DLL wouldn't load then (The specified module could not be found.), and I don't have much Windows experience to debug that. I'll see if I can reproduce.

@stillyslalom
Copy link
Contributor Author

The cutensor artifact is intact, which rules out download corruption. CUDA 11 support is a pretty recent development, right? Any chance the artifact itself is broken?

julia> artifact_toml = joinpath(pkgdir(CUDA), "Artifacts.toml")
"C:\\Users\\alexa\\.julia\\packages\\CUDA\\YU1DS\\Artifacts.toml"

julia> artifact_hash("CUTENSOR_CUDA110", artifact_toml) |> verify_artifact
true

This is the subset of dlls that's loaded during CUDA.jl initialization before the error:

julia> setdiff(finaldlls, initdlls)
23-element Array{String,1}:
 "C:\\Users\\alexa\\AppData\\Local\\Programs\\Julia 1.5.0\\bin\\libdSFMT.DLL"
 "C:\\WINDOWS\\SYSTEM32\\nvcuda.DLL"
 "C:\\WINDOWS\\SYSTEM32\\VERSION.DLL"
 "C:\\WINDOWS\\system32\\DriverStore\\FileRepository\\nvami.inf_amd64_aceb5b140d1d131e\\nvcuda64.dll"
 "C:\\WINDOWS\\SYSTEM32\\nvapi64.dll"
 "C:\\WINDOWS\\System32\\SETUPAPI.DLL"
 "C:\\WINDOWS\\System32\\cfgmgr32.dll"
 "C:\\WINDOWS\\System32\\SHLWAPI.DLL"
 "C:\\WINDOWS\\SYSTEM32\\dxgi.dll"
 "C:\\Users\\alexa\\AppData\\Local\\Programs\\Julia 1.5.0\\bin\\libopenlibm.DLL"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\cublas64_11.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\cublasLt64_11.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\cusparse64_11.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\cusolver64_10.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\cufft64_10.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\4747fe73a29e16337cb33702edae26b4e681ae52\\bin\\curand64_10.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_ops_infer64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_ops_train64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_cnn_infer64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_cnn_train64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_adv_infer64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn_adv_train64_8.dll"
 "C:\\Users\\alexa\\.julia\\artifacts\\430f2638c1882d64e7673183c57b1564edaba69f\\bin\\cudnn64_8.dll"

@stillyslalom
Copy link
Contributor Author

I downloaded cutensor.dll v11 from Nvidia, and the artifact is identical, so that's not the issue

julia> ct1 = read(raw"C:\Users\alexa\.julia\artifacts\804435c7dbe9d002d6caa0a8b3596d11a6faa778\bin\cutensor.dll");

julia> ct2 = read(raw"C:\Program Files\NVIDIA cuTENSOR\v1.2\lib\11.0\cutensor.dll");

julia> ct1 == ct2
true

@stillyslalom
Copy link
Contributor Author

The issue's coming from CUDA110, which must be improperly linking to cutensor:

julia> artifact_hash("CUDA110", artifact_toml) |> verify_artifact
false

I think it may be related to JuliaLang/Pkg.jl#1892.

@timkimd
Copy link

timkimd commented Aug 13, 2020

I think I'm having similar issue. (I'm using Windows, and I can't load Flux or DiffEqFlux with Julia 1.4.2 or 1.5.0, but the error doesn't occur when using Julia 1.3.1.)

FluxML/Flux.jl#1313

@charleskawczynski
Copy link
Contributor

I'm also using Windows, and I can't load CUDA with Julia 1.4.2 or 1.5.0, but I can with Julia 1.3.1.

@stillyslalom
Copy link
Contributor Author

Which version of CUDA are you running? (check with nvidia-smi in the console). I'm on 11.0. It's unclear whether it's just a problem with the 11.0 DLLs, or if it's more general than that.

@charleskawczynski
Copy link
Contributor

Hm, not sure how to use the nvidia-smi command:

bash: nvidia-smi: command not found

But,

julia> CUDA.version()
┌ Warning: `haskey(::TargetIterator, name::String)` is deprecated, use `Target(; name=name) !== nothing` instead.
│   caller = llvm_compat(::VersionNumber) at compatibility.jl:181
└ @ CUDA C:\Users\kawcz\.julia\packages\CUDA\42B9G\deps\compatibility.jl:181
v"10.2.0"
julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

@stillyslalom
Copy link
Contributor Author

You should be able to run nvidia-smi from one of the built-in Windows terminals (cmd or Powershell), but yeah, you're on CUDA v10.2. What's the error message you see on Julia 1.5?

@charleskawczynski
Copy link
Contributor

I believe it was the same, or very similar to the issue post.

@stillyslalom
Copy link
Contributor Author

The download issue should be fixed now, which leaves us with a secondary issue involving improperly-linked DLLs. Can you confirm that you're able to download those artifacts without a 502/504 error?

@HenriDeh
Copy link
Contributor

I get 504 error when downloading CUDNN_CUDA102 on CUDA#master. Not fixed evidently.

@maleadt
Copy link
Member

maleadt commented Aug 16, 2020

I get 504 error when downloading CUDNN_CUDA102 on CUDA#master. Not fixed evidently.

Please refer to JuliaLang/Pkg.jl#1892; this is not an issue on the CUDA.jl side.

@maleadt maleadt changed the title Artifact download fails on Windows 10 Could not load CUTENSOR artifact dll on Windows 10 Aug 16, 2020
@maleadt
Copy link
Member

maleadt commented Aug 16, 2020

For the people where the CUTENSOR dll fails to load, which version of Windows are you using exactly? Could you open the CUTENSOR dll in Dependencies and report what you see:

image

@HenriDeh
Copy link
Contributor

HenriDeh commented Aug 17, 2020

@maleadt I also have the cutensor load issue. I am on windows 10 family version 1903. Here is what Dependencies shows :
image

Hoovering over the ? entry displays "VCRUNTIME140_1.dll module could not be found on disk"
Tell me if you need more details.

@stillyslalom
Copy link
Contributor Author

Try installing Visual Studio C++ - that provides VCRUNTIME. After installing, I'm able to initialize CUDA.jl:

julia> CUDA.versioninfo()
CUDA toolkit 11.0.194, artifact installation
CUDA driver 11.0.0
NVIDIA driver 451.67.0

Libraries:
- CUBLAS: 11.1.0
- CURAND: 10.2.1
- CUFFT: 10.2.0
- CUSOLVER: 10.5.0
- CUSPARSE: 11.1.0
- CUPTI: 13.0.0
- NVML: 11.0.0+451.67
- CUDNN: 8.0.1 (for CUDA 11.0.0)
- CUTENSOR: 1.2.0 (for CUDA 11.0.0)

Toolchain:
- Julia: 1.5.0
- LLVM: 9.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4
- Device support: sm_35, sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75

1 device(s):
- GeForce RTX 2060 with Max-Q Design (sm_75, 5.840 GiB / 6.000 GiB available)

@HenriDeh
Copy link
Contributor

HenriDeh commented Aug 17, 2020

I directly downloaded the dll from here and now CUDA works. (save it in C:\Windows\sytem32)

@stillyslalom
Copy link
Contributor Author

stillyslalom commented Aug 17, 2020

It may suffice to install the 2015-2019 VS C++ redistributable. @charleskawczynski can you give that a shot? If so, we can just add a note to the installation instructions.

@charleskawczynski
Copy link
Contributor

It may suffice to install the 2015-2019 VS C++ redistributable. @charleskawczynski can you give that a shot? If so, we can just add a note to the installation instructions.

This doesn't seem like the right way of fixing this problem.

I'm running into this again after updating CUDA.

[ Info: Precompiling ClimateMachine [777c4786-024f-11e9-21a3-85d5d4106250]
┌ Error: Could not load CUTENSOR; please file an issue (if on Windows, be sure to install the VS C++ redistributable first)
│   exception =
│    could not load library "C:\Users\kawcz\.julia\artifacts\d6605f4aac98265f0bc1e17d7b6442ffbdee6e9a\bin\cutensor.dll"
│    The specified module could not be found.
│
│    Stacktrace:
│     [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Libdl\src\Libdl.jl:109
│     [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
│     [3] use_artifact_cutensor(::VersionNumber) at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\deps\bindeps.jl:318
│     [4] use_artifact_cuda() at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\deps\bindeps.jl:179
│     [5] __init_dependencies__() at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\deps\bindeps.jl:355
│     [6] __runtime_init__() at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:110
│     [7] (::CUDA.var"#609#610"{Bool})() at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:32
│     [8] lock(::CUDA.var"#609#610"{Bool}, ::ReentrantLock) at .\lock.jl:151
│     [9] _functional(::Bool) at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:26
│     [10] functional(::Bool) at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:19
│     [11] has_cuda at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:173 [inlined]
│     [12] has_cuda_gpu at C:\Users\kawcz\.julia\packages\CUDA\dZvbp\src\initialization.jl:183 [inlined] (repeats 2 times)
│     [13] #init#6(::Bool, ::Nothing, ::Bool, ::Base.Iterators.Pairs{Symbol,String,Tuple{Symbol},NamedTuple{(:output_dir,),Tuple{String}}}, ::typeof(ClimateMachine.init)) at C:\Users\kawcz\Dropbox\Caltech\work\dev\CliMA\ClimateMachine.jl\src\Driver\Driver.jl:430
│     [14] (::ClimateMachine.var"#kw##init")(::NamedTuple{(:parse_clargs, :output_dir),Tuple{Bool,String}}, ::typeof(ClimateMachine.init)) at .\none:0
│     [15] top-level scope at C:\Users\kawcz\Dropbox\Caltech\work\dev\CliMA\ClimateMachine.jl\test\Atmos\EDMF\bomex_edmf.jl:2
│     [16] include at .\boot.jl:328 [inlined]
│     [17] include_relative(::Module, ::String) at .\loading.jl:1105
│     [18] include(::Module, ::String) at .\Base.jl:31
│     [19] include(::String) at .\client.jl:424
│     [20] top-level scope at REPL[6]:1
│     [21] eval(::Module, ::Any) at .\boot.jl:330
│     [22] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:86
│     [23] macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:118 [inlined]
│     [24] (::REPL.var"#26#27"{REPL.REPLBackend})() at .\task.jl:333
└ @ CUDA C:\Users\kawcz\.julia\packages\CUDA\dZvbp\deps\bindeps.jl:320
julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PKG_DEVDIR = C:/Users/kawcz/Dropbox/Caltech/work/dev/julia

@maleadt I'm on Windows 10 Home.

@maleadt
Copy link
Member

maleadt commented Sep 2, 2020

Please inspect the dependencies then, as listed higher up.

@charleskawczynski
Copy link
Contributor

Please inspect the dependencies then, as listed higher up.

Per Dependencies documentation:

Dependencies needs Visual C++ Redistributable installed to run properly.

Once installing Visual C++ Redistributable, I'm able to using CUDA. Will installing Visual C++ Redistributable affect what Dependencies reports?

@charleskawczynski
Copy link
Contributor

I am seeing some ? display entries:

image

but the top window looks clear of issues AFAICT

image

@maleadt
Copy link
Member

maleadt commented Sep 2, 2020

So installing the Visual C++ Redistributable fixed it? Isn't that exactly what the CUDA.jl error suggested? Or what did you mean with 'This doesn't seem like the right way of fixing this problem.'?

@charleskawczynski
Copy link
Contributor

So installing the Visual C++ Redistributable fixed it? Isn't that exactly what the CUDA.jl error suggested?

Yes, and yes.

Or what did you mean with 'This doesn't seem like the right way of fixing this problem.'?

I was thinking that everything should be shipped with ] add CUDA. Sorry for the noise, I'm not very familiar with how the installation/deps work.

@maleadt
Copy link
Member

maleadt commented Sep 2, 2020

I was thinking that everything should be shipped with ] add CUDA. Sorry for the noise, I'm not very familiar with how the installation/deps work.

Ah, ok, no that makes sense but I don't think we have that readily available yet (and I thought that the VC C++ redistributable is a common thing to request users to install).

@giordano @staticfloat Do we have the VC C++ redistributable packaged up somehow?

@staticfloat
Copy link
Contributor

No, we don't. I think the best solution here would be to auto-determine that it doesn't exist, and if that is the case, download and run this executable: https://aka.ms/vs/16/release/vc_redist.x64.exe

I think using the actual executable to install the libraries is better than us trying to do it with a JLL, since these are pretty common libraries and it's better for the user to get them in the way that MS intended.

@giordano
Copy link

giordano commented Sep 2, 2020

Shameless self-plug: https://github.com/giordano/DependencyWalker.jl can be useful to debug why a library can't be loaded

@jonniedie
Copy link

It would be nice if there was a way to do this without the MS executable, as that requires admin privileges.

@SeniorCtrlPlayer
Copy link

I directly downloaded the dll from here and now CUDA works. (save it in C:\Windows\sytem32)

Work for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working installation CUDA is easy to install, right?
Projects
None yet
Development

No branches or pull requests

9 participants