Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Precompilation error while installing MPI.jl #885

Closed
Gauthameshwar opened this issue Nov 5, 2024 · 8 comments
Closed

Precompilation error while installing MPI.jl #885

Gauthameshwar opened this issue Nov 5, 2024 · 8 comments

Comments

@Gauthameshwar
Copy link

Hi, I am new to Julia and am trying out parallelization using MPI. However, while installing it, I got some precompilation error that suggests some dependencies on HDF5 are having trouble.

(@v1.10) pkg> add MPI
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.10/Project.toml`
  [da04e1cc] + MPI v0.20.22
    Updating `~/.julia/environments/v1.10/Manifest.toml`
  [da04e1cc] + MPI v0.20.22
  [eebad327] + PkgVersion v0.3.3
Precompiling project...
  ✗ HDF5 → MPIExt
  1 dependency successfully precompiled in 6 seconds. 444 already precompiled. 3 skipped during auto due to previous errors.

This error does not affect while loading MPI, but when I try to use some other dependency of HDF5, it gives lots of errors that I cant figure out what it means.

julia> using MPI

julia> using JLD
Precompiling JLD
        Info Given JLD was explicitly requested, output will be shown live 
ERROR: LoadError: InitError: could not load library "/home/gautham/.julia/artifacts/69e85a392db842d919cf5759941f8f3e8a0e569e/lib/libhdf5_fortran.so"
libmpi_usempif08.so.40: cannot open shared object file: No such file or directory
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/inPJn/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ HDF5_jll ~/.julia/packages/HDF5_jll/pjOXH/src/wrappers/x86_64-linux-gnu-libgfortran5-cxx11-mpi+openmpi.jl:48
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1134
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1122
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1067
  [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1581
  [9] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1938
 [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [11] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [12] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [14] macro expansion
    @ ./loading.jl:1790 [inlined]
 [15] macro expansion
    @ ./lock.jl:267 [inlined]
 [16] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [17] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [18] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [19] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [20] top-level scope
    @ ~/.julia/packages/HDF5/Z859u/src/api/api.jl:11
 [21] include(mod::Module, _path::String)
    @ Base ./Base.jl:495
 [22] include(x::String)
    @ HDF5 ~/.julia/packages/HDF5/Z859u/src/HDF5.jl:1
 [23] top-level scope
    @ ~/.julia/packages/HDF5/Z859u/src/HDF5.jl:68
 [24] include
    @ ./Base.jl:495 [inlined]
 [25] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2222
 [26] top-level scope
    @ stdin:3
during initialization of module HDF5_jll
in expression starting at /home/gautham/.julia/packages/HDF5/Z859u/src/api/api.jl:1
in expression starting at /home/gautham/.julia/packages/HDF5/Z859u/src/HDF5.jl:1
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to "/home/gautham/.julia/compiled/v1.10/HDF5/jl_k29J1G".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2468
  [3] compilecache
    @ ./loading.jl:2340 [inlined]
  [4] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1974
  [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
  [6] #mkpidlock#6
    @ ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
  [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
  [8] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
  [9] invokelatest
    @ ./essentials.jl:889 [inlined]
 [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2983
 [11] maybe_cachefile_lock
    @ ./loading.jl:2980 [inlined]
 [12] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1970
 [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [14] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [15] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [17] macro expansion
    @ ./loading.jl:1790 [inlined]
 [18] macro expansion
    @ ./lock.jl:267 [inlined]
 [19] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [20] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [21] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [22] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [23] include
    @ ./Base.jl:495 [inlined]
 [24] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2222
 [25] top-level scope
    @ stdin:3
in expression starting at /home/gautham/.julia/packages/JLD/Vxkax/src/JLD.jl:1
in expression starting at stdin:3
  ✗ HDF5
  ✗ HDF5 → MPIExt
  ✗ H5Zblosc
  ✗ JLD
  0 dependencies successfully precompiled in 12 seconds. 27 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

JLD [4138dd39-2aa7-5051-a626-17a0bb65d9c8]

Failed to precompile JLD [4138dd39-2aa7-5051-a626-17a0bb65d9c8] to "/home/gautham/.julia/compiled/v1.10/JLD/jl_RQqwQO".
ERROR: LoadError: InitError: could not load library "/home/gautham/.julia/artifacts/69e85a392db842d919cf5759941f8f3e8a0e569e/lib/libhdf5_fortran.so"
libmpi_usempif08.so.40: cannot open shared object file: No such file or directory
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/inPJn/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ HDF5_jll ~/.julia/packages/HDF5_jll/pjOXH/src/wrappers/x86_64-linux-gnu-libgfortran5-cxx11-mpi+openmpi.jl:48
  [5] run_module_init(mod::Module, i::Int64)
    @ Base ./loading.jl:1134
  [6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1122
  [7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1067
  [8] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1581
  [9] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1938
 [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [11] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [12] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [14] macro expansion
    @ ./loading.jl:1790 [inlined]
 [15] macro expansion
    @ ./lock.jl:267 [inlined]
 [16] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [17] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [18] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [19] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [20] top-level scope
    @ ~/.julia/packages/HDF5/Z859u/src/api/api.jl:11
 [21] include(mod::Module, _path::String)
    @ Base ./Base.jl:495
 [22] include(x::String)
    @ HDF5 ~/.julia/packages/HDF5/Z859u/src/HDF5.jl:1
 [23] top-level scope
    @ ~/.julia/packages/HDF5/Z859u/src/HDF5.jl:68
 [24] include
    @ ./Base.jl:495 [inlined]
 [25] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2222
 [26] top-level scope
    @ stdin:3
during initialization of module HDF5_jll
in expression starting at /home/gautham/.julia/packages/HDF5/Z859u/src/api/api.jl:1
in expression starting at /home/gautham/.julia/packages/HDF5/Z859u/src/HDF5.jl:1
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to "/home/gautham/.julia/compiled/v1.10/HDF5/jl_k29J1G".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2468
  [3] compilecache
    @ ./loading.jl:2340 [inlined]
  [4] (::Base.var"#968#969"{Base.PkgId})()
    @ Base ./loading.jl:1974
  [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93
  [6] #mkpidlock#6
    @ ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined]
  [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111
  [8] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
  [9] invokelatest
    @ ./essentials.jl:889 [inlined]
 [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:2983
 [11] maybe_cachefile_lock
    @ ./loading.jl:2980 [inlined]
 [12] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1970
 [13] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [14] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [15] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [16] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [17] macro expansion
    @ ./loading.jl:1790 [inlined]
 [18] macro expansion
    @ ./lock.jl:267 [inlined]
 [19] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [20] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [21] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [22] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
 [23] include
    @ ./Base.jl:495 [inlined]
 [24] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2222
 [25] top-level scope
    @ stdin:3
in expression starting at /home/gautham/.julia/packages/JLD/Vxkax/src/JLD.jl:1
in expression starting at stdin:
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70
  [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{…}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, timing::Bool, _from_loading::Bool, kwargs::@Kwargs{…})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:1700
  [3] precompile(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::@Kwargs{_from_loading::Bool})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:159
  [4] precompile
    @ ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:147 [inlined]
  [5] #precompile#114
    @ ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Pkg/src/API.jl:146 [inlined]
  [6] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
  [7] invokelatest
    @ ./essentials.jl:889 [inlined]
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1963
  [9] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [10] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [11] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [12] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [13] macro expansion
    @ ./loading.jl:1790 [inlined]
 [14] macro expansion
    @ ./lock.jl:267 [inlined]
 [15] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [16] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [17] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [18] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
Some type information was truncated. Use `show(err)` to see complete types.

Did this installation break something in the library registry? Could someone please help me out? Thank you very much!
@Gauthameshwar Gauthameshwar changed the title Precomopilation error while installing MPI.jl Precompilation error while installing MPI.jl Nov 5, 2024
@luraess
Copy link
Contributor

luraess commented Nov 5, 2024

Hi, on which platform do you see these issue?

@giordano
Copy link
Member

giordano commented Nov 5, 2024

Works for me on x86_64-linux-gnu, which is what you're using based on the tree hash of the artifact which appears in the error message (but as Ludovic suggested, providing system information would be best when reporting bugs):

% julia --project=. -q
(tmp.yMPKlFoVMv) pkg> add HDF5
   Resolving package versions...
   Installed OpenMPI_jll ─ v4.1.6+0
   Installed HDF5_jll ──── v1.14.3+3
   Installed HDF5 ──────── v0.17.2
  Downloaded artifact: HDF5
    Updating `/tmp/tmp.yMPKlFoVMv/Project.toml`
  [f67ccb44] + HDF5 v0.17.2
    Updating `/tmp/tmp.yMPKlFoVMv/Manifest.toml`
  [34da2185] + Compat v4.16.0
  [f67ccb44] + HDF5 v0.17.2
  [692b3bcd] + JLLWrappers v1.6.1
  [3da0fdf6] + MPIPreferences v0.1.11
  [21216c6a] + Preferences v1.4.3
  [ae029012] + Requires v1.3.0
  [0234f1f7] + HDF5_jll v1.14.3+3
  [e33a78d0] + Hwloc_jll v2.11.2+0
  [7cb0a576] + MPICH_jll v4.2.3+0
  [f1f71cc9] + MPItrampoline_jll v5.5.1+0
  [9237b28f] + MicrosoftMPI_jll v10.1.4+2
⌅ [fe0851c0] + OpenMPI_jll v4.1.6+0
  [458c3c95] + OpenSSL_jll v3.0.15+1
  [477f73a3] + libaec_jll v1.1.2+0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [a63ad114] + Mmap v1.11.0
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] + LibCURL_jll v8.6.0+0
  [e37daf67] + LibGit2_jll v1.7.2+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.6+0
  [14a3606d] + MozillaCACerts_jll v2023.12.12
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850ede] + nghttp2_jll v1.59.0+0
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  3 dependencies successfully precompiled in 6 seconds. 37 already precompiled.

julia> using HDF5

(tmp.yMPKlFoVMv) pkg> add MPI
   Resolving package versions...
    Updating `/tmp/tmp.yMPKlFoVMv/Project.toml`
  [da04e1cc] + MPI v0.20.22
    Updating `/tmp/tmp.yMPKlFoVMv/Manifest.toml`
  [ffbed154] + DocStringExtensions v0.9.3
  [da04e1cc] + MPI v0.20.22
  [eebad327] + PkgVersion v0.3.3
  [aea7be01] + PrecompileTools v1.2.1
  [8ba89e20] + Distributed v1.11.0
  [9e88b42a] + Serialization v1.11.0
  [6462fe0b] + Sockets v1.11.0
Precompiling project...
  1 dependency successfully precompiled in 1 seconds. 46 already precompiled.

julia> using MPI

julia> MPI.versioninfo()
MPIPreferences:
  binary:  MPICH_jll
  abi:     MPICH

Package versions
  MPI.jl:             0.20.22
  MPIPreferences.jl:  0.1.11
  MPICH_jll:          4.2.3+0

Library information:
  libmpi:  /home/mose/.julia/artifacts/e85c0a68e07fee0ee7b19c2abc210b1af2f4771a/lib/libmpi.so
  libmpi dlpath:  /home/mose/.julia/artifacts/e85c0a68e07fee0ee7b19c2abc210b1af2f4771a/lib/libmpi.so
  MPI version:  4.1.0
  Library version:  
    MPICH Version:      4.2.3
    MPICH Release date: Wed Oct  2 09:35:21 AM CDT 2024
    MPICH ABI:          16:3:4
    MPICH Device:       ch3:nemesis
    MPICH configure:    --prefix=/workspace/destdir --build=x86_64-linux-musl --host=x86_64-linux-gnu --disable-dependency-tracking --disable-doc --enable-fast=all,O3 --enable-static=no --with-device=ch3 --with-hwloc=/workspace/destdir
    MPICH CC:           cc    -DNDEBUG -DNVALGRIND -O3
    MPICH CXX:          c++   -DNDEBUG -DNVALGRIND -O3
    MPICH F77:          gfortran   -O3
    MPICH FC:           gfortran   -O3
    MPICH features:

However I note that 69e85a392db842d919cf5759941f8f3e8a0e569e refers to the OpenMPI build of HDF5, looks like your configuration is a bit messed up.

@luraess
Copy link
Contributor

luraess commented Nov 5, 2024

I know it fails for some colleagues on macOS with Apple silicon chips...

@Gauthameshwar
Copy link
Author

Gauthameshwar commented Nov 5, 2024 •
edited
Loading

Hi, on which platform do you see these issue?

I use MacOS Sonoma. My Julia version is v1.10.

I know it fails for some colleagues on macOS with Apple silicon chips...

@luraess I have Apple M2 max processor. Is it a common error for Mac users? Would you happen to know of any workaround?

@giordano
Copy link
Member

giordano commented Nov 5, 2024

I use MacOS Sonoma. My Julia version is v1.10.

Well, no you've got to explain how you ended up with Linux libraries.

@Gauthameshwar
Copy link
Author

Oh, sorry, my bad! I got these errors while installing MPI.jl on a remote server from my machine. The remote server runs Ubuntu. Here are the details of the remote server's OS.

gautham@m30:~$ screenfetch
                          ./+o+-       gautham@m30
                  yyyyy- -yyyyyy+      OS: Ubuntu 16.04 xenial
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 4.4.0-210-generic
           .++ .:/++++++/-.+sss/`      Uptime: 18d 4m
         .:++o:  /++++++++/:--:/-      Packages: 2379
        o:+o+:++.`..```.-/oo+++++/     Shell: shellIntegration-bash.sh
       .:+o:+o/.          `+sssoo+/    CPU: 2x Intel Xeon CPU E5-2698 v4 @ 3.6GHz
  .++/+:+oo+o:`             /sssooo.   RAM: 2458MiB / 257831MiB
 /+++//+:`oo+o               /::--:.  
 \+/+o+++`o++o               ++////.  
  .++.o+++oo+:`             /dddhhh.  
       .+.o+oo:.          `oddhhhh+   
        \+.++o+o``-````.:ohdhhhhh+    
         `:o+++ `ohhhhhhhhyo++os:     
           .o:`.syhhhhhhh/.oo++o`     
               /osyyyyyyo++ooo+++/    
                   ````` +oo+++o\:    
                          `oo++.

@giordano
Copy link
Member

giordano commented Nov 6, 2024

I know it fails for some colleagues on macOS with Apple silicon chips...

Works for me:

julia> versioninfo()
Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

(jl_gFatz2) pkg> add HDF5
   Resolving package versions...
   Installed OpenMPI_jll ─ v4.1.6+0
   Installed HDF5_jll ──── v1.14.3+3
   Installed libaec_jll ── v1.1.2+0
   Installed HDF5 ──────── v0.17.2
  Downloaded artifact: libaec
  Downloaded artifact: HDF5
    Updating `/private/var/folders/v2/hmy3kzgj4tb3xsy8qkltxd0r0000gn/T/jl_gFatz2/Project.toml`
  [f67ccb44] + HDF5 v0.17.2
    Updating `/private/var/folders/v2/hmy3kzgj4tb3xsy8qkltxd0r0000gn/T/jl_gFatz2/Manifest.toml`
  [34da2185] + Compat v4.16.0
  [f67ccb44] + HDF5 v0.17.2
  [692b3bcd] + JLLWrappers v1.6.1
  [3da0fdf6] + MPIPreferences v0.1.11
  [21216c6a] + Preferences v1.4.3
  [ae029012] + Requires v1.3.0
  [0234f1f7] + HDF5_jll v1.14.3+3
  [e33a78d0] + Hwloc_jll v2.11.2+0
  [7cb0a576] + MPICH_jll v4.2.3+0
  [f1f71cc9] + MPItrampoline_jll v5.5.1+0
  [9237b28f] + MicrosoftMPI_jll v10.1.4+2
⌅ [fe0851c0] + OpenMPI_jll v4.1.6+0
  [458c3c95] + OpenSSL_jll v3.0.15+1
  [477f73a3] + libaec_jll v1.1.2+0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [a63ad114] + Mmap v1.11.0
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.11.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
  [deac9b47] + LibCURL_jll v8.6.0+0
  [e37daf67] + LibGit2_jll v1.7.2+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.6+0
  [14a3606d] + MozillaCACerts_jll v2023.12.12
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850ede] + nghttp2_jll v1.59.0+0
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  4 dependencies successfully precompiled in 8 seconds. 36 already precompiled.

julia> using HDF5

(jl_gFatz2) pkg> add MPI
   Resolving package versions...
    Updating `/private/var/folders/v2/hmy3kzgj4tb3xsy8qkltxd0r0000gn/T/jl_gFatz2/Project.toml`
  [da04e1cc] + MPI v0.20.22
    Updating `/private/var/folders/v2/hmy3kzgj4tb3xsy8qkltxd0r0000gn/T/jl_gFatz2/Manifest.toml`
  [ffbed154] + DocStringExtensions v0.9.3
  [da04e1cc] + MPI v0.20.22
  [eebad327] + PkgVersion v0.3.3
  [aea7be01] + PrecompileTools v1.2.1
  [8ba89e20] + Distributed v1.11.0
  [9e88b42a] + Serialization v1.11.0
  [6462fe0b] + Sockets v1.11.0
Precompiling project...
  1 dependency successfully precompiled in 1 seconds. 46 already precompiled.

julia> using MPI

julia> MPI.versioninfo()
MPIPreferences:
  binary:  MPICH_jll
  abi:     MPICH

Package versions
  MPI.jl:             0.20.22
  MPIPreferences.jl:  0.1.11
  MPICH_jll:          4.2.3+0

Library information:
  libmpi:  /Users/mose/.julia/artifacts/5b62c899cf8329a8d07e8fe6eda96039010046c7/lib/libmpi.12.dylib
  libmpi dlpath:  /Users/mose/.julia/artifacts/5b62c899cf8329a8d07e8fe6eda96039010046c7/lib/libmpi.12.dylib
  MPI version:  4.1.0
  Library version:
    MPICH Version:      4.2.3
    MPICH Release date: Wed Oct  2 09:35:21 AM CDT 2024
    MPICH ABI:          16:3:4
    MPICH Device:       ch3:nemesis
    MPICH configure:    --prefix=/workspace/destdir --build=x86_64-linux-musl --host=aarch64-apple-darwin20 --disable-dependency-tracking --disable-doc --enable-fast=all,O3 --enable-static=no --with-device=ch3 --with-hwloc=/workspace/destdir FFLAGS=-fallow-argument-mismatch FCFLAGS=-fallow-argument-mismatch
    MPICH CC:           cc   -fno-common  -DNDEBUG -DNVALGRIND -O3
    MPICH CXX:          c++   -DNDEBUG -DNVALGRIND -O3
    MPICH F77:          gfortran -fallow-argument-mismatch  -O3
    MPICH FC:           gfortran -fallow-argument-mismatch  -O3
    MPICH features:

If anyone has problems there it'd be good to report that, but that's unrelated to this ticket, which is apparently related to a misconfiguration on @Gauthameshwar's side

@Gauthameshwar
Copy link
Author

@giordano you are right. It seems to be an issue with just my current Julia dependencies. I tried to upgrade to v1.11 and reinstall MPI and HDF5. It worked fine there and the output was just like yours above. I ran a build() on HDF5 suspecting this and it said some of my dependencies are configured to an older version of Julia and that might lead to some discrepancies.

I fixed it by rm HDF5 and add HDF5. Now it runs in v1.10 without any issues. Thank you very much for your valuable discussion. Fortunately, MPI.jl did not have any bugs and I learned how to identify and fix broken dependencies :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@giordano @Gauthameshwar @luraess