Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

precompilation problems with 1.11.0-rc2 #55340

Closed
matthias314 opened this issue Aug 2, 2024 · 5 comments · Fixed by #55355
Closed

precompilation problems with 1.11.0-rc2 #55340

matthias314 opened this issue Aug 2, 2024 · 5 comments · Fixed by #55355
Assignees
@fatteneder

Comments

@matthias314
Copy link
Contributor

matthias314 commented Aug 2, 2024 •
edited
Loading

This issue looks similar to #53983, but details are different. On top of that, I don't know how to reproduce it others can reproduce it. Here is a very simple case of the problem:

I have an existing depot in /usr/local/julia. I make sure that there are no subfolders compiled/v1.11 or environments/v1.11 and then do the following:

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
julia> using Base64
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] 

julia> exit()

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
julia> using Base64
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] (cache misses: wrong source (1))

julia> exit()

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
julia> using Base64
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] (cache misses: wrong source (1))

and so on. With a newly created, completely empty depot the problem does not arise. Is there anything else that I should delete inside the existing depot? Any other ideas?

Julia Version 1.11.0-rc2
Commit 34c3a63147b (2024-07-29 06:24 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i3-10110U CPU @ 2.10GHz
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
Environment:
  JULIA_DEPOT_PATH = /usr/local/julia
@nsajko
Copy link
Contributor

nsajko commented Aug 2, 2024

The "wrong source" message seems to indicate there's some kind of mismatch with some source file. Perhaps you or some program edited a source file? Can you reproduce this after deleting /usr/local/julia-1.11.0-rc2 and extracting the archive again?

@vtjnash
Copy link
Sponsor Member

vtjnash commented Aug 2, 2024

Try with JULIA_DEBUG=loading in your environment to get more debugging metadata associated with the loading.jl file?

@matthias314
Copy link
Contributor Author

@nsajko

Can you reproduce this after deleting /usr/local/julia-1.11.0-rc2 and extracting the archive again?

Yes, it doesn't change everything. I've also verified the SHA256 checksum of the tar file.

@vtjnash This looks promising. I start again with empty /usr/local/julia/compiled/v1.11:

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Base64/D7K0n_G9ii0.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Markdown/AREjX_G9ii0.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/InteractiveUtils/0TrXF_G9ii0.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/StyledStrings/UcVoM_G9ii0.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Unicode/E4Hzs_G9ii0.so for Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/REPL/u0gqU_G9ii0.so for REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb]
└ @ Base loading.jl:1185
julia> using Base64
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] 
┌ Debug: Generating object cache file for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:2805
┌ Debug: Loading object cache file /usr/local/julia/compiled/v1.11/Base64/D7K0n_iafuA.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185

julia> exit()

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Base64/D7K0n_G9ii0.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Markdown/AREjX_G9ii0.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/InteractiveUtils/0TrXF_G9ii0.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/StyledStrings/UcVoM_G9ii0.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Unicode/E4Hzs_G9ii0.so for Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/REPL/u0gqU_G9ii0.so for REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb]
└ @ Base loading.jl:1185
julia> using Base64
┌ Debug: Rejecting cache file /usr/local/julia/compiled/v1.11/Base64/D7K0n_iafuA.ji because it is for file @depot-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl not file /usr/local/julia-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl
└ @ Base loading.jl:3713
┌ Debug: Rejecting cache file /usr/local/julia/compiled/v1.11/Base64/D7K0n_iafuA.ji because it is for file @depot-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl not file /usr/local/julia-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl
└ @ Base loading.jl:3713
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] (cache misses: wrong source (1))
┌ Debug: Generating object cache file for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:2805
┌ Debug: Loading object cache file /usr/local/julia/compiled/v1.11/Base64/D7K0n_iafuA.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185

Note the @depot-1.11.0-rc2 on the second run. I've also noticed that on the first run Base64 is first loaded from /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Base64/D7K0n_G9ii0.so and then after saying using Base64 from /usr/local/julia/compiled/v1.11/Base64/D7K0n_iafuA.so.

With a completely empty /tmp/depot I get the following:

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Base64/D7K0n_G9ii0.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Markdown/AREjX_G9ii0.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/InteractiveUtils/0TrXF_G9ii0.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/StyledStrings/UcVoM_G9ii0.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Unicode/E4Hzs_G9ii0.so for Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/REPL/u0gqU_G9ii0.so for REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb]
└ @ Base loading.jl:1185
julia> using Base64
[ Info: Precompiling Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] 
┌ Debug: Generating object cache file for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:2805
┌ Debug: Loading object cache file /tmp/depot/compiled/v1.11/Base64/D7K0n_j1x0v.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185

julia> exit()

~$ /usr/local/julia-1.11.0-rc2/bin/julia -q --startup-file=no
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Base64/D7K0n_G9ii0.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Markdown/AREjX_G9ii0.so for Markdown [d6f4376e-aef5-505a-96c1-9c027394607a]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/InteractiveUtils/0TrXF_G9ii0.so for InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/StyledStrings/UcVoM_G9ii0.so for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/Unicode/E4Hzs_G9ii0.so for Unicode [4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5]
└ @ Base loading.jl:1185
┌ Debug: Loading object cache file /usr/local/julia-1.11.0-rc2/share/julia/compiled/v1.11/REPL/u0gqU_G9ii0.so for REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb]
└ @ Base loading.jl:1185
julia> using Base64
┌ Debug: Loading object cache file /tmp/depot/compiled/v1.11/Base64/D7K0n_j1x0v.so for Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f]
└ @ Base loading.jl:1185

@matthias314
Copy link
Contributor Author

I can now reproduce it on a different machine, even with a completely empty depot. The problem seems to arise when the path of the depot is the beginning of the path where the Julia distribution is installed. I've taken JULIA_DEPOT_PATH=/tmp/julia and the distribution in /tmp/julia-1.11.0-rc2. If I rename the depot to /tmp/julia-depot, the problem disappears. Renaming the depot to /tmp/juli brings it back, with the error message

julia> using Base64
┌ Debug: Rejecting cache file /tmp/juli/compiled/v1.11/Base64/D7K0n_1PRzk.ji because it is for file @depota-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl not file /tmp/julia-1.11.0-rc2/share/julia/stdlib/v1.11/Base64/src/Base64.jl

Note that now @depota-1.11.0-rc2 (with an extra "a") shows up, corresponding to the shortened depot name (from "/tmp/julia" to "/tmp/juli").

@fatteneder
Copy link
Member

Note that now @depota-1.11.0-rc2 (with an extra "a") shows up, corresponding to the shortened depot name (from "/tmp/julia" to "/tmp/juli").

I discovered this too recently, but forgot to make a PR right away. Will set up a fix later.

@fatteneder fatteneder mentioned this issue Aug 2, 2024
Merged
fatteneder added a commit to fatteneder/julia that referenced this issue Sep 18, 2024
@fatteneder
add test for JuliaLang#55340
deca3fc
fatteneder added a commit to fatteneder/julia that referenced this issue Sep 18, 2024
@fatteneder
add test for JuliaLang#55340
5343f73
KristofferC pushed a commit that referenced this issue Oct 1, 2024
@fatteneder @KristofferC
relocation: account for trailing path separator in depot paths (#55355)
23b7de6 
Fixes #55340

(cherry picked from commit a7c5056)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fatteneder fatteneder

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

relocation: account for trailing path separator in depot paths fatteneder/julia
4 participants
@vtjnash @nsajko @fatteneder @matthias314