Avoid duplicate stat calls during startup/loading #55331
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
IanButterworth
commented
Aug 1, 2024
Comment on lines
824
to
835
| function entry_point_and_project_file(dir::String, name::String)::Union{Tuple{Nothing,Nothing},Tuple{String,Nothing},Tuple{String,String}} | ||
| path = normpath(joinpath(dir, "$name.jl")) | ||
| isfile_casesensitive(path) && return path, nothing | ||
| dir_name = joinpath(dir, name) | ||
| path, project_file = entry_point_and_project_file_inside(dir_name, name) | ||
| path === nothing || return path, project_file | ||
| dir_jl = dir_name * ".jl" | ||
| path, project_file = entry_point_and_project_file_inside(dir_jl, name) | ||
| path === nothing || return path, project_file | ||
| # check for less likely case with a bare file and no src directory last to minimize stat calls | ||
| path = normpath(joinpath(dir, "$name.jl")) | ||
| isfile_casesensitive(path) && return path, nothing | ||
| return nothing, nothing | ||
| end |
There was a problem hiding this comment.
I don't believe this is a breaking change in precedence between a bare Foo.jl and Foo.jl/src/Foo.jl because both couldn't coexist?
fingolfin
reviewed
Aug 1, 2024
base/stat.jl
Outdated
| stat(path::AbstractString) = @stat_call jl_stat Cstring path | ||
| lstat(path::AbstractString) = @stat_call jl_lstat Cstring path | ||
| function stat(path::AbstractString) | ||
| @debug "stat($(repr(path)))" exception=(ErrorException("Fake error for backtrace printing"),stacktrace()) |
There was a problem hiding this comment.
Is this supposed to stay in?
There was a problem hiding this comment.
OP says
I left the
@debugs in as they might be useful for similar investigations.
|
I personally don't really like leaving these |
|
Agreed - they also add an inference burden, can inhibit inlining, and even introduce spurious allocations. |
IanButterworth
commented
Aug 1, 2024
This reverts commit 67f9a1251c1afefcd8318b366dfa787fd2a2deed.
IanButterworth
force-pushed
the
ib/less_stat_loading
branch
from
5934304
to
c4b1914
Compare
|
@KristofferC any idea why I had to revert Prior to that I was getting lots of |
|
Not sure from just looking at the code. |
lazarusA
pushed a commit
to lazarusA/julia
that referenced
this pull request
Aug 17, 2024
Avoids immediately successive stat calls for the same path during startup & loading. According to MacOS Instruments this reduces `stat64` calls during `--start=no -e "using Pkg"` from 844 to 672. On MacOS I don't see a speed improvement, but on WSL2 @timholy reported the test from JuliaLang#55171 sees a modest improvement. This PR (10 iterations) ``` tim@diva:~/.julia/bin$ time for i in {1..10}; do ./cli --help &> /dev/null; done real 0m2.999s user 0m3.547s sys 0m6.552s ``` master ``` real 0m3.217s user 0m3.794s sys 0m6.700s ``` 1.11-rc2: ``` real 0m3.746s user 0m4.169s sys 0m6.755s ``` I left the `@debug`s in as they might be useful for similar investigations.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Avoids immediately successive stat calls for the same path during startup & loading.
According to MacOS Instruments this reduces
stat64calls during--start=no -e "using Pkg"from 844 to 672.On MacOS I don't see a speed improvement, but on WSL2 @timholy reported the test from #55171 sees a modest improvement.
This PR (10 iterations)
master
1.11-rc2:
I left the
@debugs in as they might be useful for similar investigations.