big startup time regression with v1.11 release

[nsajko]

There was a regression between the last release candidate and the release:

$ hyperfine './julia-1.10.5/bin/julia --startup-file=no -e "1+1"'
Benchmark 1: ./julia-1.10.5/bin/julia --startup-file=no -e "1+1"
  Time (mean ± σ):     160.0 ms ±   0.9 ms    [User: 77.7 ms, System: 84.9 ms]
  Range (min … max):   158.2 ms … 161.3 ms    18 runs
 
$ hyperfine './julia-1.11.0-rc4/bin/julia --startup-file=no -e "1+1"'
Benchmark 1: ./julia-1.11.0-rc4/bin/julia --startup-file=no -e "1+1"
  Time (mean ± σ):     126.1 ms ±   1.6 ms    [User: 61.7 ms, System: 63.3 ms]
  Range (min … max):   122.8 ms … 128.6 ms    23 runs
 
$ hyperfine './julia-1.11.0/bin/julia --startup-file=no -e "1+1"'
Benchmark 1: ./julia-1.11.0/bin/julia --startup-file=no -e "1+1"
  Time (mean ± σ):     304.6 ms ±   4.4 ms    [User: 528.1 ms, System: 84.3 ms]
  Range (min … max):   300.0 ms … 315.1 ms    10 runs

Maybe because of this additional required compilation:

$ ./julia-1.11.0-rc4/bin/julia --startup-file=no -e "1+1" --trace-compile=stderr
$ ./julia-1.11.0/bin/julia --startup-file=no -e "1+1" --trace-compile=stderr
precompile(Tuple{Base.var"#759#760"{Task}})

Version info:

Julia Version 1.11.0
Commit 501a4f25c2b (2024-10-07 11:40 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × AMD Ryzen 3 5300U with Radeon Graphics
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver2)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
Environment:
  JULIA_NUM_PRECOMPILE_TASKS = 4
  JULIA_PKG_PRECOMPILE_AUTO = 0

[ufechner7]

I think the label "performance" is wrong. This is a latency regression.

[maleadt]

Bisected the Task-related compilation to #55909

b3181b8f0a71df57dac45756a03818170aeecd85 is the first bad commit
commit b3181b8f0a71df57dac45756a03818170aeecd85
Author: Jameson Nash <[email protected]>
Date:   Fri Sep 27 21:49:29 2024 -0400

    do not intentionally suppress errors in precompile script from being reported or failing the result (#55909)

    I was slightly annoying that the build was set up to succeed if this
    step failed, so I removed the error suppression and fixed up the script
    slightly

    (cherry picked from commit 4a4ca9c815207a80ea81b884b196dfeafc3cb877)

 contrib/generate_precompile.jl | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

[Moelf]

That commit looks so harmless

[KristofferC]

Reverting it fixes the regression. Guess we'll get a 1.11.1 early ;)

[ronisbr]

I am seeing a huge pre-compilation increase in 1.11 compared to 1.10. How can I check if my issue is related to this bug or if it is something else?

Julia 1.10

julia> using PrettyTables

julia> A = ones(2, 2)
2×2 Matrix{Float64}:
 1.0  1.0
 1.0  1.0

julia> @time pretty_table(A)
┌────────┬────────┐
│ Col. 1 │ Col. 2 │
├────────┼────────┤
│    1.0 │    1.0 │
│    1.0 │    1.0 │
└────────┴────────┘
  0.032161 seconds (42.28 k allocations: 3.059 MiB, 97.47% compilation time)

Julia 1.11

julia> using PrettyTables

julia> A = ones(2, 2)
2×2 Matrix{Float64}:
 1.0  1.0
 1.0  1.0

julia> @time pretty_table(A)
┌────────┬────────┐
│ Col. 1 │ Col. 2 │
├────────┼────────┤
│    1.0 │    1.0 │
│    1.0 │    1.0 │
└────────┴────────┘
  0.125523 seconds (784.80 k allocations: 42.458 MiB, 34.03% gc time, 99.67% compilation time: 78% of which was recompilation)

[ufechner7]

How can I check if my issue is related to this bug

Test if it also appears with 1.11-rc4.

[ronisbr]

Thanks! It is not. I will open a new issue.