Reduce the time-to-first-request by adding pregenerated precompile statements #805

DilumAluthge · 2022-03-10T00:42:37Z

Summary

This pull request makes two changes:

Reduce the time-to-first-request by adding a pregenerated list of precompile statements.
Add a GitHub Actions workflow for automatically regenerating list of precompile statements.

Description

This pull request reduces the time to first HTTP.get("https://example.com/") as shown in the following table. All times are in seconds.

Branch	Sample Size	Median	Δ vs master	Mean ± Standard Deviation	Δ vs master
`master`	20	5.252848		5.318844 ± 0.241039
`dpa/precompile`	20	3.723808	-1.529040	3.947450 ± 0.104822	-1.371394

All measurements were taken using Julia version 1.9.0-DEV.165:

$ julia --project -e 'import InteractiveUtils; InteractiveUtils.versioninfo()'
Julia Version 1.9.0-DEV.165
Commit 8076517c97 (2022-03-09 19:46 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin20.6.0)
  CPU: 4 × Intel(R) Core(TM) i5-4278U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, haswell)
  Threads: 1 on 4 virtual cores

Note: this pull request does not make any web requests when the user does Pkg.precompile() or import HTTP.

List of time measurements

Time measurements for the master branch (click to expand)

$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.332501 seconds (7.34 M allocations: 373.428 MiB, 3.00% gc time, 98.47% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.323024 seconds (7.34 M allocations: 373.428 MiB, 3.16% gc time, 98.49% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.236754 seconds (7.34 M allocations: 373.428 MiB, 3.13% gc time, 98.42% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.235521 seconds (7.34 M allocations: 373.428 MiB, 3.15% gc time, 98.30% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.219502 seconds (7.34 M allocations: 373.428 MiB, 3.14% gc time, 98.42% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.235383 seconds (7.34 M allocations: 373.428 MiB, 3.15% gc time, 98.35% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  6.321999 seconds (7.34 M allocations: 373.428 MiB, 2.64% gc time, 98.65% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.250412 seconds (7.34 M allocations: 373.428 MiB, 3.05% gc time, 98.41% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.323672 seconds (7.34 M allocations: 373.428 MiB, 3.16% gc time, 98.44% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.255284 seconds (7.34 M allocations: 373.428 MiB, 3.09% gc time, 98.32% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.339982 seconds (7.34 M allocations: 373.428 MiB, 3.25% gc time, 98.50% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.318435 seconds (7.34 M allocations: 373.428 MiB, 3.17% gc time, 98.35% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.211436 seconds (7.34 M allocations: 373.428 MiB, 3.14% gc time, 98.43% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.285089 seconds (7.34 M allocations: 373.428 MiB, 2.98% gc time, 98.49% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.203090 seconds (7.34 M allocations: 373.428 MiB, 3.12% gc time, 98.39% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.342181 seconds (7.34 M allocations: 373.428 MiB, 3.08% gc time, 98.47% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.191504 seconds (7.34 M allocations: 373.428 MiB, 3.37% gc time, 98.46% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.272862 seconds (7.34 M allocations: 373.428 MiB, 3.03% gc time, 98.45% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.238501 seconds (7.34 M allocations: 373.428 MiB, 3.32% gc time, 98.49% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.239750 seconds (7.34 M allocations: 373.428 MiB, 3.05% gc time, 98.46% compilation time)

Time measurements for the dpa/precompile branch (click to expand)

$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  4.626500 seconds (76.96 k allocations: 4.030 MiB, 97.84% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  4.705703 seconds (76.96 k allocations: 4.030 MiB, 97.70% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  4.739228 seconds (76.96 k allocations: 4.030 MiB, 97.95% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.817014 seconds (76.96 k allocations: 4.030 MiB, 97.74% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.720888 seconds (76.96 k allocations: 4.030 MiB, 97.91% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.679500 seconds (76.96 k allocations: 4.030 MiB, 97.84% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.669254 seconds (76.96 k allocations: 4.030 MiB, 97.68% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.733760 seconds (76.96 k allocations: 4.030 MiB, 97.75% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  5.142341 seconds (76.96 k allocations: 4.030 MiB, 97.91% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.845968 seconds (76.96 k allocations: 4.030 MiB, 97.80% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.713366 seconds (76.96 k allocations: 4.031 MiB, 97.76% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.703498 seconds (76.96 k allocations: 4.030 MiB, 97.80% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.734557 seconds (76.96 k allocations: 4.030 MiB, 97.74% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.746249 seconds (76.96 k allocations: 4.030 MiB, 97.76% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.782660 seconds (76.96 k allocations: 4.031 MiB, 97.74% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.741732 seconds (76.96 k allocations: 4.030 MiB, 97.64% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.693289 seconds (76.96 k allocations: 4.030 MiB, 97.79% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.713094 seconds (76.96 k allocations: 4.030 MiB, 97.84% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.726727 seconds (76.96 k allocations: 4.030 MiB, 97.77% compilation time)
$ julia --project -e 'import HTTP; @time(HTTP.get("https://example.com"));'
  3.713672 seconds (76.96 k allocations: 4.030 MiB, 97.88% compilation time)

How to regenerate the list of precompile statements

To regenerate the list of precompile statements, there are two options:

If you are a committer on this repository, you can go to the Actions tab of this repository, select the Precompile workflow, and manually trigger a new workflow run. This will automatically open a pull request to update the precompile statements.
You can clone the repository locally and run the following command: julia --project contrib/precompile_generate.jl

codecov-commenter · 2022-03-10T00:47:44Z

Codecov Report

Attention: Patch coverage is 0% with 61 lines in your changes missing coverage. Please review.

Project coverage is 74.79%. Comparing base (96a0b34) to head (73b1344).

❗ Current head 73b1344 differs from pull request most recent head a0df8f8

Please upload reports for the commit a0df8f8 to get more accurate results.

Files	Patch %	Lines
src/precompile.jl	0.00%	61 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (96a0b34) and HEAD (73b1344). Click for more details.

HEAD has 15 uploads more than BASE
| Flag | BASE (96a0b34) | HEAD (73b1344) | |------|------|------| ||3|18|

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #805      +/-   ##
==========================================
- Coverage   83.06%   74.79%   -8.28%     
==========================================
  Files          32       40       +8     
  Lines        3089     2567     -522     
==========================================
- Hits         2566     1920     -646     
- Misses        523      647     +124

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

fonsp · 2022-03-25T00:09:08Z

contrib/precompile_generate.jl

+        end
+    """)
+    println(io, """
+        @static if Base.VERSION < v"1.9-"


This is great, thanks for the PR @DilumAluthge ! I am surprised that this is only enabled for Julia 1.9 and up. We found that adding precompile calls also gave improvements in Julia 1.8.

@DilumAluthge why this is only enabled for 1.9 and later?

So, I was under the impression that without JuliaLang/julia#43990, we can't cache CodeInstances from other packages, and thus the benefit of this would be far less noticeable.

fonsp · 2022-03-25T00:10:07Z

Awesome! I would also like to add precompile statements for HTTP's server abilities, but this PR gives me all the tools to do that in a future PR 😊

rikhuijzer · 2022-04-22T12:17:30Z

How about writing to a release branch based on the version mentioned in Project.toml? Then, only the release branch contains all the precompile statements and that branch is the one where people download the package from. This avoids having to look at the generated output in the diff all the time. Especially for larger packages, the generated output will contain quite a lot of lines.

fredrikekre · 2022-04-22T13:45:53Z

I thought this type of explicit precompile statements wasn't recommended anymore? Experience with Plots.jl also suggest it is a pretty fragile method.

rikhuijzer · 2022-04-29T08:34:17Z

I thought this type of explicit precompile statements wasn't recommended anymore?

Depends on the use-case. If precompile can infer all types on the execution path, then it will work great. If it can't, it will waste time on trying to guess types, give up and then the types have to be inferred again during running time.

rikhuijzer · 2022-04-29T08:40:38Z

src/precompile.jl

+    precompile(Tuple{typeof(Base.bytesavailable), Sockets.TCPSocket})
+    precompile(Tuple{typeof(Base.eof), Sockets.TCPSocket})
+    precompile(Tuple{typeof(Base.alloc_buf_hook), Sockets.TCPSocket, UInt64})
+    precompile(Tuple{Base.var"#readcb_specialized#671", Sockets.TCPSocket, Int64, UInt64})


Suggested change

precompile(Tuple{Base.var"#readcb_specialized#671", Sockets.TCPSocket, Int64, UInt64})

This line causes

(HTTP) pkg> precompile Precompiling project... ✗ HTTP 0 dependencies successfully precompiled in 4 seconds. 4 already precompiled. ERROR: The following 1 direct dependency failed to precompile: HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3] Failed to precompile HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3] to /home/rik/.julia/compiled/v1.9/HTTP/jl_pwtQbZ. ERROR: LoadError: UndefVarError: #readcb_specialized#671 not defined Stacktrace: [1] _precompile() @ HTTP ~/git/HTTPDilum/src/precompile.jl:46 [2] top-level scope @ ~/git/HTTPDilum/src/HTTP.jl:627 [3] include @ ./Base.jl:427 [inlined] [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing) @ Base ./loading.jl:1423 [5] top-level scope @ stdin:1 in expression starting at /home/rik/git/HTTPDilum/src/HTTP.jl:1 in expression starting at stdin:1

on Julia 1.9.0-DEV.428.

rikhuijzer · 2022-04-29T08:45:05Z

@DilumAluthge It's best to do compilation time benchmarks with @eval: @time @eval HTTP.get("https://example.com");. From the docstring of @time:

In some cases the system will look inside the @time expression and compile some of the called code before execution of the top-level expression begins. When that happens, some compilation time will not be counted. To include this time you can run @time @eval ....

In this case, I do think that your benchmark is valid though. I could reproduce similar numbers on Julia 1.9.0-DEV.428:

master

$ julia --project -e 'import HTTP; @time @eval HTTP.get("https://example.com");'
  4.518353 seconds (5.63 M allocations: 368.014 MiB, 2.61% gc time, 91.25% compilation time)

This PR

$ julia --project -e 'import HTTP; @time @eval HTTP.get("https://example.com");'
  3.231371 seconds (147.83 k allocations: 8.562 MiB, 86.99% compilation time)

…atements

… statements

DilumAluthge force-pushed the dpa/precompile branch from f1b95f2 to 636a47e Compare March 10, 2022 01:08

DilumAluthge marked this pull request as ready for review March 10, 2022 01:09

DilumAluthge force-pushed the dpa/precompile branch from 636a47e to 73b1344 Compare March 10, 2022 01:18

fonsp reviewed Mar 25, 2022

View reviewed changes

rikhuijzer reviewed Apr 29, 2022

View reviewed changes

DilumAluthge added 2 commits June 21, 2024 14:33

Reduce the time-to-first-request by adding pregenerated precompile st…

d9b68be

…atements

Add a GitHub Actions workflow for regenerating the list of precompile…

a0df8f8

… statements

DilumAluthge force-pushed the dpa/precompile branch from 73b1344 to a0df8f8 Compare June 21, 2024 18:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce the time-to-first-request by adding pregenerated precompile statements #805

Reduce the time-to-first-request by adding pregenerated precompile statements #805

DilumAluthge commented Mar 10, 2022 •

edited

Loading

codecov-commenter commented Mar 10, 2022 •

edited

Loading

fonsp Mar 25, 2022

fonsp Mar 29, 2022

DilumAluthge Mar 29, 2022

fonsp commented Mar 25, 2022

rikhuijzer commented Apr 22, 2022

fredrikekre commented Apr 22, 2022

rikhuijzer commented Apr 29, 2022

rikhuijzer Apr 29, 2022

rikhuijzer commented Apr 29, 2022

Reduce the time-to-first-request by adding pregenerated precompile statements #805

Are you sure you want to change the base?

Reduce the time-to-first-request by adding pregenerated precompile statements #805

Conversation

DilumAluthge commented Mar 10, 2022 • edited Loading

Summary

Description

List of time measurements

How to regenerate the list of precompile statements

codecov-commenter commented Mar 10, 2022 • edited Loading

Codecov Report

fonsp Mar 25, 2022

Choose a reason for hiding this comment

fonsp Mar 29, 2022

Choose a reason for hiding this comment

DilumAluthge Mar 29, 2022

Choose a reason for hiding this comment

fonsp commented Mar 25, 2022

rikhuijzer commented Apr 22, 2022

fredrikekre commented Apr 22, 2022

rikhuijzer commented Apr 29, 2022

rikhuijzer Apr 29, 2022

Choose a reason for hiding this comment

rikhuijzer commented Apr 29, 2022

DilumAluthge commented Mar 10, 2022 •

edited

Loading

codecov-commenter commented Mar 10, 2022 •

edited

Loading