Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for 1.9.0-rc1 #48733

Merged
merged 53 commits into from
Mar 6, 2023
Merged

Backports for 1.9.0-rc1 #48733

merged 53 commits into from
Mar 6, 2023

Commits on Feb 20, 2023

  1. Restrict path to be an AbstractString in lbt_forward()

    Some users tried to pass a `dlopen()` handle into `lbt_forward()` which sadly works just fine, despite `ccall()` declaring its first argument as being a `Cstring`.  I guess it's trivial to convert a `Ptr{Cvoid}` into a `Cstring`, so this just goes through.
    
    To protect against this, restrict `path` to be an `AbstractString`.
    
    (cherry picked from commit 8492031)
    staticfloat authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    1360b3a View commit details
    Browse the repository at this point in the history
  2. Update LLVM build to 14.0.6+2 (#48544)

    (cherry picked from commit 43c6f75)
    vchuravy authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    d0a7175 View commit details
    Browse the repository at this point in the history
  3. Update checksums for llvm 14.0.6+2 (#48659)

    (cherry picked from commit ecfaef3)
    vchuravy authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    ceacba1 View commit details
    Browse the repository at this point in the history
  4. [REPL] Meta-e: Don't execute the edited code when exiting editor (#46153

    )
    
    (cherry picked from commit aaab409)
    fredrikekre authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    6213bb8 View commit details
    Browse the repository at this point in the history
  5. Disable frame-pointer-optimiation on Linux (#48660)

    (cherry picked from commit 9c4a40c)
    vchuravy authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    d43d055 View commit details
    Browse the repository at this point in the history
  6. Fix Base.libllvm_path and jl_get_libllvm don't support non-ASCII char…

    …acters in path on Windows (#45126) (#45127)
    
    * Fix jl_get_libllvm_impl to support non-ASCII characters
    
    * Fix jl_get_libllvm_impl to support non-ASCII characters, fix whitespace
    
    * Fix jl_get_libllvm_impl to support non-ASCII characters, fix null and buffer
    
    (cherry picked from commit 6976bac)
    wesjenkins authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    b3d42d1 View commit details
    Browse the repository at this point in the history
  7. fix #47658, state stack overflow on unions containing typevars (#48221)

    (cherry picked from commit 596ce65)
    JeffBezanson authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    25bad18 View commit details
    Browse the repository at this point in the history
  8. restore kwcall_mt optimizations

    Jameson Nash authored and KristofferC committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    f7c4f59 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. lattice: fix minor lattice issues (#47570)

    I found some lattice issues when implementing `MustAlias` under the new
    extendable lattice system in another PR.
    
    (cherry picked from commit ee0f3fc)
    aviatesk authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    5e39fc6 View commit details
    Browse the repository at this point in the history
  2. Put back getfield :boundscheck hack (#48677)

    We used to have this hack before #48246, but I removed it because I had
    hoped we don't need. Unfortunately, we weren't able to infer consistency of
    ```
    @inbounds (1,2)[1]
    ```
    With this hack, constprop is able to independently prove inbounded-ness,
    overriding the usual consistency taintaing that happens for inbounds.
    
    (cherry picked from commit 113c2f3)
    Keno authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    84e3545 View commit details
    Browse the repository at this point in the history
  3. make insert_extension_triggers idempotent

    (cherry picked from commit 2a0b9dd)
    KristofferC authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    2324310 View commit details
    Browse the repository at this point in the history
  4. fix printing of parents to extensions in @time_imports

    (cherry picked from commit 6bd658e)
    KristofferC authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    7734b34 View commit details
    Browse the repository at this point in the history
  5. julia_cmd() docstring: include pkgimages (#48392)

    (cherry picked from commit 4a75129)
    fonsp authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    6545919 View commit details
    Browse the repository at this point in the history
  6. Allow for use 60% of constrained/total system memory (#48614)

    Remove the high watermark logic, because it doesn't really make sense,
    and allow for use of 60% of system memory before aggressive GC kicks in.
    
    Should fix #48473
    
    (cherry picked from commit 500f561)
    gbaraldi authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    1121d25 View commit details
    Browse the repository at this point in the history
  7. update loading test to best-practices (#48429)

    Many of these test were hiding output or using semi-deprecated functions
    or failing to cleanup output or setting the wrong variables.
    
    (cherry picked from commit b48104d)
    vtjnash authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    a66d092 View commit details
    Browse the repository at this point in the history
  8. only load extension triggers from the evnironment where the parent pa…

    …ckage is loaded (#48703)
    
    (cherry picked from commit 200c962)
    KristofferC authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    5453936 View commit details
    Browse the repository at this point in the history
  9. effects: taint :nothrow effect on unknown :static_parameter (#46791)

    * effects: taint `:nothrow` effect on unknown `:static_parameter` (conservatively)
    
    With this commit, we taint `:nothrow` effect property correctly on
    access to unknown `:static_parameter`, e.g.:
    ```julia
    unknown_sparam_throw(::Union{Nothing, Type{T}}) where T = (T; nothing)
    @test Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_throw, ((Type{Int},))))
    @test !Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_throw, ((Nothing,))))
    ```
    
    This commit implements a very conservative analysis, and thus there is a
    room for improvement still, e.g.:
    ```julia
    unknown_sparam_nothrow(x::Ref{T}) where {T} = (T; nothing)
    @test_broken Core.Compiler.is_nothrow(Base.infer_effects(unknown_sparam_nothrow, (Ref,)))
    ```
    
    * inference: improve `:nothrow` modeling for `:static_parameter` (#46820)
    
    * Fix test with free type params
    
    * Test: Ignore ::Type{T} in detect_unbounded
    
    These are only technically unbounded because of the existence of
    ill-formed types. However, this function is supposed to be an API
    sanity check and ordinary users should never have ill-formed types,
    so for the purpose we want here, allow unboundedness in Type{T}.
    
    ---------
    
    Co-authored-by: Keno Fischer <[email protected]>
    Co-authored-by: Elliot Saba <[email protected]>
    (cherry picked from commit b5d17ea)
    aviatesk authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    86b9df2 View commit details
    Browse the repository at this point in the history
  10. force singlethreading during precompilation (#48592)

    (cherry picked from commit c0d2c57)
    IanButterworth authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    af0e4c1 View commit details
    Browse the repository at this point in the history
  11. Bump LLD to get dsymutil and use it for pkgimgs (#48628)

    * Run dsymutil on pkgimgs for apple platforms +
    LLD bump to get dsymutil from the jll
    
    (cherry picked from commit d8c2250)
    gbaraldi authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    0cb0bb4 View commit details
    Browse the repository at this point in the history
  12. put back DelimitedFiles as a non-sysimage "stdlib" (#48671)

    (cherry picked from commit 56667bb)
    KristofferC authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    624c665 View commit details
    Browse the repository at this point in the history
  13. [OpenBLAS_jll] Upgrade to new build optimised for PowerPC (#48689)

    * [OpenBLAS_jll] Upgrade to new build optimised for PowerPC
    
    This only difference compared to previous build is that this one enables use of
    dynamic architecture detection also for the PowerPC architecture.
    
    ---------
    
    Co-authored-by: Valentin Churavy <[email protected]>
    (cherry picked from commit aacfcf0)
    giordano authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    956f54b View commit details
    Browse the repository at this point in the history
  14. staticdata: encode link_id in tagged linkage (#48673)

    On 64-bit, we have enough space to encode (1) the tag, (2) the
    `depmods` index, and (3) the offset all in a single 64-bit pointer
    field. This means we don't need the external `link_id` arrays,
    which reduces the size of many pkgimages by ~5%.
    
    On 32-bit, we don't have enough bits to implement this strategy.
    However, most linkages seem to be against the sysimage, and so
    by giving that a separate tag we can achieve similar compression
    because the `link_id` lists will be much shorter.
    
    Co-authored-by: Tim Holy <[email protected]>
    
    (cherry picked from commit 8e3e970)
    vtjnash authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    b5d9781 View commit details
    Browse the repository at this point in the history
  15. Add world argument to jl_create_native. (#48746)

    (cherry picked from commit 0975906)
    maleadt authored and KristofferC committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    50b8c13 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2023

  1. [compiler] Teach type inference that GotoIfNot can throw (#48583)

    Previously, the effects system would ignore certain cases where
    `GotoIfNot` nodes would be capable of throwing; this resulted in simple
    examples such as the following being marked as `nothrow`:
    
    ```
    julia> foo(x) = x > 0 ? x : 0
           Base.infer_effects(foo, (Missing,))
    (+c,+e,+n,+t,+s,+m,+i)
    ```
    
    With this change, we correctly notice when a `GotoIfNot` node is given a
    non-`Bool` condition, annotate the basic block as possibly throwing, and
    further end type inference if the condition is provably non-`Bool`.
    staticfloat authored and aviatesk committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    1c22e77 View commit details
    Browse the repository at this point in the history
  2. Stop renumbering statements in inference proper (#48022)

    I don't think there's any good reason to try to delete the statements
    here. The very next thing we do is to convert to IRCode which drops
    dead code anyway, so this just seems redundant. In addition, it
    complicates Cthulhu-like analysis, which now has to track an extra
    set of statement numbers.
    Keno authored and aviatesk committed Feb 22, 2023
    Configuration menu
    Copy the full SHA
    d2de99d View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2023

  1. inference: fix the correctness of inference bail out interface (#48826)

    Since we allow overloading of the `bail_out_xxx` hooks, we need to make
    sure that we widen both type and effects to the top when bailing on
    inference regardless of the condition presumed by a hook.
    
    This commit particularly fixes the correctness of `bail_out_apply`
    (fixes #48807). I wanted to make a simplified test case for this, but
    it turns out to be a bit tricky since it relies on the details of
    multiple match analysis and the bail out logic.
    aviatesk authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    a6a19e4 View commit details
    Browse the repository at this point in the history
  2. effects: taint nonoverlayed when bailing out inference (#48838)

    aviatesk authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    09757f7 View commit details
    Browse the repository at this point in the history
  3. effects: handle va-method properly when refining `:inaccessiblememonl…

    …y` (#48854)
    
    Previously the `:inaccessiblememonly` effect bit may be wrongly refined
    when analyzing va-method, e.g.:
    ```julia
    julia> callgetfield1(xs...) = getfield(getfield(xs, 1), 1)
    callgetfield1 (generic function with 1 method)
    
    julia> Base.infer_effects(callgetfield1, (Base.RefValue{Symbol},))
    (+c,+e,!n,+t,+s,+m,+i) # inaccessiblememonly is wrongly refined here
    ```
    
    This leads to wrong concrete evaluation of `callgetfield1` and thus may
    result in a problem like below:
    ```julia
    julia> const GLOBAL_XS = Ref(:julia);
    
    julia> global_getfield() = callgetfield1(GLOBAL_XS);
    
    julia> @test let
               Base.Experimental.@force_compile
               global_getfield()
           end === :julia
    Test Passed
    
    julia> GLOBAL_XS[] = :julia2;
    
    julia> @test let
               Base.Experimental.@force_compile
               global_getfield()
           end === :julia2 # this fails
    Test Failed at REPL[8]:1
      Expression: let
            #= REPL[8]:2 =# Base.Experimental.@force_compile
            global_getfield()
        end === :julia2
       Evaluated: julia === julia2
    ```
    
    This commit fixes it up.
    aviatesk authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    2761b06 View commit details
    Browse the repository at this point in the history
  4. Test: add space after test failure for visual clarity (#48526)

    (cherry picked from commit ba1e568)
    IanButterworth authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    e25fcc9 View commit details
    Browse the repository at this point in the history
  5. [Make.inc] Define new variable PATCHELF_SET_RPATH_ARG

    (cherry picked from commit 219ef21)
    giordano authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    54bdee8 View commit details
    Browse the repository at this point in the history
  6. Accept weakdeps as a source for UUIDS for Preferences

    (cherry picked from commit d6431a3)
    vchuravy authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    a671f0c View commit details
    Browse the repository at this point in the history
  7. add a type assert to logging to prevent Base.require invalidation (#4…

    …8810)
    
    (cherry picked from commit 8570951)
    KristofferC authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    b5243af View commit details
    Browse the repository at this point in the history
  8. Add libjulia-internal to ORCJIT symbol resolution (#48712)

    We need to make sure that libjulia-internal is loaded and placed first in the
    DynamicLibrary search order so that calls to runtime intrinsics are resolved
    to the correct library when multiple libjulia-*'s have been loaded (e.g. when
    we `ccall` into a PackageCompiler.jl-created shared library).
    
    (cherry picked from commit fd45943)
    topolarity authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    2497f83 View commit details
    Browse the repository at this point in the history
  9. Update MbedTLS to 2.28.2 (#48806)

    * Update MbedTLS to 2.28.2
    
    * Update deps/mbedtls.mk
    
    * Update mbedtls checksum
    
    (cherry picked from commit 127fb73)
    mkitti authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    8ba214d View commit details
    Browse the repository at this point in the history
  10. Always use -Wl,--warn-unresolved-symbols in MSAN builds (#48835)

    * Always use `-Wl,--warn-unresolved-symbols` in MSAN builds
    
    * Use `-Wl,--warn-unresolved-symbols` only on Linux and FreeBSD
    
    (cherry picked from commit a07f512)
    giordano authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    b710bd2 View commit details
    Browse the repository at this point in the history
  11. Calculate a checksum for the system image (#48869)

    (cherry picked from commit 48b4caa)
    timholy authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    3f040ca View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    df3f686 View commit details
    Browse the repository at this point in the history
  13. Atomically order specsigflags, specptr, and invoke (#47832)

    Co-authored-by: Jameson Nash <[email protected]>
    Co-authored-by: Dilum Aluthge <[email protected]>
    (cherry picked from commit 6412a56)
    pchintalapudi authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    44085f9 View commit details
    Browse the repository at this point in the history
  14. enable re-using external code in pkgimages (#48723)

    * enable using external code in pkgimages
    
    This was unintentionally disabled (and incomplete) in the original PR
    for pkgimages.
    
    Co-authored-by: Valentin Churavy <[email protected]>
    (cherry picked from commit c136b7e)
    vtjnash authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    570251e View commit details
    Browse the repository at this point in the history
  15. Fix final gc lowering on dynamically sized allocation (#48620)

    (cherry picked from commit 7e57dc7)
    wsmoses authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    4710009 View commit details
    Browse the repository at this point in the history
  16. gf: cache cache_with_orig decision (#48833)

    Memoizing this can save a lot of repeated effort for queries such as
    `@which eltype(String)`. Otherwise we repeatedly try to check if
    `eltype(::Type)` is a good way to cache that result, even though it
    never gets better the more we check it.
    
    (cherry picked from commit bdcd5e2)
    vtjnash authored and KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    b07fb40 View commit details
    Browse the repository at this point in the history
  17. bump Pkg to latest 1.9

    KristofferC committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    d57c4ea View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    ec5e702 View commit details
    Browse the repository at this point in the history
  19. staticdata: make hookup of code instances correct (#48751)

    Previously we would double-account for many of these, leading to
    occasional chaos. Try to avoid serializing code that does not belong to
    this incremental compilation session package.
    
    Refs: #48723
    (cherry picked from commit 81f366d)
    vtjnash authored and vchuravy committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    65a025f View commit details
    Browse the repository at this point in the history
  20. staticdata: make completeinfo memory-/gc-safe (#48832)

    There is actually almost no cases where `jl_alloc_svec_uninit` is safe,
    since if was safe, you would likely would prefer to use the `jl_svec`
    constructor instead.
    
    (cherry picked from commit 0608824)
    vtjnash authored and vchuravy committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    c1cb058 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. Configuration menu
    Copy the full SHA
    fcde569 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #48881 from JuliaLang/vc/backports-1.9

    [release-1.9] Backport #48751 and #48832
    KristofferC authored Mar 6, 2023
    Configuration menu
    Copy the full SHA
    be9df3f View commit details
    Browse the repository at this point in the history
  3. Add note to NEWS about keywords in tab completion (#48863)

    (cherry picked from commit 3850e88)
    pbouffard authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    8024dd6 View commit details
    Browse the repository at this point in the history
  4. staticdata: set method.nroots_sysimg in jl_write_values (#48875)

    This ensures that the value gets set after all possible compilation
    has occurred.
    
    (cherry picked from commit 0ec704e)
    timholy authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    fd1108b View commit details
    Browse the repository at this point in the history
  5. staticdata: check ci->relocatability before caching external speciali…

    …zations (#48882)
    
    Fixes #48837
    
    (cherry picked from commit 89f30ff)
    timholy authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    96b3fe3 View commit details
    Browse the repository at this point in the history
  6. Initialize JIT bytes

    (cherry picked from commit 39e34ee)
    pchintalapudi authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    9f3b747 View commit details
    Browse the repository at this point in the history
  7. Fix sorting missing values with offset indices (#48864)

    * Fix sorting missing values with offset indices
    
    * Add tests
    
    (cherry picked from commit 7eb9615)
    LilithHafner authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    e6249e7 View commit details
    Browse the repository at this point in the history
  8. Fix heapsize hint and use a line for max memory (#48747)

    * Fix heapsize hint and use a line
    so that large machines utilize more of their ram
    
    (cherry picked from commit 51db8af)
    gbaraldi authored and KristofferC committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    9ced070 View commit details
    Browse the repository at this point in the history