Backports release 1.3.1 #33979

KristofferC · 2019-11-29T13:07:44Z

Backported PRs:

Need manual backport:

fix #34080, regression in printing qualified macro calls #34081 - fix Behavior change in v1.3.0 regarding Symbol in a QuoteNode #34080, regression in printing qualified macro calls

Contains multiple commits, manual intervention needed:

Update Mozilla CA certificate store to latest (10-16-2019) for libgit2 SSL. #33617 - Update Mozilla CA certificate store to latest (10-16-2019) for libgit2 SSL.

Non-merged PRs with backport label:

Use UUID's to generate random tempname on Windows #33785 - Use UUID's to generate random tempname on Windows

maleadt · 2019-12-03T07:39:03Z

fredrikekre · 2019-12-03T07:44:34Z

@StefanKarpinski, @staticfloat: AsyPlots fails with a tree hash mismatch on this branch but not on 1.3.0 😕 :

┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   e80d1b7446d31c7738ed51fda764944d4bf1fcbe
│   Calculated git-tree-sha1: b37f6296f6e49bb2d602909f058cefbac9cdaa6e
└ @ Pkg.Artifacts /workspace/srcdir/julia/usr/share/julia/stdlib/v1.3/Pkg/src/Artifacts.jl:775
ERROR: Unable to automatically install 'FreeType2' from '/root/.julia/packages/FreeType2_jll/BxhKx

KristofferC · 2019-12-03T07:45:06Z

Most of them are hash errors.

When testing for directories to exclude from hashing, we must exclude not only empty directories, but also directories that themselves contain nothing but empty directories; in essence, we suppress adding directories that have no files contained within their entire subtree. While fixing this, it seemed prudent to eliminate the `names` argument to `tree_hash()`, especially as it was not actually used to iterate over. Fixes JuliaLang/julia#33979 (comment)

KristofferC · 2019-12-03T08:48:33Z

Revise: passes locally
Autologistics: passed locally
SMM: passed locally
FFmpegPipe passed locally:
JuDoc: the test version guard in tlienart/Franklin.jl@52f0ce2 is incorrect

When testing for directories to exclude from hashing, we must exclude not only empty directories, but also directories that themselves contain nothing but empty directories; in essence, we suppress adding directories that have no files contained within their entire subtree. While fixing this, it seemed prudent to eliminate the `names` argument to `tree_hash()`, especially as it was not actually used to iterate over. Fixes JuliaLang/julia#33979 (comment) (cherry picked from commit 648d66c, PR #1522)

StefanKarpinski · 2019-12-03T14:58:03Z

Looks like most failures were the tree hashing bug I introduced while fixing the other tree hashing bug. Better now?

maleadt · 2019-12-03T15:08:29Z

I'll rerun.

KristofferC · 2019-12-03T15:09:22Z

Wait, this branch isn't updated yet, no?

maleadt · 2019-12-03T15:10:23Z

Wait, this branch isn't updated yet, no?

Oh right, it's only a mention not yet a commit on this branch. Thanks!

KristofferC · 2019-12-03T15:28:40Z

@nanosoldier runbenchmarks(ALL, vs = ":release-1.3")

fredrikekre · 2019-12-03T15:41:54Z

Updated.

nanosoldier · 2019-12-03T22:51:00Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

KristofferC · 2019-12-03T23:09:13Z

Should check #34013 here.

daviehh · 2019-12-03T23:43:51Z

@KristofferC Thanks! Re: #34013, looks like backports-release-1.3 is affected too:

using LinearAlgebra
using BenchmarkTools

ndim = 3

m1 = rand(ComplexF64,ndim,ndim);
m2 = rand(ComplexF64,ndim,ndim);
ou = rand(ComplexF64,ndim,ndim);

@btime mul!($ou, $m1, $m2);

returns

386.193 ns (1 allocation: 16 bytes)

with

julia> versioninfo()
Julia Version 1.3.1-pre.14
Commit 1a6baefb51* (2019-12-03 15:40 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.0.0)
  CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

whereas release-1.3 version:

33.471 ns (0 allocations: 0 bytes)

Edit

Likely happened some time between mul!/gemm_wrapper! and matmul3x3!:

@btime LinearAlgebra.gemm_wrapper!($ou, $t, $t, $m1, $m2, 1.2, .5im)

388.228 ns (3 allocations: 80 bytes)

ad = LinearAlgebra.MulAddMul(1.2, 0.5im)
t = 'N'
@btime LinearAlgebra.matmul3x3!($ou, $t, $t, $m1, $m2, $ad)

48.857 ns (0 allocations: 0 bytes)

But there's not much between:

julia/stdlib/LinearAlgebra/src/matmul.jl

Lines 552 to 578 in 1a6baef

    
           function gemm_wrapper!(C::StridedVecOrMat{T}, tA::AbstractChar, tB::AbstractChar, 
        
                                  A::StridedVecOrMat{T}, B::StridedVecOrMat{T}, 
        
                                  α::Number=true, β::Number=false) where {T<:BlasFloat} 
        
               mA, nA = lapack_size(tA, A) 
        
               mB, nB = lapack_size(tB, B) 
        
               if nA != mB 
        
                   throw(DimensionMismatch("A has dimensions ($mA,$nA) but B has dimensions ($mB,$nB)")) 
        
               end 
        
               if C === A || B === C 
        
                   throw(ArgumentError("output matrix must not be aliased with input matrix")) 
        
               end 
        
               if mA == 0 || nA == 0 || nB == 0 || iszero(α) 
        
                   if size(C) != (mA, nB) 
        
                       throw(DimensionMismatch("C has dimensions $(size(C)), should have ($mA,$nB)")) 
        
                   end 
        
                   return _rmul_or_fill!(C, β) 
        
               end 
        
               if mA == 2 && nA == 2 && nB == 2 
        
                   return matmul2x2!(C, tA, tB, A, B, MulAddMul(α, β)) 
        
               end 
        
               if mA == 3 && nA == 3 && nB == 3 
        
                   return matmul3x3!(C, tA, tB, A, B, MulAddMul(α, β)) 
        
               end

more:

t = 'N'
al = 1.2
bt = .5im
@btime LinearAlgebra.matmul3x3!($ou, $t, $t, $m1, $m2, LinearAlgebra.MulAddMul($al, $bt))
@btime LinearAlgebra.matmul3x3!($ou, $t, $t, $m1, $m2, LinearAlgebra.MulAddMul(1.2, .5im))

returns:

359.817 ns (3 allocations: 80 bytes)
48.784 ns (0 allocations: 0 bytes)

fredrikekre · 2019-12-04T00:21:27Z

#33743 then maybe

(cherry picked from commit 9235f45)

This avoids cases where getMostGenericTBAA could return broken answers. fix #33590 (cherry picked from commit 1ae9902)

More cases similar to those identified in #23914 fix #33899 (cherry picked from commit e1086fe)

We were releasing this lock a bit too early, before we finished with using `buf`. (cherry picked from commit 60ec5f3)

This fixes problems with binaries on these two platforms that often use these large page sizes. (cherry picked from commit 5ceed92)

(cherry picked from commit ffb3ed0)

maleadt · 2019-12-06T07:00:25Z

w00t, the first successfull PkgEval@Nanosoldier run: maleadt@ac555be#commitcomment-36287280 (on ac555be):

Your test job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

Still in development and only triggered on my fork though.

EDIT, better formatted output: https://github.com/maleadt/BasePkgEvalReports/blob/master/ac555be_vs_46ce4d7/report.md

maleadt · 2019-12-08T08:30:40Z

@nanosoldier runtests(ALL, vs = "JuliaLang/julia#v1.3.0")

maleadt · 2019-12-08T20:31:47Z

Your test job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

KristofferC · 2019-12-11T08:12:13Z

@nanosoldier runtests(["ApproxFunOrthogonalPolynomials" ,"BlackBoxOptim" ,"ChaosTools" ,"GlobalSensitivityAnalysis" ,"Graphics" ,"HyperDualMatrixTools" ,"IdentityRanges", "ImageCore" ,"NeuralNetDiffEq" ,"OffsetArrays" ,"PowerModels" ,"Reactive" ,"TensorNetworkAD" ,"UncertainData"], vs = "JuliaLang/julia#v1.3.0")

maleadt · 2019-12-11T09:23:30Z

Your test job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

KristofferC · 2019-12-11T11:47:46Z

UncertainData.jl passed locally.

This should be good to go.

(cherry picked from commit 2ced6c4)

(cherry picked from commit bc82c35)

* Fix colon-reshaping of OffsetVector * reshape(::AbstractVector, ::Colon) is a no-op (cherry picked from commit f80c3ee)

* ccall: report static compile-time load issues correctly fix #34061 * add a test * Update ccall.jl (cherry picked from commit 8a47793)

…33926) (cherry picked from commit cbdb321)

This was introduced in: e2821f594c5e9b2ec0fa1603f4e90628d2e40cd7 Package managers track which package owns a file, and if the icon-theme.cache is owned by Julia, the package manager will get confused when that file already exists on the system or when another process overwrites it. It's the DE's job to update that cache. (cherry picked from commit 8bef999)

(cherry picked from commit e467661)

This handles cases like `Type{<:Tuple{A}} <: Type{Tuple{Vararg{_,N} where N}}`, i.e. where a fixed-length tuple type cannot ever equal an indefinite-length tuple type.

maleadt · 2019-12-17T18:55:38Z

@nanosoldier runtests(ALL, vs = "JuliaLang/julia#v1.3.0")

nanosoldier · 2019-12-18T11:43:01Z

Your test job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

KristofferC · 2019-12-18T11:52:21Z

Interval Intersection: Test Failed at /home/pkgeval/.julia/packages/IntervalTrees/wh2ex/test/runtests.jl:331
  Expression: !((IntervalTrees.firstintersection(t.root, Interval(4, 4), Interval(2, 2)))[1] === nothing)
   Evaluated: !(nothing === nothing)

I'm pretty sure I have seen this happened before, so don't think it is a regression with 1.3.1.

Other than that, anything that looked suspicious passed locally.

KristofferC · 2019-12-18T16:48:46Z

@nanosoldier runbenchmarks(ALL, vs = ":release-1.3")

nanosoldier · 2019-12-19T00:09:54Z

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

KristofferC added the DO NOT MERGE Do not merge this PR! label Nov 29, 2019

staticfloat mentioned this pull request Dec 3, 2019

Fix tree hashing with nested empty directories JuliaLang/Pkg.jl#1522

Merged

KristofferC force-pushed the backports-release-1.3 branch from 78c3633 to ee7098c Compare December 3, 2019 15:18

KristofferC closed this in JuliaLang/Pkg.jl@49ab53e Dec 3, 2019

KristofferC reopened this Dec 3, 2019

fredrikekre force-pushed the backports-release-1.3 branch from ee7098c to 1a6baef Compare December 3, 2019 15:41

KristofferC changed the title ~~WIP: Backports release 1.3~~ WIP: Backports release 1.3.1 Dec 4, 2019

staticfloat and others added 7 commits December 4, 2019 11:45

Use cache.julialang.org to download busybox.exe at test time

1ab02c6

(cherry picked from commit 9235f45)

codegen: avoid bad tbaa paths (#33706)

a706c27

This avoids cases where getMostGenericTBAA could return broken answers. fix #33590 (cherry picked from commit 1ae9902)

fix #33841, regression in let-bound function with kwargs (#33848)

51591e4

fix some more invalid unsafe code

efae1fb

More cases similar to those identified in #23914 fix #33899 (cherry picked from commit e1086fe)

fix race in writing a byte to a buffer

bbec89a

We were releasing this lock a bit too early, before we finished with using `buf`. (cherry picked from commit 60ec5f3)

Force patchelf to use 64KB page size on aarch64/powerpc64le

f0668b6

This fixes problems with binaries on these two platforms that often use these large page sizes. (cherry picked from commit 5ceed92)

Force PATCHELF override on fixup-libgfortran.sh as well

4a0f790

(cherry picked from commit ffb3ed0)

KristofferC mentioned this pull request Dec 10, 2019

Major slowdown in package load time JuliaGraphics/Gtk.jl#466

Closed

fredrikekre and others added 4 commits December 11, 2019 14:49

Bump Pkg version to 1.3.1.

3bb7ce4

fix #33987, some varargs not recognized in kwarg lowering (#33992)

dac742d

(cherry picked from commit 2ced6c4)

fix #33974, wrong integer types used in jl_array_sizehint (#34005)

955417f

(cherry picked from commit bc82c35)

Fix colon-reshaping of OffsetVector (#33890)

aee26cf

* Fix colon-reshaping of OffsetVector * reshape(::AbstractVector, ::Colon) is a no-op (cherry picked from commit f80c3ee)

fredrikekre force-pushed the backports-release-1.3 branch from 8639e57 to aee26cf Compare December 11, 2019 13:49

vtjnash and others added 5 commits December 17, 2019 13:45

ccall: report static compile-time load issues correctly (#34062)

7861501

* ccall: report static compile-time load issues correctly fix #34061 * add a test * Update ccall.jl (cherry picked from commit 8a47793)

BUGFIX avoid silent overflow in lcm least common multiple function (#…

cf96e13

…33926) (cherry picked from commit cbdb321)

tweak test for calling unknown libraries in --output-o mode (#34096)

be009f1

(cherry picked from commit e467661)

add another tuple subtyping fast path (#34065)

480d867

This handles cases like `Type{<:Tuple{A}} <: Type{Tuple{Vararg{_,N} where N}}`, i.e. where a fixed-length tuple type cannot ever equal an indefinite-length tuple type.

maleadt mentioned this pull request Dec 18, 2019

check sizes of arguments in dot; fixes #28617 #28666

Open

Bump Pkg again to include artifact load time optimizations.

5e06534

ararslan added release Release management and versioning. and removed DO NOT MERGE Do not merge this PR! labels Dec 19, 2019

ararslan added this to the 1.3 milestone Dec 19, 2019

ararslan changed the title ~~WIP: Backports release 1.3.1~~ Backports release 1.3.1 Dec 19, 2019

ararslan merged commit 90620af into release-1.3 Dec 19, 2019

StefanKarpinski deleted the backports-release-1.3 branch December 19, 2019 19:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backports release 1.3.1 #33979

Backports release 1.3.1 #33979

KristofferC commented Nov 29, 2019 •

edited

Loading

maleadt commented Dec 3, 2019 •

edited by KristofferC

Loading

fredrikekre commented Dec 3, 2019

KristofferC commented Dec 3, 2019

KristofferC commented Dec 3, 2019

StefanKarpinski commented Dec 3, 2019

maleadt commented Dec 3, 2019

KristofferC commented Dec 3, 2019

maleadt commented Dec 3, 2019

KristofferC commented Dec 3, 2019

fredrikekre commented Dec 3, 2019

nanosoldier commented Dec 3, 2019

KristofferC commented Dec 3, 2019

daviehh commented Dec 3, 2019 •

edited

Loading

fredrikekre commented Dec 4, 2019

maleadt commented Dec 6, 2019 •

edited

Loading

maleadt commented Dec 8, 2019

maleadt commented Dec 8, 2019

KristofferC commented Dec 11, 2019

maleadt commented Dec 11, 2019

KristofferC commented Dec 11, 2019

maleadt commented Dec 17, 2019

nanosoldier commented Dec 18, 2019

KristofferC commented Dec 18, 2019 •

edited

Loading

KristofferC commented Dec 18, 2019

nanosoldier commented Dec 19, 2019

Backports release 1.3.1 #33979

Backports release 1.3.1 #33979

Conversation

KristofferC commented Nov 29, 2019 • edited Loading

maleadt commented Dec 3, 2019 • edited by KristofferC Loading

fredrikekre commented Dec 3, 2019

KristofferC commented Dec 3, 2019

KristofferC commented Dec 3, 2019

StefanKarpinski commented Dec 3, 2019

maleadt commented Dec 3, 2019

KristofferC commented Dec 3, 2019

maleadt commented Dec 3, 2019

KristofferC commented Dec 3, 2019

fredrikekre commented Dec 3, 2019

nanosoldier commented Dec 3, 2019

KristofferC commented Dec 3, 2019

daviehh commented Dec 3, 2019 • edited Loading

fredrikekre commented Dec 4, 2019

maleadt commented Dec 6, 2019 • edited Loading

maleadt commented Dec 8, 2019

maleadt commented Dec 8, 2019

KristofferC commented Dec 11, 2019

maleadt commented Dec 11, 2019

KristofferC commented Dec 11, 2019

maleadt commented Dec 17, 2019

nanosoldier commented Dec 18, 2019

KristofferC commented Dec 18, 2019 • edited Loading

KristofferC commented Dec 18, 2019

nanosoldier commented Dec 19, 2019

KristofferC commented Nov 29, 2019 •

edited

Loading

maleadt commented Dec 3, 2019 •

edited by KristofferC

Loading

daviehh commented Dec 3, 2019 •

edited

Loading

maleadt commented Dec 6, 2019 •

edited

Loading

KristofferC commented Dec 18, 2019 •

edited

Loading