Simple segfault on 1.8, 1.9, and master #46879

jariji · 2022-09-24T05:41:05Z

julia> versioninfo()
Julia Version 1.8.1
Commit afb6c60d69a (2022-09-06 15:09 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 24 × AMD Ryzen 9 3900XT 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver2)
  Threads: 1 on 24 virtual cores
Environment:
  LD_LIBRARY_PATH = 

julia> xs = [:a,:b,:c]
3-element Vector{Symbol}:
 :a
 :b
 :c

julia> foldl(xs; init=[empty(xs)]) do acc,x
           map(a->push!(acc, a, [a..., x]), acc)
           @show acc
       end

signal (11): Segmentation fault
in expression starting at REPL[3]:1
jl_field_offset at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1133 [inlined]
set_nth_field at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/datatype.c:1486
jl_f_tuple at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/builtins.c:837 [inlined]
jl_f_tuple at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/builtins.c:825
jl_method_error at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:1895
jl_lookup_generic_ at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2530 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2545
do_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/builtins.c:703
#2 at ./REPL[3]:2 [inlined]
iterate at ./generator.jl:47 [inlined]
collect_to! at ./array.jl:845
collect_to_with_first! at ./array.jl:823 [inlined]
_collect at ./array.jl:817
collect_similar at ./array.jl:716 [inlined]
map at ./abstractarray.jl:2933 [inlined]
#1 at ./REPL[3]:2 [inlined]
BottomRF at ./reduce.jl:81 [inlined]
_foldl_impl at ./reduce.jl:58
foldl_impl at ./reduce.jl:48 [inlined]
mapfoldl_impl at ./reduce.jl:44
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
#mapfoldl#259 at ./reduce.jl:170
mapfoldl##kw at ./reduce.jl:170 [inlined]
#foldl#260 at ./reduce.jl:193 [inlined]
foldl##kw at ./reduce.jl:193
unknown function (ip: 0x7f33278f5618)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
do_call at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:126
eval_value at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:215
eval_stmt_value at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:166 [inlined]
eval_body at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:612
jl_interpret_toplevel_thunk at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:750
jl_toplevel_eval_flex at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:906
jl_toplevel_eval_flex at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:850
jl_toplevel_eval_flex at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
eval_user_input at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:151
repl_backend_loop at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:247
start_repl_backend at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:232
#run_repl#47 at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:369
run_repl at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:355
jfptr_run_repl_66557.clone_1 at /nix/store/ca4hhym3f57vpmhgvylvqp86cmz9gbis-julia-bin-1.8.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
#967 at ./client.jl:419
jfptr_YY.967_49700.clone_1 at /nix/store/ca4hhym3f57vpmhgvylvqp86cmz9gbis-julia-bin-1.8.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
jl_f__call_latest at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/builtins.c:774
#invokelatest#2 at ./essentials.jl:729 [inlined]
invokelatest at ./essentials.jl:726 [inlined]
run_main_repl at ./client.jl:404
exec_options at ./client.jl:318
_start at ./client.jl:522
jfptr__start_61720.clone_1 at /nix/store/ca4hhym3f57vpmhgvylvqp86cmz9gbis-julia-bin-1.8.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
true_main at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/jlapi.c:575
jl_repl_entrypoint at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/jlapi.c:719
main at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/cli/loader_exe.c:59
__libc_start_call_main at /nix/store/bzd91shky9j9d43girrrj6vmqlw7x9m8-glibc-2.35-163/lib/libc.so.6 (unknown line)
__libc_start_main at /nix/store/bzd91shky9j9d43girrrj6vmqlw7x9m8-glibc-2.35-163/lib/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Allocations: 8515591 (Pool: 8509763; Big: 5828); GC: 10
zsh: segmentation fault (core dumped)  result/bin/julia

Keno · 2022-09-24T09:24:44Z

acc is being modified while being mapped over which the system doesn't like. We should potentially remove the @inbounds though. With --check-bounds=yes:

julia> foldl(xs; init=[empty(xs)]) do acc,x
           map(a->push!(acc, a, [a..., x]), acc)
           @show acc
       end
ERROR: BoundsError: attempt to access 1-element Vector{Vector{Vector{Symbol}}} at index [2]
Stacktrace:
  [1] setindex!
    @ ./array.jl:957 [inlined]
  [2] collect_to!(dest::Vector{Vector{Vector{Symbol}}}, itr::Base.Generator{Vector{Vector{Symbol}}, var"#4#6"{Vector{Vector{Symbol}}, Symbol}}, offs::Int64, st::Int64)
    @ Base ./array.jl:844
  [3] collect_to_with_first!
    @ ./array.jl:818 [inlined]
  [4] _collect(c::Vector{Vector{Symbol}}, itr::Base.Generator{Vector{Vector{Symbol}}, var"#4#6"{Vector{Vector{Symbol}}, Symbol}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base ./array.jl:812
  [5] collect_similar
    @ ./array.jl:711 [inlined]

vtjnash · 2022-09-24T09:25:25Z

ah, the classic "mutate a vector while you are mutating it" trick

$ ./julia --check-bounds=yes -q
julia> foldl(xs; init=[empty(xs)]) do acc,x
           map(a->push!(acc, a, [a..., x]), acc)
           @show acc
       end
ERROR: BoundsError: attempt to access 1-element Vector{Vector{Vector{Symbol}}} at index [2]
Stacktrace:
  [1] setindex!
    @ ./array.jl:957 [inlined]
  [2] collect_to!(dest::Vector{Vector{Vector{Symbol}}}, itr::Base.Generator{Vector{Vector{Symbol}}, var"#8#10"{Vector{Vector{Symbol}}, Symbol}}, offs::Int64, st::Int64)
    @ Base ./array.jl:844
  [3] collect_to_with_first!
    @ ./array.jl:818 [inlined]
  [4] _collect(c::Vector{Vector{Symbol}}, itr::Base.Generator{Vector{Vector{Symbol}}, var"#8#10"{Vector{Vector{Symbol}}, Symbol}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base ./array.jl:812
  [5] collect_similar
    @ ./array.jl:711 [inlined]
  [6] map
    @ ./abstractarray.jl:3219 [inlined]
  [7] #7
    @ ./REPL[3]:2 [inlined]
  [8] BottomRF
    @ ./reduce.jl:81 [inlined]
  [9] _foldl_impl(op::Base.BottomRF{var"#7#9"}, init::Vector{Vector{Symbol}}, itr::Vector{Symbol})
    @ Base ./reduce.jl:58
 [10] foldl_impl
    @ ./reduce.jl:48 [inlined]
 [11] mapfoldl_impl(f::typeof(identity), op::var"#7#9", nt::Vector{Vector{Symbol}}, itr::Vector{Symbol})
    @ Base ./reduce.jl:44
 [12] mapfoldl(f::Function, op::Function, itr::Vector{Symbol}; init::Vector{Vector{Symbol}})
    @ Base ./reduce.jl:170
 [13] #foldl#289
    @ ./reduce.jl:193 [inlined]
 [14] top-level scope
    @ REPL[3]:1

Remove some unnecessary and incorrect inbounds annotations. Fix #46879

lxvm · 2024-01-17T21:41:10Z

Hi, I observed a related issue by mapping over a vector while appending to it and wanted to share some observations I made. The crash is GC-related so I can also open a separate issue if appropriate.

System information

julia> versioninfo()
Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
  Threads: 1 on 12 virtual cores
Environment:
  LD_PRELOAD = /usr/lib/x86_64-linux-gnu/libstdc++.so.6

Now these tests are on the command line since they may hang. First, a simple test that doesn't hang already causes me some confusion. Say I want to map over a vector and make it longer for a finite number of iterations. I would expect that the result has the same length as the original vector before mapping since the output array size is determined before the mapping for iterators with length or shape.

julia/base/array.jl

Line 704 in 3ed49fd

_collect(cont, itr, ::HasEltype, isz::Union{HasLength,HasShape}) =

On the other hand, if we go along with the self-mutating idea, that would only work as expected if the vector had SizeUnknown

julia/base/array.jl

Line 707 in 3ed49fd

function _collect(cont, itr, ::HasEltype, isz::SizeUnknown)

Now the following test shows that both seem to be happening at the same time

$ julia -e 'a = [1]; println(map(b -> (b < 5 && push!(a, b+1); nothing), a)); @show a' > log.txt 2>&1
$ cat log.txt 
[nothing]
a = [1, 2, 3, 4, 5]

And the issue is worse if the function has an ambiguous return type, causing the program to hang

$ julia -e 'a = [1]; map(b -> (b < 5 && push!(a, b+1)), a)' > log_mwe.txt 2>&1

After stopping the shell I get this log file, which is too big to attach so below I share the beginning

log_mwe.txt

GC error (probable corruption)
Allocations: 587100 (Pool: 586101; Big: 999); GC: 0
Array{Int64, (5,)}[1, 2, 3, 4, 5]

thread 0 ptr queue:
~~~~~~~~~~ ptr queue top ~~~~~~~~~~
Task(next=nothing, queue=nothing, storage=Base.IdDict{Any, Any}(ht=Array{Any, (32,)}[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<nu
ll>, #<null>, #<null>, #<null>, #<null>, #<null>, nothing, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>], count=0, ndel=8), do
nenotify=nothing, result=nothing, logstate=nothing, code=#<null>, rngState0=0xceabf3857cbc9303, rngState1=0x0c517e6b70c5ff6d, rngState2=0x5749032fcd849773, rngState3=0x1f17612c22a2fffc, rngState4=0
x602576c6bc7298e7, _state=0x00, sticky=true, _isexception=false, priority=0x0000)
==========
Main
==========
Array{Any, (0,)}[]
==========
Type{T} where T<:Tuple
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.getproperty), LineNumberNode, Symbol}, simplesig=nothing, guardsigs=svec(), min_world=0x00000000000015b5, max_world=0xffffffffffffffff, func=getproperty(Line
NumberNode, Symbol) from getproperty(Any, Symbol), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Core.Compiler._topmod), Module}, simplesig=nothing, guardsigs=svec(), min_world=0x0000000000000f3e, max_world=0xffffffffffffffff, func=_topmod(Module) from _topmo
d(Module), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.indexed_iterate), Tuple{Symbol, Array{Any, 1}, Array{Any, 1}}, Int64, Int64}, simplesig=nothing, guardsigs=svec(), min_world=0x00000000000017e1, max_world=0x
ffffffffffffffff, func=indexed_iterate(Tuple{Symbol, Array{Any, 1}, Array{Any, 1}}, Int64, Int64) from indexed_iterate(Tuple, Int64, Any), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.isempty), Array{Any, 1}}, simplesig=nothing, guardsigs=svec(), min_world=0x0000000000001ea2, max_world=0xffffffffffffffff, func=isempty(Array{Any, 1}) from i
sempty(AbstractArray{T, N} where N where T), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.append!), Array{Any, 1}, Array{Any, 1}}, simplesig=nothing, guardsigs=svec(), min_world=0x0000000000001d9d, max_world=0xffffffffffffffff, func=append!(Array{
Any, 1}, Array{Any, 1}) from append!(Array{T, 1} where T, AbstractArray{T, 1} where T), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.:(*)), String, String}, simplesig=nothing, guardsigs=svec(), min_world=0x0000000000002f7d, max_world=0xffffffffffffffff, func=*(String, String) from *(Union{
AbstractChar, AbstractString}, Union{AbstractChar, AbstractString}...), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.getproperty), Expr, Symbol}, simplesig=nothing, guardsigs=svec(), min_world=0x00000000000015b5, max_world=0xffffffffffffffff, func=getproperty(Expr, Symbol) from getproperty(Any, Symbol), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Base.push!), Array{Int64, 1}, Int64}, simplesig=nothing, guardsigs=svec(), min_world=0x0000000000001d9c, max_world=0xffffffffffffffff, func=push!(Array{Int64, 1}, Int64) from push!(Array{T, 1}, Any) where {T}, isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========
Core.TypeMapEntry(sig=Tuple{typeof(Core.Compiler.:(-)), Int64, Bool}, simplesig=nothing, guardsigs=svec(), min_world=0x00000000000006a6, max_world=0xffffffffffffffff, func=-(Int64, Bool) from -(Integer, Integer), isleafsig=true, issimplesig=true, va=false, next=↩︎
  nothing)
==========

This seems quite strange so I thought I should bring it up

jariji changed the title ~~foldl segfault~~ Simple segfault on 1.8.1 Sep 24, 2022

vtjnash added a commit that referenced this issue Sep 26, 2022

audit inbounds for arrays

7510def

Remove some unnecessary and incorrect inbounds annotations. Fix #46879

vtjnash linked a pull request Sep 26, 2022 that will close this issue

audit inbounds for arrays #46911

Draft

LilithHafner changed the title ~~Simple segfault on 1.8.1~~ Simple segfault on 1.8, 1.9, and master Jan 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simple segfault on 1.8, 1.9, and master #46879

Simple segfault on 1.8, 1.9, and master #46879

jariji commented Sep 24, 2022

Keno commented Sep 24, 2022

vtjnash commented Sep 24, 2022

lxvm commented Jan 17, 2024

Simple segfault on 1.8, 1.9, and master #46879

Simple segfault on 1.8, 1.9, and master #46879

Comments

jariji commented Sep 24, 2022

Keno commented Sep 24, 2022

vtjnash commented Sep 24, 2022

lxvm commented Jan 17, 2024