Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broadcast w/ exotic Union eltype causes julia to max out CPU #22688

Closed
quinnj opened this issue Jul 6, 2017 · 7 comments
Closed

Broadcast w/ exotic Union eltype causes julia to max out CPU #22688

quinnj opened this issue Jul 6, 2017 · 7 comments
Labels
types and dispatch Types, subtyping and method dispatch

Comments

@quinnj
Copy link
Member

quinnj commented Jul 6, 2017

Here's a minimum repro, Julia 0.6 (same behavior on master), OSX:

julia> U = Union{Int8, UInt8, Int16, UInt16, Int32, UInt32, Int64, UInt64, Int128, UInt128}
Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}

julia> N = 1
1

julia> len = ntuple(x->10, N)
(10,)

julia> mxsz = maximum(sizeof, Base.uniontypes(U))
16

julia> @time A = Array{U}(len)
  0.018541 seconds (34.43 k allocations: 1.104 MiB)
10-element Array{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8},1}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef

julia> A4 = copy(A)
10-element Array{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8},1}:
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef
 #undef

julia> @time all(A .== A4);
# CPU goes to 100% and this doesn't return (at least after 10 minutes or so)
@martinholters
Copy link
Member

Observations:

  • identity.(A) has the same problem
  • For len=0, identity.(A) works, while A .== A4 triggers this assertion.
  • Base.Broadcast._broadcast_eltype(==, A, A) works

@pabloferz
Copy link
Contributor

pabloferz commented Jul 6, 2017 •
edited
Loading

More observations (with A = U[1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

  • A .== A works on 0.6, but hangs on master
  • A4 = copy(A); A4 .== A4 also works on 0.6
  • A .== A4 segfaults on 0.6

@pabloferz
Copy link
Contributor

Here's the output of an interrupted call to A .== A on master

julia> A .== A
^C^C^C^C^C^C^C^C^C^C^C^C^C^CWARNING: Force throwing a SIGINT
Internal error: encountered unexpected error in runtime:
InterruptException()
statestack_get at /home/data/Public/GitHub/julia/src/subtype.c:116 [inlined]
pick_union_element at /home/data/Public/GitHub/julia/src/subtype.c:387 [inlined]
subtype at /home/data/Public/GitHub/julia/src/subtype.c:797
forall_exists_equal at /home/data/Public/GitHub/julia/src/subtype.c:950
subtype at /home/data/Public/GitHub/julia/src/subtype.c:929
var_gt at /home/data/Public/GitHub/julia/src/subtype.c:496
forall_exists_equal at /home/data/Public/GitHub/julia/src/subtype.c:950
subtype at /home/data/Public/GitHub/julia/src/subtype.c:929
subtype_unionall at /home/data/Public/GitHub/julia/src/subtype.c:570
subtype_unionall at /home/data/Public/GitHub/julia/src/subtype.c:570
subtype_unionall at /home/data/Public/GitHub/julia/src/subtype.c:599
subtype_unionall at /home/data/Public/GitHub/julia/src/subtype.c:599
exists_subtype at /home/data/Public/GitHub/julia/src/subtype.c:972 [inlined]
forall_exists_subtype at /home/data/Public/GitHub/julia/src/subtype.c:1000
jl_subtype_env at /home/data/Public/GitHub/julia/src/subtype.c:1054
subtype_tuple at /home/data/Public/GitHub/julia/src/subtype.c:762 [inlined]
subtype at /home/data/Public/GitHub/julia/src/subtype.c:909
exists_subtype at /home/data/Public/GitHub/julia/src/subtype.c:972 [inlined]
forall_exists_subtype at /home/data/Public/GitHub/julia/src/subtype.c:1000
jl_subtype_env at /home/data/Public/GitHub/julia/src/subtype.c:1054
jl_types_equal at /home/data/Public/GitHub/julia/src/subtype.c:1079
jl_typemap_lookup_by_type_ at /home/data/Public/GitHub/julia/src/typemap.c:679
jl_specializations_get_linfo at /home/data/Public/GitHub/julia/src/gf.c:148
code_for_method at ./inference.jl:2799
jlcall_code_for_method_773 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
typeinf_edge at ./inference.jl:2886
jlcall_typeinf_edge_750 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_method at ./inference.jl:1784
jlcall_abstract_call_method_727 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_gf_by_type at ./inference.jl:1669
jlcall_abstract_call_gf_by_type_660 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call at ./inference.jl:2248
jlcall_abstract_call_577 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_eval_call at ./inference.jl:2278
abstract_eval at ./inference.jl:2301
jlcall_abstract_eval_531 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
typeinf_work at ./inference.jl:3101
typeinf at ./inference.jl:3166
typeinf_edge at ./inference.jl:2913
jlcall_typeinf_edge_750 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_method at ./inference.jl:1784
jlcall_abstract_call_method_727 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_gf_by_type at ./inference.jl:1669
jlcall_abstract_call_gf_by_type_660 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call at ./inference.jl:2248
jlcall_abstract_call_577 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_eval_call at ./inference.jl:2278
abstract_eval at ./inference.jl:2301
#188 at ./<missing>:0
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
next at ./generator.jl:45
copy! at ./abstractarray.jl:571
_collect at ./array.jl:437
collect at ./array.jl:434
abstract_eval_call at ./inference.jl:2252
abstract_eval at ./inference.jl:2301
jlcall_abstract_eval_531 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
typeinf_work at ./inference.jl:3101
typeinf at ./inference.jl:3166
typeinf_edge at ./inference.jl:2913
jlcall_typeinf_edge_750 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_method at ./inference.jl:1784
jlcall_abstract_call_method_727 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call_gf_by_type at ./inference.jl:1669
jlcall_abstract_call_gf_by_type_660 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_call at ./inference.jl:2248
jlcall_abstract_call_577 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
abstract_eval_call at ./inference.jl:2278
abstract_eval at ./inference.jl:2301
jlcall_abstract_eval_531 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
typeinf_work at ./inference.jl:3101
typeinf at ./inference.jl:3166
typeinf_frame at ./inference.jl:2880
typeinf_code at ./inference.jl:2962
jlcall_typeinf_code_678 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
typeinf_ext at ./inference.jl:3001
jlcall_typeinf_ext_0 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
jl_apply at /home/data/Public/GitHub/julia/src/julia.h:1450 [inlined]
jl_apply_with_saved_exception_state at /home/data/Public/GitHub/julia/src/rtutils.c:249
jl_type_infer at /home/data/Public/GitHub/julia/src/gf.c:263
jl_compile_for_dispatch at /home/data/Public/GitHub/julia/src/gf.c:1663
jl_compile_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:337 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:384 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
collect at ./array.jl:516
broadcast_t at ./broadcast.jl:255
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
broadcast_c at ./broadcast.jl:318 [inlined]
broadcast at ./broadcast.jl:433
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
do_call at /home/data/Public/GitHub/julia/src/interpreter.c:66
eval at /home/data/Public/GitHub/julia/src/interpreter.c:246
eval_body at /home/data/Public/GitHub/julia/src/interpreter.c:548
jl_interpret_toplevel_thunk at /home/data/Public/GitHub/julia/src/interpreter.c:700
jl_toplevel_eval_flex at /home/data/Public/GitHub/julia/src/toplevel.c:597
jl_toplevel_eval_in at /home/data/Public/GitHub/julia/src/builtins.c:495
eval at ./boot.jl:236
jlcall_eval_18423 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
eval_user_input at ./repl/REPL.jl:67
jlcall_eval_user_input_20475 at /home/data/Public/GitHub/julia/usr/lib/julia/sys.so (unknown line)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
macro expansion at ./repl/REPL.jl:98 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x7fb68c575bff)
jl_call_fptr_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:369 [inlined]
jl_call_method_internal at /home/data/Public/GitHub/julia/src/julia_internal.h:388 [inlined]
jl_apply_generic at /home/data/Public/GitHub/julia/src/gf.c:1928
jl_apply at /home/data/Public/GitHub/julia/src/julia.h:1450 [inlined]
start_task at /home/data/Public/GitHub/julia/src/task.c:267
unknown function (ip: 0xffffffffffffffff)
10-element BitArray{1}:
 true
 true
 true
 true
 true
 true
 true
 true
 true
 true

@pabloferz pabloferz added the types and dispatch Types, subtyping and method dispatch label Jul 6, 2017
@quinnj quinnj mentioned this issue Aug 15, 2017
Merged
@ararslan ararslan mentioned this issue Aug 24, 2017
Merged
@quinnj quinnj mentioned this issue Sep 21, 2017
Closed
@Keno
Copy link
Member

Keno commented Sep 22, 2017

The original example now segfaults for me on master. Will take another look.

@Keno
Copy link
Member

Keno commented Sep 22, 2017 •
edited
Loading

(slight) reduction:

julia> X = Ref{Tuple{Array{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}, 1}, Array{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}, 1}}}

julia> Y = Ref{Tuple{Array{Union{Int8, UInt128, UInt16, UInt32, UInt64, UInt8, S}, 1}, Array{Union{Int8, UInt128, UInt16, UInt32, UInt64, UInt8, T}, 1}}} where S where T

julia> X <: Y

@Keno
Copy link
Member

Keno commented Sep 22, 2017

Simpler still. Doesn't crash, but does hang:

julia> X = Ref{Tuple{Array{Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}, 1}}}

julia> Y = Ref{Tuple{Array{Union{Int8, UInt128, UInt16, UInt32, UInt64, UInt8, S}}}} where S

julia> X <: Y

@Keno
Copy link
Member

Keno commented Sep 22, 2017

Looks similar to #21191

@JeffBezanson JeffBezanson mentioned this issue Sep 27, 2017
Closed
JeffBezanson added a commit that referenced this issue Oct 5, 2017
@JeffBezanson
add a fastpath to forall_exists_equal in subtyping
4580788 
fixes #22688, fixes #23908
@JeffBezanson JeffBezanson mentioned this issue Oct 5, 2017
Merged
JeffBezanson added a commit that referenced this issue Oct 5, 2017
@JeffBezanson
add a fastpath to forall_exists_equal in subtyping
7c9c60e 
fixes #22688, fixes #23908
JeffBezanson added a commit that referenced this issue Oct 10, 2017
@JeffBezanson
add a fastpath to forall_exists_equal in subtyping
d930724 
fixes #22688, fixes #23908
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
types and dispatch Types, subtyping and method dispatch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Keno @quinnj @pabloferz @martinholters