Crashes when running test project

[qarmin]

Goost and Godot version:
fb4fe2b
Built with address sanitizer - use_asan =yes

OS/device including version:
Ubuntu 20.04

Issue description:
My test project found some crashes and invalid memory usage in Goost

Some of crashes:
res://AutomaticBugs/FunctionExecutor.tscn

Grid2D._iter_get
Parameters [False]
ERROR: get: FATAL: Index p_index = 0 is out of bounds (size() = 0).
   At: ./core/cowdata.h:156.
handle_crash: Program crashed with signal 4
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] ./godot.x11.tools.64.goosts() [0x17d2310] (/home/rafal/Downloads/goost-gd3/godot/platform/x11/crash_handler_x11.cpp:54)
[2] /lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7f55af45b210] (??:0)
[3] CowData<Variant>::get(int) const (/home/rafal/Downloads/goost-gd3/godot/./core/cowdata.h:156 (discriminator 7))
[4] Vector<Variant>::operator[](int) const (/home/rafal/Downloads/goost-gd3/godot/./core/vector.h:85)
[5] Grid2D::_iter_get(Variant const&) (/home/rafal/Downloads/goost-gd3/goost/core/types/grid_2d.cpp:102)
[6] MethodBind1R<Variant, Variant const&>::call(Object*, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/./core/method_bind.gen.inc:961 (discriminator 8))
[7] Object::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:919 (discriminator 1))
[8] Object::callv(StringName const&, Array const&) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:826 (discriminator 1))
[9] MethodBind2R<Variant, StringName const&, Array const&>::call(Object*, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/./core/method_bind.gen.inc:1717 (discriminator 12))
[10] Object::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:919 (discriminator 1))
[11] Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/variant_call.cpp:1149 (discriminator 1))
[12] GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript_function.cpp:1089)
[13] GDScriptInstance::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript.cpp:1208)
[14] Object::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:898 (discriminator 1))
[15] Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/variant_call.cpp:1149 (discriminator 1))
[16] GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript_function.cpp:1089)
[17] GDScriptInstance::_ml_call_reversed(GDScript*, StringName const&, Variant const**, int) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript.cpp:1239)
[18] GDScriptInstance::call_multilevel_reversed(StringName const&, Variant const**, int) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript.cpp:1248)
[19] Node::_notification(int) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:152)
[20] Node::_notificationv(int, bool) (/home/rafal/Downloads/goost-gd3/godot/./scene/main/node.h:46 (discriminator 14))
[21] Object::notification(int, bool) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:931)
[22] Node::_propagate_ready() (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:197)
[23] Node::_set_tree(SceneTree*) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:2626)
[24] Node::_add_child_nocheck(Node*, StringName const&) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:1154)
[25] Node::add_child(Node*, bool) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:1169)
[26] MethodBind2<Node*, bool>::call(Object*, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/./core/method_bind.gen.inc:1523 (discriminator 19))
[27] Object::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:919 (discriminator 1))
[28] Variant::call_ptr(StringName const&, Variant const**, int, Variant*, Variant::CallError&) (/home/rafal/Downloads/goost-gd3/godot/core/variant_call.cpp:1149 (discriminator 1))
[29] GDScriptFunction::call(GDScriptInstance*, Variant const**, int, Variant::CallError&, GDScriptFunction::CallState*) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript_function.cpp:1089)
[30] GDScriptInstance::_ml_call_reversed(GDScript*, StringName const&, Variant const**, int) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript.cpp:1239)
[31] GDScriptInstance::call_multilevel_reversed(StringName const&, Variant const**, int) (/home/rafal/Downloads/goost-gd3/godot/modules/gdscript/gdscript.cpp:1248)
[32] Node::_notification(int) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:152)
[33] Node::_notificationv(int, bool) (/home/rafal/Downloads/goost-gd3/godot/./scene/main/node.h:46 (discriminator 14))
[34] CanvasItem::_notificationv(int, bool) (/home/rafal/Downloads/goost-gd3/godot/./scene/2d/canvas_item.h:166 (discriminator 3))
[35] Control::_notificationv(int, bool) (/home/rafal/Downloads/goost-gd3/godot/./scene/gui/control.h:48 (discriminator 3))
[36] Object::notification(int, bool) (/home/rafal/Downloads/goost-gd3/godot/core/object.cpp:931)
[37] Node::_propagate_ready() (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:197)
[38] Node::_propagate_ready() (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:186 (discriminator 2))
[39] Node::_set_tree(SceneTree*) (/home/rafal/Downloads/goost-gd3/godot/scene/main/node.cpp:2626)
[40] SceneTree::init() (/home/rafal/Downloads/goost-gd3/godot/scene/main/scene_tree.cpp:466)
[41] OS_X11::run() (/home/rafal/Downloads/goost-gd3/godot/platform/x11/os_x11.cpp:3635)
[42] ./godot.x11.tools.64.goosts(main+0x326) [0x17c8f9c] (/home/rafal/Downloads/goost-gd3/godot/platform/x11/godot_x11.cpp:57)
[43] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f55af43c0b3] (??:0)
[44] ./godot.x11.tools.64.goosts(_start+0x2e) [0x17c8bbe] (??:?)

res://Nodes/Nodes.tscn

==67428==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x60400068b190 in thread T0:
  object passed to delete has wrong type:
  size of the allocated type:   48 bytes;
  size of the deallocated type: 32 bytes.
    #0 0x7f6c2a22df7f in operator delete(void*, unsigned long) (/lib/x86_64-linux-gnu/libasan.so.6+0xb2f7f)
    #1 0x7d54769 in clipperlib::DisposeOutPt(clipperlib::OutPt*) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:66
    #2 0x7d4de51 in clipperlib::ClipperTri::Triangulate(clipperlib::OutRec*) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:184
    #3 0x7d4ec45 in clipperlib::ClipperTri::AddOutPt(clipperlib::Active&, clipperlib::Point64) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:196
    #4 0x7cf4932 in clipperlib::Clipper::DoTopOfScanbeam(long) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper.cpp:1781
    #5 0x7ce8ed2 in clipperlib::Clipper::ExecuteInternal(clipperlib::ClipType, clipperlib::FillRule) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper.cpp:1337
    #6 0x7d52c45 in clipperlib::ClipperTri::Execute(clipperlib::ClipType, std::vector<std::vector<clipperlib::Point64, std::allocator<clipperlib::Point64> >, std::allocator<std::vector<clipperlib::Point64, std::allocator<clipperlib::Point64> > > >&, clipperlib::FillRule) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:303
    #7 0x7c4ac38 in PolyDecomp2DClipper10::triangulate_mono(Vector<Vector<Vector2> > const&) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/decomp/clipper10/poly_decomp_clipper10.cpp:14
    #8 0x7c2b56e in PolyDecomp2D::triangulate_polygons(Vector<Vector<Vector2> > const&, Ref<PolyDecompParameters2D> const&) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/decomp/poly_decomp.cpp:39
    #9 0x61801d7 in PolyNode2D::_draw() /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.cpp:42
    #10 0x61825bd in PolyNode2D::_notification(int) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.cpp:91
    #11 0x619e925 in PolyNode2D::_notificationv(int, bool) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.h:7
    #12 0x626a0c8 in PolyCircle2D::_notificationv(int, bool) /home/rafal/Downloads/goost-gd3/goost/scene/2d/poly_generators_2d.h:7
    #13 0x120d6fa1 in Object::notification(int, bool) core/object.cpp:929
    #14 0xe802afb in CanvasItem::_update_callback() scene/2d/canvas_item.cpp:443
    #15 0x1b9eb5f in MethodBind0::call(Object*, Variant const**, int, Variant::CallError&) core/method_bind.gen.inc:59
    #16 0x120d6b07 in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:919
    #17 0x120a66d2 in MessageQueue::_call_function(Object*, StringName const&, Variant const*, int, bool) core/message_queue.cpp:250
    #18 0x120a7574 in MessageQueue::flush() core/message_queue.cpp:297
    #19 0xcd50788 in SceneTree::idle(float) scene/main/scene_tree.cpp:542
    #20 0x1978f10 in Main::iteration() main/main.cpp:2123
    #21 0x185cc72 in OS_X11::run() platform/x11/os_x11.cpp:3641
    #22 0x17c8f9b in main platform/x11/godot_x11.cpp:56
    #23 0x7f6c290b20b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #24 0x17c8bbd in _start (/home/rafal/Downloads/goost-gd3/godot/bin/godot.x11.tools.64.goosts+0x17c8bbd)

0x60400068b190 is located 0 bytes inside of 48-byte region [0x60400068b190,0x60400068b1c0)
allocated by thread T0 here:
    #0 0x7f6c2a22cf17 in operator new(unsigned long) (/lib/x86_64-linux-gnu/libasan.so.6+0xb1f17)
    #1 0x7d4c94a in clipperlib::ClipperTri::CreateOutPt() /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:139
    #2 0x7ce00d3 in clipperlib::Clipper::AddLocalMinPoly(clipperlib::Active&, clipperlib::Active&, clipperlib::Point64) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper.cpp:987
    #3 0x7d4f705 in clipperlib::ClipperTri::AddLocalMinPoly(clipperlib::Active&, clipperlib::Active&, clipperlib::Point64) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:208
    #4 0x7cddfee in clipperlib::Clipper::InsertLocalMinimaIntoAEL(long) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper.cpp:895
    #5 0x7ce8865 in clipperlib::Clipper::ExecuteInternal(clipperlib::ClipType, clipperlib::FillRule) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper.cpp:1332
    #6 0x7d52c45 in clipperlib::ClipperTri::Execute(clipperlib::ClipType, std::vector<std::vector<clipperlib::Point64, std::allocator<clipperlib::Point64> >, std::allocator<std::vector<clipperlib::Point64, std::allocator<clipperlib::Point64> > > >&, clipperlib::FillRule) /home/rafal/Downloads/goost-gd3/goost/thirdparty/clipper/clipper_triangulation.cpp:303
    #7 0x7c4ac38 in PolyDecomp2DClipper10::triangulate_mono(Vector<Vector<Vector2> > const&) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/decomp/clipper10/poly_decomp_clipper10.cpp:14
    #8 0x7c2b56e in PolyDecomp2D::triangulate_polygons(Vector<Vector<Vector2> > const&, Ref<PolyDecompParameters2D> const&) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/decomp/poly_decomp.cpp:39
    #9 0x61801d7 in PolyNode2D::_draw() /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.cpp:42
    #10 0x61825bd in PolyNode2D::_notification(int) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.cpp:91
    #11 0x619e925 in PolyNode2D::_notificationv(int, bool) /home/rafal/Downloads/goost-gd3/goost/core/math/2d/geometry/poly/poly_node_2d.h:7
    #12 0x626a0c8 in PolyCircle2D::_notificationv(int, bool) /home/rafal/Downloads/goost-gd3/goost/scene/2d/poly_generators_2d.h:7
    #13 0x120d6fa1 in Object::notification(int, bool) core/object.cpp:929
    #14 0xe802afb in CanvasItem::_update_callback() scene/2d/canvas_item.cpp:443
    #15 0x1b9eb5f in MethodBind0::call(Object*, Variant const**, int, Variant::CallError&) core/method_bind.gen.inc:59
    #16 0x120d6b07 in Object::call(StringName const&, Variant const**, int, Variant::CallError&) core/object.cpp:919
    #17 0x120a66d2 in MessageQueue::_call_function(Object*, StringName const&, Variant const*, int, bool) core/message_queue.cpp:250
    #18 0x120a7574 in MessageQueue::flush() core/message_queue.cpp:297
    #19 0xcd50788 in SceneTree::idle(float) scene/main/scene_tree.cpp:542
    #20 0x1978f10 in Main::iteration() main/main.cpp:2123
    #21 0x185cc72 in OS_X11::run() platform/x11/os_x11.cpp:3641
    #22 0x17c8f9b in main platform/x11/godot_x11.cpp:56
    #23 0x7f6c290b20b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

SUMMARY: AddressSanitizer: new-delete-type-mismatch (/lib/x86_64-linux-gnu/libasan.so.6+0xb2f7f) in operator delete(void*, unsigned long)

Steps to reproduce:

1. Compile Goost with address sanitizer use_asan=yes
2. Run project
3. Run each scene alone to see which scene crash with which backrace

Minimal reproduction project:
https://github.com/qarmin/RegressionTestProject

[Xrayez]

Hopefully resolved all crashes with #62.

The ones coming from newest Clipper version are sort of known (still in development), it generally works ok apart from bugs like this. There was some recent updates upstream, maybe updating should fix the issue. EDIT: just checked, seems like triangulation unit is not ported to C++ yet, and the current version Goost uses is temporarily removed from the sandbox: https://sourceforge.net/p/polyclipping/code/HEAD/tree/sandbox/Clipper2/cpp/.

There's also a project setting which allows to switch to previous version of Clipper (stable). But using it forces to use Godot's triangulation code (since triangulation unit was only added in newest Clipper 10 version), which is less robust than Clipper's one and performance is sub-optimal, in my experience.

Do you know if there's a way to exclude thirdparty code from ASAN checks (so other issues can be fixed)?

[qarmin]

Looks that Address sanitizer allow to disable certains folders/functions, but I've never been able to do it.

Looks that this patch fixes this address sanitizer error:

diff --git a/thirdparty/clipper/clipper.h b/thirdparty/clipper/clipper.h
index 453b341..f7f1fe3 100644
--- a/thirdparty/clipper/clipper.h
+++ b/thirdparty/clipper/clipper.h
@@ -97,6 +97,8 @@ public:
   Point64      pt;
   OutPt       *next;
   OutPt       *prev;
+
+  virtual ~OutPt(){};
 };
 
 enum OutRecFlag { orInner, orOuter, orOpen};
@@ -111,6 +113,8 @@ struct OutRec {
   OutPt       *pts;
   PolyPath    *polypath;
   OutRecFlag  flag;
+
+  virtual ~OutRec(){};
 };
 
 //Active: an edge in the AEL that may or may not be 'hot' (part of the clip solution).

[Xrayez]

Thanks for reports, added your fix in #63! (co-authored).

If you'd like to integrate your fuzzing project in CI, feel free. 🙂

Goost runs GUT tests as well: https://github.com/goostengine/goost/blob/gd3/.github/workflows/linux_builds.yml#L121-L123, perhaps can be integrated after unit tests are run.