Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.x] Add optional allocation tracking #65189

Draft
wants to merge 1 commit into
base: 3.x
Choose a base branch
from

Conversation

lawnjelly
Copy link
Member

@lawnjelly lawnjelly commented Sep 1, 2022

Adds optional tracking to most memory allocations. It can be compiled in using the scons argument "use_allocation_tracking".

This is somewhat similar to the rids=tracked_handles build we have, but for general allocations rather than RIDs.

Example output:

=============================================
Final Report
8 allocation sources found.

1.62 KiB in 4 allocs in ./core/local_vector.h : 165
736 B in 20 allocs in ./core/local_vector.h : 65
136 B in 4 allocs in ./core/hash_map.h : 203
108 B in 3 allocs in ./core/pool_vector.h : 625
48 B in 2 allocs in core/array.cpp : 490
24 B in 1 allocs in drivers/unix/os_unix.cpp : 590
24 B in 1 allocs in modules/fbx/tools/validation_tools.cpp : 38
8 B in 1 allocs in drivers/unix/os_unix.cpp : 589

Total 2.68 KiB in tracked allocations.
Static memory usage : 1017.6 KiB
Static memory peak : 119.4 MiB
Dynamic memory usage : 0 B
=============================================

Usage

Call AllocationTracker::report("my report"); to get a report of all current allocations.

To compare allocations between two points in time, use snapshots.

Call int snap_id = AllocationTracker::take_snapshot(); to take snapshots.
Call AllocationTracker::compare_snapshots(snap_a, snap_b, "my comparison"); to get a diff report.

Notes

  • Is compiled out in regular builds, and only added when the scons argument is used. (It will slow down allocations a tad, and use extra memory).
  • We could actually consider compiling it in by default for DEV_ENABLED builds, and just have a bool whether it is active, and switch on / off with a command line switch. This would avoid having to recompile engine when wanting to do some memory tracking. 🤔
  • I had been meaning to add this for some time, every engine should have allocation tracking, useful for identifying memory leaks, runaway allocations etc. I've been using this to examine memory usage in the merging code.
  • The main functions are reporting all current allocations (sorted by size), and reporting differences in allocations between snapshots. Snapshots can be taken at any time when debugging sections of code.
  • Due to the chicken / egg problem, containers used inside are STL rather than Godot. This is to prevent recursive loops of the allocation tracking depending on a Godot allocation which depends on the allocation tracking etc.
  • In order to avoid construction / destruction issues (which are often present with allocation trackers) I'm using a primitive singleton on first use, and the memory used by the tracker is simply not deleted on shutdown and left to the OS. This should be fine for such a tool, as ensuring order of destruction on multiplatform is probably not worth the bother in this case.
  • Also offers functions for reading the number of allocation and size of allocations per frame and tick.
  • May not catch all allocations yet, I'll see if this can be improved. In particular I'll see if we can use some macros to pass on originating file / line numbers to containers such as LocalVector and PoolVector.
  • Adds a leak report at shutdown, although there are 8 or so small false flags currently.
  • Similar approach could be used in 4.x if it has no alloc tracking, but it might need some tweaking there as I expect memory is handled differently.

Adds optional tracking to most memory allocations. It can be compiled in using the scons argument "use_allocation_tracking".
@lawnjelly lawnjelly marked this pull request as ready for review September 1, 2022 13:09
@lawnjelly lawnjelly requested review from a team as code owners September 1, 2022 13:09
@lawnjelly lawnjelly marked this pull request as draft September 3, 2022 07:27
@akien-mga akien-mga changed the title Add optional allocation tracking [3.x] Add optional allocation tracking Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant