ObjectDB Profiling Tool #97210
Open
ObjectDB Profiling Tool #97210
+3,957
−81
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- made changes to how ObjectDB locks when debug_objects is called so we don't get inconsistant snapshots - made change to debug_objects so userdata can be included - Added readonly flag to EditorDebuggerRemoteObject - Made EditorDebuggerRemoteObject constructorable from a SceneDebuggerObject - prevent level editor from initializing if we're running in a debug game client (it's thinly initializes and full of nullptrs when we try to actually access it) - Added constructor for SceneDebuggerObject which bypasses objectid lookup, in the case we're using a remote object that won't be in the ObjectDB - Added new 'snapshot_debugger' module - module adds a new tab to debugger that can serialize the ObjectDB in a running game and send it to the editor to be viewed. Multiple timestamped snapshots can be captured - added several views of the snapshot to make it easier to spot memory irregularities - Added classes, objects, nodes, refcounted, raw, and summary views - objects, nodes, refcounted group by their respective type - classes groups by class - raw dumps the snapshot to json - summary highlight possible issues for the user
- renamed module to 'objectdb_profiler', as I think that most closely describes what it really does - renamed raw_view to json_view to be more descriptive - made summary_view work independently so it doesn't depend on other views to generate data for it. Summary view is not longer special relative to the other views - snapshots are now saved into the user folder in an objectdb_snapshots folder - only 10 snapshots are held in memory at a time. The rest are loaded from disk as needed (using an LRUCache) - when starting, on disk snapshots are loaded immediately - added a right click menu to see each snapshot in the file system or to delete the snapshot - snapshots are serialized to base64 via the Marshalls class, so the on disk view is kind of hard to read. JSON managed uint64 numbers, so reloading was hard that way. Marshalls seemed to just handle it
- made classes view work better when editor is small - added icons to classes view - fixed bug in classes view where lots of nodes didn't show up - added summed counts to classes view - added filter/sort/diff UI to classes view, but haven't wired it up yet. Planning to move diff to the far left column (because its universal across views) and make filter/sort buttons part of the left tree view
- propagate selected diff through to all views - diff dropdown box populates correctly - first item is automatically selected from snapshots list on load
…es view!) - added diff view to json (it's really simple, obviously!) - added shared controls file to make a few widgets I can use across different views
- able to do either a split view or a combined view - highlight rows in combined view to show additions/removals - filter either view
- can click properties in the object view on right to highlight them on left - changed all trees to use row select so they select the whole row instead of single cells (just looks better imo)
…but leaves room to change the format going forwards) - do more to null out pointers which I will be comparing to nullptr - only call show_snapshot when a tab is actually viewed. Saves a lot of time rendering the Object, RefCounted, and JSON tabs when they may not be seen - added a context dictionary to snapshots so I can include stuff like memory usage - added support for renaming snapshots from the odb profiler menu (it just renames them on disk) - got rid of the idea of a unix time as the file name. Now, the name just defaults to a valid unix timestamp name - modified how Ref<GameStateSnapshotRef> is made so it's a little simpler?? (nothing is simple about having a ref to a non-refcounted object) to set up
- also added overview data like memory used and total nodes/objects to the summary page
- move all classes to top level
- move all methods out of headers
- order types withing classes
- make sure utility classes are reasonable
- cleanup and revise comments
- p_* for parameters
- r_* for retvals
- always use snake case
- always use {}, never one-line if statements
- delete unused imports (hit or miss, vs code isn't doing a great job telling me what's unused)
- start private/protected methods with _*
…ssages to chunk the data into arrays of 100 elements - better approach is probably to have the game write out the odb_snapshot file itself so we don't have to send this via debug messages. That doesn't work for truely remote debugging, though, so unclear
- don't show multiple root nodes warning when there is only one active root node
- convert to a string in the client to avoid the max_queued_messages limit - send as 6mb string slices to avoid the 8mb in/out buffer limit - reconstruct the string on the client side
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Intro
This PR adds a new tab to the debug menu that can help profile a game's memory usage. It's a follow up to this proposal - godotengine/godot-proposals#10628.
Specifically, this lets you save a snapshot of all the objects in a running game's ObjectDB to disk. It then lets you view the snapshot and diff two snapshots against each other. This is meant to work similarly to Chrome's heap shopshot tool or unity's memory profiler.
UI Overview
The column on the left is universal functionality.
user://objectdb_snapshotsfolder. Click a snapshot to view it. Right click a snapshot to rename, delete, or view in your file browser.The space on the right contains 6 tabs. Each tab shows the contents of the snapshot in a different way. All tabs support diffing in some form.
a. Total memory used
b. Name and creation time of the snapshot
c. Total object count
d. Total node count
Other thoughts
Other than a few fixes in core, most of the code is in a new module. The biggest change to core is adding a lock to
Object::setso that objects cannot be modified whenObjectDB::debug_objectsis running. I'm fairly new to C++, so I'm hoping someone can double check I'm using locks correctly there.Snapshots can be in the 10-100s of megabytes, and the debug network peer has a maximum buffer size of 8mb, so I'm chunking the snapshots into smaller slices and sending them 6mb at a time.
Since snapshots can be pretty big, I store them in an LRUCache and only keep 10 in the editor's memory at a time. If it feels like selecting a snapshot for the first time is slow, that's probably why.
Also along those lines, I don't populate each tab until you actually click on it. It makes loading a snapshot a little faster, but slows down clicking on each tab. The Objects, RefCounted, and JSON tabs can be brutally slow.
In order to show snapshot objects in the inspector, I'm extending the
EditorDebuggerRemoteObjectclass used in the scene tree's remote tab.Snapshots have the
.odb_snapshotfile extension. The first line is a version number, and the second like is the output of callingMarshalls::variant_to_base64. I was going to save this as a .tres, but realized 99% of the data I was saving wasn't a resource, andMarshalls::variant_to_base64was the easiest way I could find to get the data in and out of a string.