Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NativeAOT app crashes when run on a machine with an ETW session with GCHeapDump and Type keywords enabled #88491

Closed
elinor-fung opened this issue Jul 6, 2023 · 7 comments · Fixed by #94673
Assignees
Milestone

Comments

@elinor-fung
Copy link
Member

elinor-fung commented Jul 6, 2023

  1. Start an ETW session, enabling the .NET runtime provider with GCHeapDump and Type keywords
    • For example, collect with PerfView and set additional providers to Microsoft-Windows-DotNETRuntime:GCHeapDumpKeyword|TypeKeyword:Verbose
  2. Run a nativeaot app that has <EventSourceSupport>true</EventSourceSupport> and does a GC (explicit call or otherwise)
    • The ETW session is for the machine, so regardless of whether whoever started the session cares about the nativeaot app, the app is affected.

App crashes trying to get the type manager from a method table. The runtime is trying to use the GCStaticEEType method table, which is intentionally not fully populated.

> helloworld.exe!TypeManagerHandle::AsTypeManager() Line 105
  helloworld.exe!BulkTypeEventLogger::LogSingleType(MethodTable * pEEType) Line 252
  helloworld.exe!BulkTypeEventLogger::LogTypeAndParameters(unsigned __int64 thAsAddr, ETW::TypeSystemLog::TypeLogBehavior typeLogBehavior) Line 54
  helloworld.exe!ETW::TypeSystemLog::LogTypeAndParametersIfNecessary(BulkTypeEventLogger * pLogger, unsigned __int64 thAsAddr, ETW::TypeSystemLog::TypeLogBehavior typeLogBehavior) Line 357
  helloworld.exe!ETW::GCLog::ObjectReference(ProfilerWalkHeapContext * profilerWalkHeapContext, Object * pObjReferenceSource, unsigned __int64 typeID, unsigned __int64 cRefs, Object * * rgObjReferenceTargets) Line 1046
  helloworld.exe!HeapWalkHelper(Object * pBO, void * pvContext) Line 188
  helloworld.exe!WKS::gc_heap::walk_heap_per_heap(bool(*)(Object *, void *) fn, void * context, int gen_number, int walk_large_object_heap_p) Line 51537
  helloworld.exe!WKS::gc_heap::walk_heap(bool(*)(Object *, void *) fn, void * context, int gen_number, int walk_large_object_heap_p) Line 51563
  helloworld.exe!WKS::GCHeap::DiagWalkHeap(bool(*)(Object *, void *) fn, void * context, int gen_number, bool walk_large_object_heap_p) Line 51617
  helloworld.exe!GCProfileWalkHeapWorker(int fShouldWalkHeapRootsForEtw, int fShouldWalkHeapObjectsForEtw) Line 897
  helloworld.exe!GCProfileWalkHeap() Line 927
  helloworld.exe!GCToEEInterface::DiagGCEnd(unsigned __int64 index, int gen, int reason, bool fConcurrent) Line 955
  helloworld.exe!WKS::gc_heap::do_post_gc() Line 49754

cc @LakshanF

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 6, 2023
@ghost
Copy link

ghost commented Jul 6, 2023

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details
  1. Start an ETW session, enabling the .NET runtime provider with GCHeapDump and Type keywords
    • For example, collect with PerfView and set additional providers to Microsoft-Windows-DotNETRuntime:GCHeapDumpKeyword|TypeKeyword:Verbose
  2. Run a nativeaot app that does a GC (explicit call or otherwise)
    • The ETW session is for the machine, so regardless of whether whoever started the session cares about the nativeaot app, the app is affected.

App crashes trying to get the type manager from a method table. The runtime is trying to use the GCStaticEEType method table, which is intentionally not fully populated.

> helloworld.exe!TypeManagerHandle::AsTypeManager() Line 105
  helloworld.exe!BulkTypeEventLogger::LogSingleType(MethodTable * pEEType) Line 252
  helloworld.exe!BulkTypeEventLogger::LogTypeAndParameters(unsigned __int64 thAsAddr, ETW::TypeSystemLog::TypeLogBehavior typeLogBehavior) Line 54
  helloworld.exe!ETW::TypeSystemLog::LogTypeAndParametersIfNecessary(BulkTypeEventLogger * pLogger, unsigned __int64 thAsAddr, ETW::TypeSystemLog::TypeLogBehavior typeLogBehavior) Line 357
  helloworld.exe!ETW::GCLog::ObjectReference(ProfilerWalkHeapContext * profilerWalkHeapContext, Object * pObjReferenceSource, unsigned __int64 typeID, unsigned __int64 cRefs, Object * * rgObjReferenceTargets) Line 1046
  helloworld.exe!HeapWalkHelper(Object * pBO, void * pvContext) Line 188
  helloworld.exe!WKS::gc_heap::walk_heap_per_heap(bool(*)(Object *, void *) fn, void * context, int gen_number, int walk_large_object_heap_p) Line 51537
  helloworld.exe!WKS::gc_heap::walk_heap(bool(*)(Object *, void *) fn, void * context, int gen_number, int walk_large_object_heap_p) Line 51563
  helloworld.exe!WKS::GCHeap::DiagWalkHeap(bool(*)(Object *, void *) fn, void * context, int gen_number, bool walk_large_object_heap_p) Line 51617
  helloworld.exe!GCProfileWalkHeapWorker(int fShouldWalkHeapRootsForEtw, int fShouldWalkHeapObjectsForEtw) Line 897
  helloworld.exe!GCProfileWalkHeap() Line 927
  helloworld.exe!GCToEEInterface::DiagGCEnd(unsigned __int64 index, int gen, int reason, bool fConcurrent) Line 955
  helloworld.exe!WKS::gc_heap::do_post_gc() Line 49754

cc @LakshanF

Author: elinor-fung
Assignees: -
Labels:

area-NativeAOT-coreclr

Milestone: -

@MichalStrehovsky MichalStrehovsky added this to the 8.0.0 milestone Jul 11, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Jul 11, 2023
@agocke
Copy link
Member

agocke commented Jul 26, 2023

@VSadov could you look into this one?

@elinor-fung
Copy link
Member Author

Note that after #88800, you also need to specify <EventSourceSupport>true</EventSourceSupport> in the app (updated the issue description) because firing ETW events was moved to no longer be enabled by default.

@LakshanF
Copy link
Member

LakshanF commented Aug 1, 2023

I think it would be OK to return false on these complex events for this release, given our position only to support basic events for now.

@LakshanF LakshanF mentioned this issue Aug 4, 2023
@agocke
Copy link
Member

agocke commented Aug 15, 2023

Looks like the crash is fixed. Work to enable is tracked in #88162

@agocke agocke closed this as completed Aug 15, 2023
@elinor-fung
Copy link
Member Author

I'll leave it to @LakshanF to decide how he'd like to track it, but I think it is worth noting that #88162 will be fixed naturally by switching to the generated scripts (generated Fire* functions replacing dotnetruntime.cpp will use both EventPipe and ETW - also generated - on Windows). However, the crash here was in trying to determine the information to include in the events - so there'll still be different work to sort that out.

@LakshanF LakshanF assigned LakshanF and unassigned VSadov Aug 16, 2023
@LakshanF LakshanF modified the milestones: 8.0.0, 9.0.0 Aug 16, 2023
@LakshanF LakshanF reopened this Aug 16, 2023
@LakshanF
Copy link
Member

Will fix this after #88162

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Nov 13, 2023
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Nov 21, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Dec 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

5 participants