Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allocate string literals on frozen segments #49576

Merged
merged 69 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
166ccfe
Naive implementation of FrozenObjectHeap
EgorBo Jul 3, 2021
1cb0927
Add RegisterFrozenSegment
EgorBo Jul 3, 2021
d2b9cbf
Clean up
EgorBo Jul 4, 2021
641365a
Add #include "common.h"
EgorBo Jul 4, 2021
33a126b
Fix build error
EgorBo Jul 4, 2021
bb1b835
Fix build
EgorBo Jul 4, 2021
2ef204a
Add lock
EgorBo Jul 4, 2021
a84a6ee
Clean up
EgorBo Jul 4, 2021
142295a
init memory on first allocation
EgorBo Jul 4, 2021
a446dd6
Clean up
EgorBo Jul 4, 2021
41665c3
revert changes in emitxarch.cpp
EgorBo Jul 4, 2021
66d6106
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Jul 28, 2022
67f32a7
Update branch, fix asserts
EgorBo Jul 28, 2022
9ecf7d7
fix release build
EgorBo Jul 28, 2022
45a454a
Ignore collectible assemblies, fix build
EgorBo Jul 29, 2022
1d0a710
Fix failing tests, address feedback
EgorBo Jul 29, 2022
201565e
Rename bAppDomainWontUnload to bIsCollectible
EgorBo Jul 29, 2022
c7dad85
Address feedback
EgorBo Jul 29, 2022
4202d9f
Clean up
EgorBo Jul 29, 2022
69ab9d2
Merge branch 'main' of github.com:dotnet/runtime into poh2
EgorBo Jul 29, 2022
a144453
Fix assert
EgorBo Jul 29, 2022
2571c9c
Fix more asserts not expecting TYP_REF constants being non zero
EgorBo Jul 29, 2022
286f8ab
Merge branch 'main' of github.com:dotnet/runtime into poh2
EgorBo Jul 29, 2022
28e0331
Test gc fix
EgorBo Jul 29, 2022
6625759
move check to IsEphemeral
EgorBo Jul 30, 2022
89df356
Test commit (do I need to update "ibAllocated"?)
EgorBo Jul 30, 2022
3cb98e9
Implement commit-on-demand
EgorBo Jul 30, 2022
c6df9ed
Clean up
EgorBo Jul 30, 2022
f50b4b9
bump FOH_COMMIT_PAGES
EgorBo Jul 30, 2022
2b55835
Merge branch 'main' of github.com:dotnet/runtime into poh2
EgorBo Jul 30, 2022
6ea2a50
Address feedback, clean up
EgorBo Jul 30, 2022
15ca880
Fix build
EgorBo Jul 30, 2022
193676e
Merge branch 'main' of github.com:dotnet/runtime into poh2
EgorBo Jul 30, 2022
9198043
Unix's ClrVirtualAlloc seems to be non-aligned
EgorBo Jul 30, 2022
7c98ef2
Oops, fix red CI
EgorBo Jul 30, 2022
af6e445
Multiple frozen segments
EgorBo Jul 30, 2022
2183c2f
Merge branch 'main' of github.com:dotnet/runtime into poh2
EgorBo Jul 31, 2022
4ef34dd
Clean up
EgorBo Jul 31, 2022
722eb82
fix typo
EgorBo Jul 31, 2022
0ae890a
Fix assert in assertionprop.cpp
EgorBo Jul 31, 2022
e59d27a
Address feedback
EgorBo Aug 18, 2022
90feb96
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Aug 18, 2022
ce3e19e
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Sep 1, 2022
b6fae80
Address feedback
EgorBo Sep 1, 2022
32cb338
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Sep 1, 2022
d20cc47
Address feedback
EgorBo Sep 2, 2022
31b5fdc
Fix potential issues
EgorBo Sep 3, 2022
1b1ab4c
Fix potential issues
EgorBo Sep 3, 2022
ed8ed7b
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Sep 4, 2022
88ad320
Address feedback
EgorBo Sep 4, 2022
943a235
Address feedback
EgorBo Sep 4, 2022
a3bfc8c
Check if this fixes all asserts
EgorBo Sep 5, 2022
1be12c5
Handle string.Empty (it used to be ignored)
EgorBo Sep 5, 2022
5c2a222
Apply suggestions from code review
EgorBo Sep 5, 2022
19d1734
Address feedback
EgorBo Sep 5, 2022
4762bcc
Address feedback
EgorBo Sep 5, 2022
2e03cc2
Update src/coreclr/vm/stringliteralmap.cpp
EgorBo Sep 5, 2022
2f00f71
Update src/coreclr/vm/stringliteralmap.cpp
EgorBo Sep 5, 2022
ddfbf73
fix compilation errors
EgorBo Sep 5, 2022
598b9c8
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Sep 7, 2022
802106e
Address Jan's feedback
EgorBo Sep 7, 2022
bfe96e7
Update src/coreclr/jit/gentree.cpp
EgorBo Sep 7, 2022
2e10c9c
Update gentree.cpp
EgorBo Sep 7, 2022
3415dc3
Merge branch 'main' of github.com:dotnet/runtime into poh-string-lite…
EgorBo Sep 12, 2022
bac0389
Address Maoni's feedback:
EgorBo Sep 13, 2022
01b5731
heap -> segment in comments
EgorBo Sep 13, 2022
8d139c9
Update src/coreclr/vm/frozenobjectheap.h
EgorBo Sep 13, 2022
b6deb3c
Address Jan's feedback around m_Crst creation
EgorBo Sep 13, 2022
1ba9d97
Fix order of initialization
EgorBo Sep 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions src/coreclr/gc/gcee.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,15 @@ bool GCHeap::IsInFrozenSegment(Object *object)
#endif
}

void GCHeap::UpdateFrozenSegment(segment_handle seg, uint8_t* allocated, uint8_t* committed)
{
#ifdef FEATURE_BASICFREEZE
heap_segment* heap_seg = reinterpret_cast<heap_segment*>(seg);
heap_segment_committed(heap_seg) = committed;
heap_segment_allocated(heap_seg) = allocated;
#endif // FEATURE_BASICFREEZE
}

bool GCHeap::RuntimeStructuresValid()
{
return GCScan::GetGcRuntimeStructuresValid();
Expand Down
1 change: 1 addition & 0 deletions src/coreclr/gc/gcimpl.h
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ class GCHeap : public IGCHeapInternal
virtual segment_handle RegisterFrozenSegment(segment_info *pseginfo);
virtual void UnregisterFrozenSegment(segment_handle seg);
virtual bool IsInFrozenSegment(Object *object);
virtual void UpdateFrozenSegment(segment_handle seg, uint8_t* allocated, uint8_t* committed);

// Event control functions
void ControlEvents(GCEventKeyword keyword, GCEventLevel level);
Expand Down
3 changes: 3 additions & 0 deletions src/coreclr/gc/gcinterface.h
Original file line number Diff line number Diff line change
Expand Up @@ -955,6 +955,9 @@ class IGCHeap {

// Gets all the names and values of the GC configurations.
virtual void EnumerateConfigurationValues(void* context, ConfigurationValueFunc configurationValueFunc) = 0;

// Updates given frozen segment
virtual void UpdateFrozenSegment(segment_handle seg, uint8_t* allocated, uint8_t* committed) = 0;
};

#ifdef WRITE_BARRIER_CHECK
Expand Down
5 changes: 5 additions & 0 deletions src/coreclr/inc/clrconfigvalues.h
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,11 @@ RETAIL_CONFIG_DWORD_INFO(INTERNAL_CodeHeapReserveForJumpStubs, W("CodeHeapReserv
RETAIL_CONFIG_DWORD_INFO(INTERNAL_NGenReserveForJumpStubs, W("NGenReserveForJumpStubs"), 0, "Percentage of ngen image size to reserve for jump stubs")
RETAIL_CONFIG_DWORD_INFO(INTERNAL_BreakOnOutOfMemoryWithinRange, W("BreakOnOutOfMemoryWithinRange"), 0, "Break before out of memory within range exception is thrown")

///
/// Frozen segments (aka Frozen Object Heap)
///
RETAIL_CONFIG_DWORD_INFO(INTERNAL_UseFrozenObjectHeap, W("UseFrozenObjectHeap"), 1, "Use frozen object heap for certain types of objects (e.g. string literals) as an optimization.")

///
/// Log
///
Expand Down
1 change: 1 addition & 0 deletions src/coreclr/inc/corinfo.h
Original file line number Diff line number Diff line change
Expand Up @@ -1931,6 +1931,7 @@ struct CORINFO_VarArgInfo
#define SIZEOF__CORINFO_Object TARGET_POINTER_SIZE /* methTable */

#define CORINFO_Array_MaxLength 0x7FFFFFC7
#define CORINFO_String_MaxLength 0x3FFFFFDF

#define OFFSETOF__CORINFO_Array__length SIZEOF__CORINFO_Object
#ifdef TARGET_64BIT
Expand Down
5 changes: 4 additions & 1 deletion src/coreclr/inc/crsttypes.h
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,8 @@ enum CrstType
CrstUnwindInfoTableLock = 113,
CrstVSDIndirectionCellLock = 114,
CrstWrapperTemplate = 115,
kNumberOfCrstTypes = 116
CrstFrozenObjectHeap = 116,
kNumberOfCrstTypes = 117
};

#endif // __CRST_TYPES_INCLUDED
Expand Down Expand Up @@ -258,6 +259,7 @@ int g_rgCrstLevelMap[] =
3, // CrstUnwindInfoTableLock
4, // CrstVSDIndirectionCellLock
3, // CrstWrapperTemplate
0, // CrstFrozenObjectHeap
};

// An array mapping CrstType to a stringized name.
Expand Down Expand Up @@ -379,6 +381,7 @@ LPCSTR g_rgCrstNameMap[] =
"CrstUnwindInfoTableLock",
"CrstVSDIndirectionCellLock",
"CrstWrapperTemplate",
"CrstFrozenObjectHeap"
};

// Define a special level constant for unordered locks.
Expand Down
34 changes: 22 additions & 12 deletions src/coreclr/jit/assertionprop.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1155,8 +1155,14 @@ void Compiler::optPrintAssertion(AssertionDsc* curAssertion, AssertionIndex asse

if (op1Type == TYP_REF)
{
assert(curAssertion->op2.u1.iconVal == 0);
printf("null");
if (curAssertion->op2.u1.iconVal == 0)
{
printf("null");
}
else
{
printf("[%08p]", dspPtr(curAssertion->op2.u1.iconVal));
}
}
else
{
Expand Down Expand Up @@ -2646,8 +2652,10 @@ AssertionInfo Compiler::optAssertionGenJtrue(GenTree* tree)
GenTree* objectNode = call->gtArgs.GetArgByIndex(1)->GetNode();
GenTree* methodTableNode = call->gtArgs.GetArgByIndex(0)->GetNode();

assert(objectNode->TypeGet() == TYP_REF);
assert(methodTableNode->TypeGet() == TYP_I_IMPL);
// objectNode can be TYP_I_IMPL in case if it's a constant handle
// (e.g. a string literal from frozen segments)
assert(objectNode->TypeIs(TYP_REF, TYP_I_IMPL));
assert(methodTableNode->TypeIs(TYP_I_IMPL));

// Reverse the assertion
assert((assertionKind == OAK_EQUAL) || (assertionKind == OAK_NOT_EQUAL));
Expand Down Expand Up @@ -3134,11 +3142,9 @@ GenTree* Compiler::optVNConstantPropOnTree(BasicBlock* block, GenTree* tree)

case TYP_REF:
{
assert(vnStore->ConstantValue<size_t>(vnCns) == 0);
// Support onle ref(ref(0)), do not support other forms (e.g byref(ref(0)).
if (tree->TypeGet() == TYP_REF)
{
conValTree = gtNewIconNode(0, TYP_REF);
conValTree = gtNewIconNode(vnStore->ConstantValue<size_t>(vnCns), TYP_REF);
}
}
break;
Expand Down Expand Up @@ -4285,8 +4291,14 @@ GenTree* Compiler::optAssertionPropGlobal_RelOp(ASSERT_VALARG_TP assertions, Gen
else if (op1->TypeGet() == TYP_REF)
{
// The only constant of TYP_REF that ValueNumbering supports is 'null'
assert(vnStore->ConstantValue<size_t>(vnCns) == 0);
printf("null\n");
if (vnStore->ConstantValue<size_t>(vnCns) == 0)
{
printf("null\n");
}
else
{
printf("%d (gcref)\n", static_cast<target_ssize_t>(vnStore->ConstantValue<size_t>(vnCns)));
}
}
else if (op1->TypeGet() == TYP_BYREF)
{
Expand Down Expand Up @@ -4339,9 +4351,7 @@ GenTree* Compiler::optAssertionPropGlobal_RelOp(ASSERT_VALARG_TP assertions, Gen
}
else if (op1->TypeGet() == TYP_REF)
{
op1->BashToConst(0, TYP_REF);
// The only constant of TYP_REF that ValueNumbering supports is 'null'
noway_assert(vnStore->ConstantValue<size_t>(vnCns) == 0);
op1->BashToConst(static_cast<target_ssize_t>(vnStore->ConstantValue<size_t>(vnCns)), TYP_REF);
}
else if (op1->TypeGet() == TYP_BYREF)
{
Expand Down
2 changes: 0 additions & 2 deletions src/coreclr/jit/codegencommon.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1374,8 +1374,6 @@ bool CodeGen::genCreateAddrMode(

if (varTypeIsGC(rv2->TypeGet()))
{
noway_assert(rv1 && !varTypeIsGC(rv1->TypeGet()));

tmp = rv1;
rv1 = rv2;
rv2 = tmp;
Expand Down
10 changes: 8 additions & 2 deletions src/coreclr/jit/gentree.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3838,10 +3838,16 @@ bool Compiler::gtIsLikelyRegVar(GenTree* tree)
//
bool Compiler::gtCanSwapOrder(GenTree* firstNode, GenTree* secondNode)
{
// Relative of order of global / side effects can't be swapped.

bool canSwap = true;

// Don't swap "CONST_HDL op CNS"
if (firstNode->IsIconHandle() && secondNode->IsIntegralConst())
{
canSwap = false;
}

// Relative of order of global / side effects can't be swapped.

if (optValnumCSE_phase)
{
canSwap = optCSE_canSwap(firstNode, secondNode);
Expand Down
2 changes: 2 additions & 0 deletions src/coreclr/vm/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,7 @@ set(VM_SOURCES_WKS
fcall.cpp
fieldmarshaler.cpp
finalizerthread.cpp
frozenobjectheap.cpp
gccover.cpp
gcenv.ee.static.cpp
gcenv.ee.common.cpp
Expand Down Expand Up @@ -424,6 +425,7 @@ set(VM_HEADERS_WKS
fcall.h
fieldmarshaler.h
finalizerthread.h
frozenobjectheap.h
gcenv.h
gcenv.ee.h
gcenv.os.h
Expand Down
22 changes: 21 additions & 1 deletion src/coreclr/vm/appdomain.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include "assemblynative.hpp"
#include "shimload.h"
#include "stringliteralmap.h"
#include "frozenobjectheap.h"
#include "codeman.h"
#include "comcallablewrapper.h"
#include "eventtrace.h"
Expand Down Expand Up @@ -97,7 +98,8 @@ CrstStatic BaseDomain::m_SpecialStaticsCrst;
int BaseDomain::m_iNumberOfProcessors = 0;

// System Domain Statics
GlobalStringLiteralMap* SystemDomain::m_pGlobalStringLiteralMap = NULL;
GlobalStringLiteralMap* SystemDomain::m_pGlobalStringLiteralMap = NULL;
FrozenObjectHeapManager* SystemDomain::m_FrozenObjectHeapManager = NULL;

DECLSPEC_ALIGN(16)
static BYTE g_pSystemDomainMemory[sizeof(SystemDomain)];
Expand Down Expand Up @@ -1191,6 +1193,24 @@ void SystemDomain::LazyInitGlobalStringLiteralMap()
}
}

void SystemDomain::LazyInitFrozenObjectsHeap()
{
CONTRACTL
{
THROWS;
GC_TRIGGERS;
MODE_ANY;
INJECT_FAULT(COMPlusThrowOM(););
}
CONTRACTL_END;

NewHolder<FrozenObjectHeapManager> pFoh(new FrozenObjectHeapManager());
if (InterlockedCompareExchangeT<FrozenObjectHeapManager*>(&m_FrozenObjectHeapManager, pFoh, nullptr) == nullptr)
{
pFoh.SuppressRelease();
}
}

/*static*/ void SystemDomain::EnumAllStaticGCRefs(promote_func* fn, ScanContext* sc)
{
CONTRACT_VOID
Expand Down
12 changes: 12 additions & 0 deletions src/coreclr/vm/appdomain.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ class SystemDomain;
class AppDomain;
class GlobalStringLiteralMap;
class StringLiteralMap;
class FrozenObjectHeapManager;
class MngStdInterfacesInfo;
class DomainAssembly;
class LoadLevelLimiter;
Expand Down Expand Up @@ -2388,6 +2389,7 @@ class SystemDomain : public BaseDomain
void Init();
void Stop();
static void LazyInitGlobalStringLiteralMap();
static void LazyInitFrozenObjectsHeap();

//****************************************************************************************
//
Expand Down Expand Up @@ -2460,6 +2462,15 @@ class SystemDomain : public BaseDomain
_ASSERTE(m_pGlobalStringLiteralMap);
return m_pGlobalStringLiteralMap;
}
static FrozenObjectHeapManager* GetFrozenObjectHeapManager()
{
WRAPPER_NO_CONTRACT;
if (m_FrozenObjectHeapManager == NULL)
{
LazyInitFrozenObjectsHeap();
}
return m_FrozenObjectHeapManager;
}
#endif // DACCESS_COMPILE

#if defined(FEATURE_COMINTEROP_APARTMENT_SUPPORT)
Expand Down Expand Up @@ -2629,6 +2640,7 @@ class SystemDomain : public BaseDomain
static CrstStatic m_SystemDomainCrst;

static GlobalStringLiteralMap *m_pGlobalStringLiteralMap;
static FrozenObjectHeapManager *m_FrozenObjectHeapManager;
#endif // DACCESS_COMPILE

public:
Expand Down
4 changes: 2 additions & 2 deletions src/coreclr/vm/ceeload.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2835,7 +2835,7 @@ void ModuleBase::InitializeStringData(DWORD token, EEStringData *pstrData, CQuic
}


OBJECTHANDLE ModuleBase::ResolveStringRef(DWORD token)
OBJECTHANDLE ModuleBase::ResolveStringRef(DWORD token, void** ppPinnedString)
{
CONTRACTL
{
Expand Down Expand Up @@ -2864,7 +2864,7 @@ OBJECTHANDLE ModuleBase::ResolveStringRef(DWORD token)

pLoaderAllocator = this->GetLoaderAllocator();

string = (OBJECTHANDLE)pLoaderAllocator->GetStringObjRefPtrFromUnicodeString(&strData);
string = (OBJECTHANDLE)pLoaderAllocator->GetStringObjRefPtrFromUnicodeString(&strData, ppPinnedString);

return string;
}
Expand Down
2 changes: 1 addition & 1 deletion src/coreclr/vm/ceeload.h
Original file line number Diff line number Diff line change
Expand Up @@ -579,7 +579,7 @@ class ModuleBase
}

// Resolving
OBJECTHANDLE ResolveStringRef(DWORD Token);
OBJECTHANDLE ResolveStringRef(DWORD Token, void** ppPinnedString = nullptr);
private:
// string helper
void InitializeStringData(DWORD token, EEStringData *pstrData, CQuickBytes *pqb);
Expand Down
3 changes: 2 additions & 1 deletion src/coreclr/vm/dynamicmethod.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1275,7 +1275,8 @@ STRINGREF* LCGMethodResolver::GetOrInternString(STRINGREF *pProtectedStringRef)
// lock the global string literal interning map
CrstHolder gch(pStringLiteralMap->GetHashTableCrstGlobal());

StringLiteralEntryHolder pEntry(pStringLiteralMap->GetInternedString(pProtectedStringRef, dwHash, /* bAddIfNotFound */ TRUE));
StringLiteralEntryHolder pEntry(pStringLiteralMap->GetInternedString(pProtectedStringRef, dwHash,
/* bAddIfNotFound */ TRUE, /* bPreferFrozenObjectHeap */ FALSE));

DynamicStringLiteral* pStringLiteral = (DynamicStringLiteral*)m_jitTempData.New(sizeof(DynamicStringLiteral));
pStringLiteral->m_pEntry = pEntry.Extract();
Expand Down
Loading