feat: GC #2735
feat: GC #2735
Conversation
github-actions
bot
added
📦 🤖 gnovm
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2735 +/- ##
==========================================
+ Coverage 61.10% 61.19% +0.09%
==========================================
Files 564 566 +2
Lines 75355 75643 +288
==========================================
+ Hits 46045 46293 +248
- Misses 25946 25978 +32
- Partials 3364 3372 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
petar-dambovaliev
requested review from
jaekwon,
moul,
piux2,
deelawn,
thehowl,
mvertes and
gfanton
as code owners
| @@ -141,6 +141,31 @@ func TestRunEmptyMain(t *testing.T) { | |||
| m.RunMain() | |||
| } | |||
|
|
|||
| func TestGCLoopy(t *testing.T) { | |||
There was a problem hiding this comment.
Do we expect this to succeed or fail?
It failed with the error message panic: allocation limit exceed on my local machine
There was a problem hiding this comment.
It should pass. I just ran this test locally and it passes.
It also has passed in CI. This is strange that it fails on your machine.
| m.Alloc = NewAllocator(22000, 3000, NewHeap()) | ||
| m.RunMain() | ||
| } | ||
|
|
||
| // run main() with a for loop. |
There was a problem hiding this comment.
I updated ShouldRunGC() so that we can control when to trigger the GC.
func (alloc *Allocator) ShouldRunGC() bool {
// return false
return alloc.bytes >= alloc.minGcRun
}
For the following tests. regardless we triggered GC or not, Alloc.bytes are the same. Not sure if this is the expected behavior
run two simple GC deallocation test.
func TestGCAlloc(t *testing.T) {
c := `package test
func simpleGC() {
{
item1 := make([]byte, 100)
}
}`
m := NewMachine("test", nil)
n := MustParseFile("main.go", c)
m.RunFiles(n)
//m.Alloc = NewAllocator(22000, 10000, NewHeap())
m.Alloc = NewAllocator(22000, 1, NewHeap())
m.Eval(Call("simpleGC"))
fmt.Printf("%+v\n", m.Alloc)
}
//&{maxBytes:22000 bytes:1140 lastGcRun:0 minGcRun:10000 heap:0x1400030bbc0 detonate:false}
//&{maxBytes:22000 bytes:1140 lastGcRun:2348 minGcRun:1 heap:0x14000341bc0 detonate:false}
func TestGCAlloc(t *testing.T) {
c := `package test
func simpleGC() {
{
item1 := make([]byte, 100)
}
}`
m := NewMachine("test", nil)
n := MustParseFile("main.go", c)
m.RunFiles(n)
//m.Alloc = NewAllocator(22000, 10000, NewHeap())
m.Alloc = NewAllocator(22000, 1, NewHeap())
m.Eval(Call("simpleGC"))
fmt.Printf("%+v\n", m.Alloc)
}
// &{maxBytes:22000 bytes:1920 lastGcRun:2440 minGcRun:1 heap:0x14000330120 detonate:false}
// &{maxBytes:22000 bytes:1920 lastGcRun:0 minGcRun:10000 heap:0x1400035a120 detonate:false}
There was a problem hiding this comment.
I updated ShouldRunGC() so that we can control when to trigger the GC.
func (alloc *Allocator) ShouldRunGC() bool { // return false return alloc.bytes >= alloc.minGcRun }For the following tests. regardless we triggered GC or not, Alloc.bytes are the same. Not sure if this is the expected behavior
run two simple GC deallocation test.
func TestGCAlloc(t *testing.T) { c := `package test func simpleGC() { { item1 := make([]byte, 100) } }` m := NewMachine("test", nil) n := MustParseFile("main.go", c) m.RunFiles(n) //m.Alloc = NewAllocator(22000, 10000, NewHeap()) m.Alloc = NewAllocator(22000, 1, NewHeap()) m.Eval(Call("simpleGC")) fmt.Printf("%+v\n", m.Alloc) } //&{maxBytes:22000 bytes:1140 lastGcRun:0 minGcRun:10000 heap:0x1400030bbc0 detonate:false} //&{maxBytes:22000 bytes:1140 lastGcRun:2348 minGcRun:1 heap:0x14000341bc0 detonate:false}func TestGCAlloc(t *testing.T) { c := `package test func simpleGC() { { item1 := make([]byte, 100) } }` m := NewMachine("test", nil) n := MustParseFile("main.go", c) m.RunFiles(n) //m.Alloc = NewAllocator(22000, 10000, NewHeap()) m.Alloc = NewAllocator(22000, 1, NewHeap()) m.Eval(Call("simpleGC")) fmt.Printf("%+v\n", m.Alloc) } // &{maxBytes:22000 bytes:1920 lastGcRun:2440 minGcRun:1 heap:0x14000330120 detonate:false} // &{maxBytes:22000 bytes:1920 lastGcRun:0 minGcRun:10000 heap:0x1400035a120 detonate:false}
Jae said we should only run the GC if we run out of memory. This is why it was commented and always returned false.
There was a problem hiding this comment.
The code you wrote does not reach the limit in memory so the GC is never triggered.
There was a problem hiding this comment.
Thanks, got it.
What about these two testing cases? Does the GC kick in? Is the result expected?
func TestGCAlloc(t *testing.T) {
c := `package test
func simpleGC() {
{
item1 := make([]byte, 10000)
}
{
item2 := make([]byte, 10000)
}
}`
m := NewMachine("test", nil)
n := MustParseFile("main.go", c)
m.RunFiles(n)
m.Alloc = NewAllocator(22000, NewHeap())
m.Eval(Call("simpleGC"))
fmt.Printf("%+v\n", m.Alloc)
}
// panic: allocation limit exceeded
func TestGCAlloc2(t *testing.T) {
c := `package test
func simpleGC() {
for i:=0; i< 1000;i++{
item1 := [10]int{}
}
}`
m := NewMachine("test", nil)
n := MustParseFile("main.go", c)
m.RunFiles(n)
m.Alloc = NewAllocator(22000, NewHeap())
m.Eval(Call("simpleGC"))
fmt.Printf("%+v\n", m.Alloc)
// panic: allocation limit exceeded
}
// panic: allocation limit exceeded
There was a problem hiding this comment.
Yes. These are expected because all the builtins and std are allocating too. So the allowed maximum allocation number needs to be increased. 22000 is not enough for builtins and std plus the stuff you are allocating in your code.
The way the allocator interacts with persistent storage, it shouldn't cause issues for the GC. The GC runs in all tests. We can reduce this now. |
|
@piux2 i reduced the max allocation for the tests by a factor of 10 and everything works. That means the GC is doing its job :) |
Co-authored-by: Lee ByeongJun <[email protected]>
petar-dambovaliev
requested review from
zivkovicmilos and
a team
as code owners
| alloc.AllocateStruct() | ||
| alloc.AllocateStructFields(int64(len(v.Fields))) | ||
| alloc.AllocateType() | ||
| alloc.AllocateHeapItem() |
There was a problem hiding this comment.
StructValue does not imply heapItemValue allocation, IMO.
There was a problem hiding this comment.
This code refers to StructValue as a result of a RefExpr.
The VM currently uses NewHeapItem for it.
case *CompositeLitExpr: // for *RefExpr
tv := *m.PopValue()
hv := m.Alloc.NewHeapItem(tv)
return PointerValue{
TV: &hv.Value,
Base: hv,
Index: 0,
}| alloc.DeallocateStruct() | ||
| alloc.DeallocateStructFields(int64(len(v.Fields))) | ||
| alloc.DeallocateType() | ||
| alloc.DeallocateHeapItem() |
There was a problem hiding this comment.
same as above
Co-authored-by: ltzmaxwell <[email protected]>
Deterministic garbage collector for gnovm.
Tracks allocations and deallocates objects that aren't used anymore.
It implements a simple mark and sweep garbage collector.
Related issue
Root objects (objects through which heap allocated objects are accessed) are identified in each scope and are released at the end of that scope.
When a heap allocated object has no roots, it is considered garbage and it is deallocated.
Maps are not implemented here.
For simplicity for reviewing and merging, they will be added in a following PR.