When/why does VM heap decrease?

[jensjoha]

I'm playing around with the analyzers memory usage.

Via #41793 (comment) I found and checked out https://github.com/kodecocodes/rwf-materials.git and opened the entire directory (after running flutter pub get 500+ times --- I don't know if there's a helper command but that's what I did anyway).

With an empty cache the memory usage goes to (on my machine) ~14GB.

Now comes a hack: I've found that creating a file and deleting it again makes the analyzer clear some (unneeded) cache stuff (and do all kinds of stuff again, but still) --- once it's done the memory usage is now ~11GB, but the used heap is in the ~3.5GB range. Doing the same thing again makes the memory usage go to ~9GB and the heap usage is still in the ~3.5GB range (e.g. it's displayed as heap 3.49GB of 9.13GB in Observatory).

So: Why doesn't the VM reduce the heap size further? Is there something else that has to be satisfied for it to reduce the heap or something?

/cc @mkustermann @mraleph

[mkustermann]

Generally old space is organized in pages, so if there's at least one object on a page, the whole heap page is retained. So a partial reason may be fragmentation. Though if the analyzer is ever idle, it may trigger compacting GC, which can get rid of this fragmentation.

/cc @rmacnak-google

[rmacnak-google]

@jensjoha What command are you using to run the analyzer?

[jensjoha]

I'm opening the directory with Android Studio (and enabling dart support for the dir).

[rmacnak-google]

How are you getting to Observatory? I can see some analyzer-specific diagnostic page, but it claims the VM service is not running, and sending the usual sigquit to the pid doesn't enable the VM service. I also don't see anywhere to set flags passed to the analyzer invocation.

[rmacnak-google]

I suspect what is happening is that the heap size is so large that the VM estimates it will taking longer than --idle_duration_micros to complete a compaction, and so chooses mark-sweep instead of mark-compact to avoid blocking for too long.

[jensjoha]

I'm cheating...
I'm applying this patch:

diff --git a/pkg/analysis_server/lib/src/status/diagnostics.dart b/pkg/analysis_server/lib/src/status/diagnostics.dart
index 82aa23c37cd..6558597e9e4 100644
--- a/pkg/analysis_server/lib/src/status/diagnostics.dart
+++ b/pkg/analysis_server/lib/src/status/diagnostics.dart
@@ -460,10 +460,10 @@ class CollectReportPage extends DiagnosticPage {
       }
     }
 
-    if (startedServiceProtocol) {
-      await developer.Service.controlWebServer(
-          enable: false, silenceOutput: true);
-    }
 
     const JsonEncoder encoder = JsonEncoder.withIndent('  ');
     return encoder.convert(collectedData);

opening the "Analysis Server Diagnostics" via Android Studio -> "Dart Analysis" bottom page thing -> wheel on the left -> "View Analyzer Diagnostics" -> Clicking "Collect Report" and "Download report". Then from "Diagnostics" -> "Memory and CPU Usage" I can see the URL for Observatory.

[mraleph]

I suspect what is happening is that the heap size is so large that the VM estimates it will taking longer than --idle_duration_micros to complete a compaction, and so chooses mark-sweep instead of mark-compact to avoid blocking for too long.

I guess this might be one of those huge-heap use cases where compacting gradually via selective/partial evacuation of pages is better than our current full sliding compaction which is tiered towards smaller heaps.

[mraleph]

@rmacnak-google did you have any chance to look at this? it would be nice to find out why VM does not shrink the heap more aggressively.

[rmacnak-google]

It was the estimated time exceeding the deadline. See 2b8ca6f (I linked the wrong issue).

[mraleph]

@rmacnak-google thanks! I suspected it was related.

@jensjoha when you have some free time could you check how things look on master now wrt to analyzer heap? Maybe we can close this issue then.

[jensjoha]

It's now at ~3GB so seems fixed to me :).
Closing!