Memory use regression in 1.4.0?

[cptwunderlich]

I've been using HLS in VS Code for working on GHC for a few months and am very happy with it.
In fact, I can't imagine working without it anymore.

But in the last few days, memory use exploded several times. So much, that I had to hard reset my laptop 3 times in 2 days bc. it was unresponsive for a long time.

I'm sure you get a lot of complaints about memory usage, but is this a regression/new bug? At this rate, I can't use HLS bc. it will lock up my PC every few hours and I can't even kill the process bc. it's swapping like crazy.
I think others are also affected.

Repro: Uhm, check out GHC source tree and start editing?

Versions:

haskell-language-server version: 1.4.0.0 (GHC: 8.10.4) (PATH: /home/ben/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-1.4.0-linux-8.10.4) (GIT hash: 2535478)
Tool versions found on the $PATH
cabal: 3.4.0.0
stack: 2.5.1
ghc: 8.10.4

vscode-haskell: v1.6.1
VS Code: 1.60.2

OS: Ubuntu 20.04.3 LTS, Linux 5.11.0-34-generic
AMD Ryzen 7 4700U, 16 GB RAM

[jneira]

Hi, recently @pepeiborra has indentified a perfomance regression for 1.4.0 and it is working in improving it:

• Parallel fuzzy filtering #2225
• Return completions lazily for massive savings #2217
• Inline Text.Fuzzy to add INLINABLE pragmas #2215

Not sure if problems loading ghc in hls would be related but it seems those improvements which will be included in 1.5.0 could help
We have an issue about perfomance issues in large projects where you can find some workarounds and ways to measure and help to diagnose the ghc case:
#2151

[pepeiborra]

If you could quantify the memory use regression between 1.3 and 1.4, that would be very helpful.

[jneira]

Maybe @mpickering could help as afaik he is actively developing ghc using hls and due his expertise in performance issues

[mpickering]

@cptwunderlich I am using a build from this commit d56ea60 with most of the plugins disabled and I am not seeing any really bad memory issues at the moment.

diff --git a/haskell-language-server.cabal b/haskell-language-server.cabal
index 8dd9e26a..0b582ad0 100644
--- a/haskell-language-server.cabal
+++ b/haskell-language-server.cabal
@@ -103,7 +103,7 @@ flag all-formatters
 
 flag class
   description: Enable class plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag callHierarchy
@@ -113,12 +113,12 @@ flag callHierarchy
 
 flag haddockComments
   description: Enable haddockComments plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag eval
   description: Enable eval plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag importLens
@@ -138,22 +138,22 @@ flag rename
 
 flag retrie
   description: Enable retrie plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag tactic
   description: Enable tactic plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag hlint
   description: Enable hlint plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag moduleName
   description: Enable moduleName plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag pragmas
@@ -163,34 +163,34 @@ flag pragmas
 
 flag splice
   description: Enable splice plugin
-  default:     True
+  default:     False
   manual:      True
 
 -- formatters
 
 flag floskell
   description: Enable floskell plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag fourmolu
   description: Enable fourmolu plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag ormolu
   description: Enable ormolu plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag stylishHaskell
   description: Enable stylishHaskell plugin
-  default:     True
+  default:     False
   manual:      True
 
 flag brittany
   description: Enable brittany plugin
-  default:     True
+  default:     False
   manual:      True
 
 common example-plugins

[jneira]

Alternatively plugins can be deactivated at runtime to test its impact quickly, although it is not exactly the same than remove it from builds. See an example on how to do it here: #2151 (comment)

[Vlix]

I've noticed something too because I've switched over to HLS-1.4.0 (caveat: I always build manually with -dynamic flags because of external C library reasons)

When I start up VSCode and open a *.hs file, it's all fine and dandy, but with every other *.hs I open, another instance of the HLS seems to be started (or something) because I see another Haskell (workspacename) popping up in the dropdown list on the Output tab (next to Problems and Terminal) and especially when opening a *.hs file outside of the workspace/stack project I'm in, the performance seems to start to degrade.

I wonder how much this might be part of the problem, but I'd at least wanted to share what I've noticed.

[cptwunderlich]

If you could quantify the memory use regression between 1.3 and 1.4, that would be very helpful.

I'm sorry, I know this was kind of vague. I'm afraid I can't quantify it. I used to be able to work and recently I'm running out of memory.

Anyway, it seems that these are known issues and fixes are in the making!

I did deactivate a few plugins and set "Haskell: Log File", "Haskell > Trace: Client: debug" and "Haskell > Trace: Server: messages" - I will keep an eye on the log file when this happens again.

@Vlix well, I was using all defaults with the VS Code extension downloaded binary. However, I did see two running instances of HLS once or twice. Not sure if I can reproduce or see why it happened...

[jneira]

Hi, duplicate output logs could be caused by haskell/vscode-haskell#448
(so it not should degrade performance per se)
Will be fixed in the incoming next extension release

[Vlix]

Hi, duplicate output logs could be caused by haskell/vscode-haskell#448
(so it not should degrade performance per se)
Will be fixed in the incoming next extension release

Ah, that might very well be it. Though I did still notice degraded performance when opening files outside of the workspace/cradle I was in. 🤔

[Friede80]

I've noticed something too because I've switched over to HLS-1.4.0 (caveat: I always build manually with -dynamic flags because of external C library reasons)

When I start up VSCode and open a *.hs file, it's all fine and dandy, but with every other *.hs I open, another instance of the HLS seems to be started (or something) because I see another Haskell (workspacename) popping up in the dropdown list on the Output tab (next to Problems and Terminal) and especially when opening a *.hs file outside of the workspace/stack project I'm in, the performance seems to start to degrade.

I wonder how much this might be part of the problem, but I'd at least wanted to share what I've noticed.

This might deserve it's own issue, but I wanted to add that I've experienced this as well. When opening .hs files outside of the current project, a new instance of the haskell-language-server process is launched. This second instance uses the current projects cradle, so it inevitably fails to do anything useful, but it does persist and it's resource usage continues to mirror that of the initial instance. This is effectively doubling the cpu/memory usage of hls and obviously more if I continue opening external files.

Additionally, the extra processes are not removed upon using the "Restart Haskell LSP server" command. It takes a "Reload Window" command to get back to a single instance of hls.

It's an interesting question how hls should behave in this situation, but burning through extra resources for no gain is probably not it.

[jneira]

This might deserve it's own issue, but I wanted to add that I've experienced this as well.

I have experienced it too just today, please open it, although maybe should belong to the vscode extension issue tracker

[Vlix]

@Friede80 Do you also get like double suggestions for e.g. imports or type signatures?

[Friede80]

@Friede80 Do you also get like double suggestions for e.g. imports or type signatures?

Yes. I get double errors/warnings reported and double hover docs too.

[jneira]

@cptwunderlich @Vlix and other participants: related only with performance, hls 1.5.0 includes several improvements about, could you check if memory usage is improved also for you?

[Vlix]

I'm now getting the following error on 2 files in a dependency of the library I'm working in.

tried to look at linkable for GetModIfaceWithoutLinkable for NormalizedFilePath "<manually removed>"
CallStack (from HasCallStack):
  error, called at src/Development/IDE/Core/Rules.hs:888:79 in ghcide-1.5.0-24rvFpA7X9i5SYHmUHLBtC:Development.IDE.Core.Rules

I've built HLS with -dynamic flags again, like always.

For one of the files, the module pointed to in the NormalizedFilePath is an import, for the other, it is not. (though the error's path is the same for both)
If I open the one where it isn't, and then close it, the error disappears most of the time. In the module where it is an import it stays, but it seems everything else does work (hovering, import suggestions, etc.) I'll see if I notice any memory usage difference.

[pepeiborra]

This is due to a regression in 1.5.0, fixed in #2380

[Vlix]

Does it have any impact on performance or anything else? Or is it just an annoying error that I should ignore?

[jneira]

No performance impact afaik, the error will be fixed in a minor bug fix release

[Vlix]

The GetModIfaceWithoutLinkable error is pretty annoying now and does seem to hamper IDE functionality in the modules where it occurs, which is a bit meh.
Haven't noticed any bad memory usage, so far, so I can at least tell you it's not gotten worse 👍 We'll see if the smoothness persists.

[jneira]

@cptwunderlich @Vlix hi, i am gonna close it optimistically, feel free to reopen or open a new one if you continue experiencing memory usage issues with newer versions please