Loading workspace in 2.6.0 became really slow

[FlashHit]

I have 3 workspaces in one project with 2500+ files each (because of libraries). Before this release it was loading the workspaces pretty fast. Now it is really really slow. I would say it became unusable for me. So rn I have to roll back to the previous release.

[ur4ltz]

I have exactly the same problem!

[sumneko]

Previously I would start a process per workspace, now I have one process for all workspaces.
With no duplicate files, you should now have around 3x the loading time.
If this is indeed the case, then I might need to consider loading files in multiple threads.
If the actual loading time is much more than 3 times the original, then there should be other bugs that need to be resolved as soon as possible.

[FoolHen]

Same here, it seems to be way more than 3 times. It usually takes less than a minute, now I've measured 12 minutes. That said, automatically detecting and processing all workspaces is a nice addition, having to open a file from each workspace before was a bit annoying. 👍

[sumneko]

Same here, it seems to be way more than 3 times. It usually takes less than a minute, now I've measured 12 minutes. That said, automatically detecting and processing all workspaces is a nice addition, having to open a file from each workspace before was a bit annoying. 👍

It does sound like there is a bug causing the loading speed to be too slow, and I'll take that as a priority.
Thanks for your feedback!

[FlashHit]

All 3 workspaces have a library that is the same. Is it now possible to load a library 1 time for all 3 workspaces? Not sure if this is something that could work but this would reduce the size of the files to cache a lot for me.

workspace 1 libraries: fb, shared, client
workspace 2 libraries: fb, shared
workspace 3 libraries: fb, shared, server

it’s looks a bit like this now and it’s the fb library that has over 2000 files.

[sumneko]

Please check if this version works: https://github.com/sumneko/lua-language-server/actions/runs/1711860828

[Sedas25]

@sumneko that version is much faster

[FlashHit]

Still slow for me

[sumneko]

Still slow for me

Please provide your logs, see https://github.com/sumneko/lua-language-server/wiki/Default-log-path

[FoolHen]

@sumneko Takes longer for me than 2.6.0, took around 30 min. Diagnosing workspace is what takes most of it. Loading workspace was around 4 min.
Log

[sumneko]

Sorry I've been busy these days, this issue may take next week to resolve

[sumneko]

Please check if this version works: https://github.com/sumneko/lua-language-server/actions/runs/1728033602

[FoolHen]

@sumneko Still the same issue, log. This time it took around 15 min.

[megalithic]

same.

[sumneko]

Please test this version: https://github.com/sumneko/lua-language-server/actions/runs/1730943446

[FoolHen]

@sumneko took 6 minutes now, better than before but not as good as 2.5.6 (<1min). Here's the log

[FoolHen]

Also I've noticed that the diagnosing isn't working correctly, it's not showing any of the warnings i get with the other versions in the problems tab

[sumneko]

@sumneko took 6 minutes now, better than before but not as good as 2.5.6 (<1min). Here's the log

I checked the logs carefully and the phenomenon is very strange. It only takes about 15 milliseconds to load a file at first. According to this speed, 3000 unique files should be loaded in 1 minute.
However, as the number of loaded files increases, it takes hundreds of milliseconds to load a file. It even appeared that parsing a file that was only 0.5kb took over 100ms.
Therefore, please confirm whether your computer's memory is exhausted during the loading process, resulting in frequent full gc.

[FlashHit]

If you like to you can check it yourself by installing the Vua extension that builds on top of your extension.
Just run the command: Download and build content
And then: Create New Project

just search for Vua in the vs code marketplace and you will find it.

[FoolHen]

Memory is fine, lua server continues rising its memory since it begins loading and goes up to around 500MB. I have 16GB of RAM and overall my pc uses around 7GB. As flashhit said, you can probably reproduce it yourself with the Vua plugin, I use it as well.

[sumneko]

Thanks for the project, it only took me a minute to find the problem:
I opened the project, attached it with the lua-debug, clicked pause, and the code just paused in a giant cross-comparison, so I immediately understood the problem.

Please test this version, it should be completely fixed this time: https://github.com/sumneko/lua-language-server/actions/runs/1734161110

[FoolHen]

@sumneko Awesome that fixed it 😄 altho the diagnostics aren't working (not getting any errors in the problems tab), not sure if that's intended with that version.

[FlashHit]

ye the issue is fixed but I would say compared to 2.5.6 it's still slower.

[sumneko]

@sumneko Awesome that fixed it 😄 altho the diagnostics aren't working (not getting any errors in the problems tab), not sure if that's intended with that version.

I cannot reproduce this, please provide your log

[FoolHen]

log

[sumneko]

@FoolHen Are there no diagnostics in the window and tab, or is it just not displayed in the tab?

Please check your tab config.

[FoolHen]

Errors, warnings and info are enabled, they just dont show up. In previous versions (including 2.6.0) it shows the warnings that my project has, but not with this version you sent
https://i.imgur.com/1Z5pGMq.png

[sumneko]

I add some logs in this version: https://github.com/sumneko/lua-language-server/actions/runs/1735579998
Please provide your log using this version.

[FoolHen]

log

[sumneko]

ye the issue is fixed but I would say compared to 2.5.6 it's still slower.

https://github.com/sumneko/lua-language-server/actions/runs/1735671951
I think this version should be as fast as 2.5.6

[sumneko]

log

From your log, I can see it is indeed pushed 2 diag results to Client/__init__.lua.
What warnings are you expected?

EDIT: It would be nice to provide me with a test project and explicitly tell me what type of warning is missing in which file and where.

[FoolHen]

From your log, I can see it is indeed pushed 2 diag results to Client/init.lua.

There weren't any problems in the tab. The warning I was expecting are on the Shared folder, seems like errors in Client and Server folder work fine, but any error in shared doesn't show. I'll provide you with a project in a bit.

[sumneko]

The warning I was expecting are on the Shared folder, seems like errors in Client and Server folder work fine, but any error in shared doesn't show.

Thank you, I can reproduce it now.

[FoolHen]

TestVS.zip In this project the shared/__init__.lua file should have many errors.

[sumneko]

@FoolHen should be fixed: https://github.com/sumneko/lua-language-server/actions/runs/1735759774

[FoolHen]

Yep that fixed it. It's also back to how fast it was in 2.5.6. Thanks!

[ur4ltz]

The workspace is loaded only at 98%. :-(
nvim_log.zip

[sumneko]

Workspace load only 98% nvim_log.zip

Check this version: https://github.com/sumneko/lua-language-server/actions/runs/1736040553

[ur4ltz]

OK. 100%

[FlashHit]

ye this version is pretty good.
The only thing I could say something about is that there is a 5 seconds delay.
See time: [17:34:49.467] and [17:34:54.435]
It seems like it loads ~90 files and then it stops for 5 seconds before loading the rest.
log

[FlashHit]

I think this video shows another bug as well. probably not related to this issue, but it also shows the loading time it takes
https://streamable.com/cl047g

The other bug is at the end of the video. You see that it doesn't show the warnings in KitHelper until I modify the file.
Let me know if I should open a separate issue for that.

[FoolHen]

My log, in case it's useful.

[sumneko]

I think this video shows another bug as well. probably not related to this issue, but it also shows the loading time it takes https://streamable.com/cl047g

The other bug is at the end of the video. You see that it doesn't show the warnings in KitHelper until I modify the file. Let me know if I should open a separate issue for that.

Let's move to a new issue

[FlashHit]

ye this version is pretty good.
The only thing I could say something about is that there is a 5 seconds delay.
See time: [17:34:49.467] and [17:34:54.435]
It seems like it loads ~90 files and then it stops for 5 seconds before loading the rest.
log

What about this?

[sumneko]

What about this?

Can you reproduce it stably? I have never reproduced this problem with empty Vua project.

[FlashHit]

What about this?

Can you reproduce it stably? I have never reproduced this problem with empty Vua project.

In the project that I used I had it every time.

[sumneko]

Would it be convenient to send me your entire project?
You can send it to my email.

[FlashHit]

Let me check if I get it with a blank project as well first

[FlashHit]

Oh I see why. It seems that I have a WebUI folder with a bunch of stuff thats not needed for Lua. Like a folder with node_modules that is causing this delay.

[sumneko]

I can do some optimizations later and warn if scanning some folders takes too long.

[FlashHit]

So it looks like it works fine if I enable the Ignore .gitignore setting in the user section. I already had it enabled in the workspace section but somehow there it was still causing this delay.
Could this be an issue with the Ignore .gitignore setting? I mean shouldn't it work if I enable it just in the workspace?

[sumneko]

So it looks like it works fine if I enable the Ignore .gitignore setting in the user section. I already had it enabled in the workspace section but somehow there it was still causing this delay. Could this be an issue with the Ignore .gitignore setting? I mean shouldn't it work if I enable it just in the workspace?

It seems like a bug, let's open a new issue about it