Intellisense really slow and taking lots of memory

[Skrypt]

Issue Description

While working on https://github.com/OrchardCMS/OrchardCore
The intellisense starts working only like 5 minutes after the working folder has been opened.
Also, Omnisharp memory usage indicates that I'm using 3 gb approximately while I'm just opening VS Code and not debugging anything. The memory usage seems to be related with loading document symbols for the intellisense to work. Omnisharp memory usage goes from 1 gb to 3 gb before I can see intellisence working.

I'm running this on a intel Core i7 with a NVME drive.

Will efforts done on this PR #3817 will fix the issue eventually?

Environment information

VSCode version: 1.49.1
C# Extension: 1.23.2

Dotnet Information

SDK .NET Core (refl�tant tous les global.json)�: Version: 5.0.100-rc.1.20452.10 Commit: 473d1b592e

Environnement d'ex�cution�:
OS Name: Windows
OS Version: 10.0.19042
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\5.0.100-rc.1.20452.10\

Host (useful for support):
Version: 5.0.0-rc.1.20451.14
Commit: 38017c3935

.NET SDKs installed:
3.1.100-preview1-014459 [C:\Program Files\dotnet\sdk]
3.1.402 [C:\Program Files\dotnet\sdk]
5.0.100-preview.8.20417.9 [C:\Program Files\dotnet\sdk]
5.0.100-rc.1.20452.10 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.0-rc.1.20451.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.0-rc.1.20451.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.0-rc.1.20452.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
https://aka.ms/dotnet-download

Visual Studio Code Extensions

Extension	Author	Version
csharp	ms-dotnettools	1.23.2
remote-wsl	ms-vscode-remote	0.44.5
vscode-docker	ms-azuretools	1.6.0

[jmarolf]

dupe of #3421

[Skrypt]

Thanks I agree it's a duplicate but any answer on my question would have been appreciated.

[jmarolf]

sorry @Skrypt, I missed your question. As with anything performance related it is unwise to speculate, but here I go.

In general, we should have memory usage scale linearly with the number of projects/documents. We will always load and evaluate each file to build up a cache so things like find-all-references work quickly. Because of this design I would always expect some degree of CPU and memory useage spikes when a project or workspace is first opened regardless of how we improve performance. In your case the number sound rather excessive and we should be able to provide completions even with a partially completed model. However I would not expect the general design of open workspace -> compute and cache results -> update results incrementally to change.

This issue #3817 is unrelated in my opionion. Once a everything is loaded, on each keystroke we send the entire contents of the text buffer to the omnisharp process. This has scalability issues but I would only expect them to show up after everything is loaded. If the file you are typing in is very large, then that may be the root cause but you are talking about it taking a long time to update after open. I assume things are not slow after that?

[Skrypt]

Once the Omnisharp process has completed processing everything works like expected (after 6 minutes). So, it's basically non-usable for us in this state. On the opposite it seems to work fine on Visual Studio Community edition so I'm moving from one editor to the other lately because both have their caveats.

In your case the number sound rather excessive and we should be able to provide completions even with a partially completed model

I expect memory usage to scale linearly too but I'm surprised that I get delayed results like this. Should this not try to build a cache of the things we ask manually for first? Or allow it to be asynchronous while it's building up.

Example : if I'm doing "go to definition" would it not be smarter to do a search in the documents and then try to build a cache for these first? I'm not sure how the intellisense works and how it's engineered I find this quite interesting. I would have thought that it was a simple Full Text like indexation of all the code files. But it's probably more than this.