Multiple hanging dir/find commands for large searches

[bpasero]

Testing #11035

• setup symlinks in a way that you end up having circular references for a large workspace (I was using the VS Code repository to have meaningful large data)
• use quick open to find a file
• watch the processes that get spawned
• due to Windows: circular symlinks protection lost #11179 we end up doing a lot more work than needed
• close quick open
• open quick open and run the same search again

=> you see more and more dir commands piling up (interestingly the second time you try it seems 2 new dir commands popup instead of one)
=> I would expect the command to be killed when another one starts

[chrmarti]

Updating title to reflect that this is also a problem with the other platforms.

[bpasero]

@chrmarti any advise how to reproduce this? i think it was easy on Windows with the circular symlinks, but now we do not have the dir command anymore on Windows.

[chrmarti]

On Linux/OSX opening/closing QuickOpen multiple time while find still runs would spawn an additional find process. The fix should prevent this and let the first find run to its end before spawning a new one when opening QuickOpen the next time.

[bpasero]

@chrmarti thanks

[bpasero]

@chrmarti I am still able to get 2 find processes running though:

Steps:

• large folder (I copied 5 copies of typescript sourcecode with node modules installed and explicitly chose to not ignore node modules folder in search.exclude setting)
• open window and quickly open quick open multiple times

This is on Mac OS btw.

[chrmarti]

@bpasero Cannot reproduce.

This would happen if the search process has terminated after the cache was populated. QuickOpen would first ask for a new cache to be populated and then start using the old cache (which is gone) which triggers another find.

One way for the search process to terminate is the timeout we've set to 1 hour. Any chance you hit that case? Does it reproduce on your machine?

[bpasero]

Yes, reproducible right on startup with insiders build:

- the workspace contains the typescript sources with installed node_modules 6 times - contents:  - my settings:

{
    "files.autoSave": "afterDelay",
    "editor.wrappingColumn": 3000,
    "editor.folding": false,
    "explorer.openEditors.visible": 0,
    "workbench.editor.showTabs": false,
    "workbench.editor.enablePreview": false,
    "git.enableLongCommitWarning": false,
    "window.openFilesInNewWindow": true,
    "workbench.quickOpen.closeOnFocusLost": true,
    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/node_modules/**": true
    }
}

[bpasero]

All I do is open/close quick open via Cmd+P in some random order.

[isidorn]

Leaving this for september, @chrmarti please investigate and decide if for september

[chrmarti]

@bpasero Might be an artifact from the Activity Monitor. I sometimes see two find listed there when ps only shows one or none. Could you compare what the Activity Monitor shows you with what ps shows: watch -n 1 'ps ax | grep find'

Moving milestone as there is currently no action planned. Thanks.

[chrmarti]

@bpasero Let me know if you get a chance to verify the above.

[bpasero]

@chrmarti yes there are multiple finds spawned:

~/Development/Microsoft/monaco> ps ax | grep find
  173   ??  Ss     0:00.53 /usr/libexec/findmydeviced
37682 s001  U+     0:02.89 find -L . -not ( ( -name .git -o -name .svn -o -name .hg -o -name .DS_Store -o -name bower_components ) -prune ) -type f
37683 s001  U+     0:02.78 find -L . -not ( ( -name .git -o -name .svn -o -name .hg -o -name .DS_Store -o -name bower_components ) -prune ) -type f
37684 s001  U+     0:02.19 find -L . -not ( ( -name .git -o -name .svn -o -name .hg -o -name .DS_Store -o -name bower_components ) -prune ) -type f
37686 s002  R+     0:00.00 grep find

Easier for me to reproduce:

• copy the full typescript sources 10 times (this is roughly 500.000 files with 4.6 GB size
• start an instance and open quick open without typing
• open quick open and start typing

=> maybe it is an issue that only shows when you open quick open once without typing and once with typing?

[chrmarti]

Good finding, the cached promise ended up being canceled.

For verification: With a sufficiently large folder it was easy to reproduce. E.g., open the folder containing all your projects, the home folder, the root folder. Make sure there is at most one find process. (Only OSX / Linux.)

[codedrift]

This bug is still present for me. Loading folders with many files (in my case loading the home folder with code .) leads to those find processes piling up.

find -L . -not ( ( -name .git -o -name .svn -o -name .hg -o -name .DS_Store ) -prune ) -type f

Every process uses ~100% of the assigned CPU core which eventually leads to the whole CPU being used.

My setup:
Version 1.8.1
Commit ee428b0
Date 2016-12-19T14:41:20.664Z
Shell 1.4.6
Renderer 53.0.2785.143
Node 6.5.0
Ubuntu 16.04 LTS

[bpasero]

@AlphaBetaPeter maybe better to report this as separate issue to have a fresh look on it 👍

[codedrift]

@bpasero will do 😉