%200 CPU usage with tons of flow processes #201

oguzbilgic · 2015-09-18T21:17:29Z

nuclide-flow is installed but disabled in my atom env. every time there is an update it gets enabled automatically by atom. As soon as it is enabled, It starts 10-15 flow processes and crashes my 2015 macbook pro. I just uninstalled nuclide-flow and flow.

pinguinjkeke · 2015-09-20T11:47:40Z

Same here

h3l · 2015-09-21T10:13:54Z

Same here

nmote · 2015-09-21T23:57:41Z

Sorry about that. Do you have .flowconfig files in the projects you are editing? If not, Flow shouldn't run at all. If you do, I would expect 10-15 Flow processes (Flow starts worker processes), but I wouldn't expect that kind of CPU usage, let alone a crashed MBP. I use this every day to develop Nuclide and don't experience anything like that.

What happens when you navigate to that project and run flow server from the command line? If causes the same problem, file an issue with Flow. If not, it's a problem with Nuclide.

I do have some work planned to reduce the number of requests we send to Flow. Currently we spawn quite a few more processes than we need in certain instances, which could be contributing to the problems here.

Could you also share some details about the projects you're working on? Even if they are very large, I wouldn't expect this sort of problem. If they are open-source, though, I could try to reproduce on that specific codebase. Also, what version of Flow and Nuclide do you have?

h3l · 2015-09-22T05:46:35Z

Sorry, I found I didn't install flow, after I brew install flow, everything is OK.
thx

nmote · 2015-09-22T15:59:58Z

@h3l To confirm, your issue was just that Flow wasn't working, correct? I find it hard to believe that you had Flow processes eating up all your CPU and crashing your computer if you never installed Flow.

oguzbilgic · 2015-09-22T16:08:03Z

I had flow installed for sure, and since I don't use it, I didn't have flowconfig in the project root.

h3l · 2015-09-23T03:11:05Z

I just have a test, after I use cmd + q to quit Atom totally, and use Atom to open the project, Atom will be stuck for a few minutes. The images is below.

and the ps aux get Atom state is below

xxx 55477 105.2 6.5 4448880 1083564 ?? R 11:02上午 1:53.71 /Users/xxx/Downloads/Atom.app/Contents/Frameworks/Atom Helper.app/Contents/MacOS/Atom Helper --type=renderer --js-flags=--har

Atom really use a lot of CPU and memory!!!!!

nmote · 2015-09-23T14:22:50Z

@oguzbilgic Could you provide the other details I requested?

@h3l That is a separate issue, please see #199.

kagha · 2015-10-06T14:22:27Z

I believe I am also experiencing the same issue. I have been using Atom off and on for a couple months and now that I am dabbling in React Native development, I installed the Nuclide packages as instructed by the official page.

Result:
I see several (10 to 15) flow processes at 0% with atom at 100 to 150 percent cpu usage for 5 minutes every time I start Atom/Nuclide. During this time the editor is unusable and OS X often asks me if I want to keep waiting for it to unblock or kill it.

Checking for updates to my Atom/Nuclide packages is on the order of 10+ minutes of the same.

I have a very fast OS X machine with an SSD.

EDIT: Uninstalling all nuclide-* packages from atom has stopped this issue from occurring in my case.

nmote · 2015-10-06T16:37:25Z

@kagha That is a separate issue, please see #199. It is expected to have 10-15 Flow processes idling while you are working. Flow keeps a server running so that it can quickly answer queries, and that server spawns some worker processes.

kagha · 2015-10-06T19:52:59Z

@nmote You are right. My apologies.

mostafaeweda · 2016-01-14T22:55:53Z

Nuclide's high CPU issue should have been addressed with the UP release, see: http://nuclide.io/blog/2016/01/13/Nuclide-v0.111.0-The-Unified-Package/ - feel free to reopen if still applies.

alvaromb · 2016-01-26T15:48:02Z

This is the CPU usage with Flow enabled.

This happens when I save some /* @flow */ headed file.

viatsko · 2016-01-26T15:49:16Z

Anything strange in console.log? Flow version mismatch? Do you have .flowconfig?

alvaromb · 2016-01-26T15:56:13Z

Nothing strange. I do have a .flowconfig file and flow version matches on 0.20.1. Console output:

Happens since the Nuclide unified package update ¯_(ツ)_/¯

alvaromb · 2016-01-26T16:20:54Z

Console also shows this, not sure if related:

[2016-01-26T16:20:05.186Z] [ERROR] nuclide - Problem parsing type hint: Kind AnyFunT not supported

alvaromb · 2016-01-27T10:06:11Z

Here's another capture from console.

Let me know if I can do something to help you guys with this issue.

nmote · 2016-01-27T17:02:07Z

[2016-01-26T16:20:05.186Z] [ERROR] nuclide - Problem parsing type hint: Kind AnyFunT not supported

That's unrelated -- it's due to the experimental "enable structured type hints" setting being checked. I haven't implemented structured type hints for all types yet (one of the reasons it's labeled experimental) so for some it will log these errors. The UI for that was either just released or will be in the next release (don't remember off hand), so some types will be expandable trees.

I think the best thing you could do is get the output of ps -eaf | grep flow when you're experiencing this issue.

To be clear, it's expected that Flow will consume a lot of CPU when it first starts up. On the Nuclide codebase it takes in the ballpark of 30 seconds. Then, whenever you save it has to recompute some information. In large projects I've seen this take as long as maybe 5 seconds. However it should only be the Flow worker processes consuming any significant amount of CPU (10-15 of them, maybe). We've seen a few cases where the Flow client processes also consume CPU which is weird and can lead to your system locking up. The ps command above should help us figure out if that's what's happening.

alex-wilmer · 2016-10-19T17:41:16Z

As soon as I open a large file (like something built via webpack) Nuclide takes over my entire machine and nothing seems to drop down the memory usage short of killing nuclide and restarting. Ouch!

nmote · 2016-10-20T16:05:33Z

That memory usage is absurd. If you can reproduce please report on the Flow repository.

iddan · 2017-02-06T11:49:38Z

Happens to me too. Both in Webstorm and VSCode on my MacBook Pro once I enable flow the CPU rockets to 200% usage
Is it has to do with my .flowconfig-s? Should it ignore node_modules to prevent those kind of situations?
If so, can you prevent this boilerplate situation?

Arual90 · 2017-02-23T16:10:17Z

Same problem

MihaiDamian · 2017-03-15T15:00:13Z

Make sure all node modules are declared as libraries. This definitely has an impact on CPU usage:

[libs]
node_modules/*

rajat1saxena · 2017-10-16T03:28:50Z

Flow processes brought my laptop down on its knees.
Config:
OS: Ubuntu 16.04
Processor: i5 4th Gen

See screenshots:

csardinha · 2017-10-19T11:12:33Z

Same problem.
OS: macOS Sierra
Processor: 2.2 GHz Intel Core i7

byoigres · 2017-10-24T22:09:32Z

Same here:

OS: macOS 10.13 High Sierra
atom version: 1.22.0-beta1 x64
project: react-native init default

abourget · 2017-11-03T18:22:37Z

Same here on Linux, with emacs though.

mostafaeweda · 2017-11-03T21:21:47Z

@abourget what version of flow are you running?

mostafaeweda · 2017-11-17T23:01:25Z

That should be fixed in the latest nuclide release

Dakuan · 2017-11-27T07:37:30Z

@mostafaeweda I'm also seeing this, but I'm a VS Code user. Is it an issue with Flow? Did anyone else find a fix?

jamsch · 2018-01-16T03:30:16Z

Just updated Nuclide today on my Linux machine and came across the same issue. Heaps of flow processes spawned as soon as I open any file and then freezes. Any file I open seems to spawn 8 worker processes and then freezes. Running this on an i7 7700 / 16GB RAM machine. After taking this screenshot, my system froze for a good two minutes. Not experiencing the issue with VSCode.

Running flow@0.61.0

theweekendgeek · 2018-02-04T00:04:03Z

I'm using vscode on Windows 10 and i have noticed that even 15 - 20 minutes after closing the editor there still several processes of flow that consume cpu and don't seem to stop running. I've restarted, tried an alternative extension for flow in the marketplace, but to no avail.

I don't seem to have problems with memory though.

jony89 · 2018-02-07T15:15:09Z

This might help, if u didn't try already, add to your .flowconfig file :

[ignore]
.*/node_modules/.*

although this will also exclude flow types of installed packages and might break some rules

Zdend · 2018-04-23T03:37:24Z

I'm facing the same issue which makes the whole mac unusable (not just my VS Code) because I run out of available processes and so the only workaround is to run pkill flow which helps until the processes accumulate again. The problem is that I don't know when does it happen because I can code days without this problem and then suddenly things stop working because flow consumed every available process.

Ignoring node_modules results in breaking every import statement which is not a solution either.

Sample ps aux | grep flow out of about 450 spawned flows

zdv              37384   0.0  0.1  3079876  12392   ??  S    Fri04pm   0:00.39 node /usr/local/bin/flow force-recheck /Users/project/components/standard-layout.js --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37378   0.0  0.0  2488648   2856   ??  S    Fri04pm   0:00.02 /usr/local/share/.config/yarn/global/node_modules/flow-bin/flow-osx-v0.67.1/flow autocomplete --json /Users/project/components/standard-layout.js --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37376   0.0  0.0  2489672   2856   ??  S    Fri04pm   0:00.02 /usr/local/share/.config/yarn/global/node_modules/flow-bin/flow-osx-v0.67.1/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 12 --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37375   0.0  0.1  3079876  12388   ??  S    Fri04pm   0:00.35 node /usr/local/bin/flow autocomplete --json /Users/project/components/standard-layout.js --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37373   0.0  0.1  3079876  12388   ??  S    Fri04pm   0:00.31 node /usr/local/bin/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 12 --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37370   0.0  0.0  2469192   2852   ??  S    Fri04pm   0:00.02 /usr/local/share/.config/yarn/global/node_modules/flow-bin/flow-osx-v0.67.1/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 22 --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37369   0.0  0.0  2461000   2852   ??  S    Fri04pm   0:00.02 /usr/local/share/.config/yarn/global/node_modules/flow-bin/flow-osx-v0.67.1/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 21 --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37367   0.0  0.1  3079876  12388   ??  S    Fri04pm   0:00.37 node /usr/local/bin/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 22 --retry-if-init false --retries 0 --no-auto-start --from nuclide

zdv              37365   0.0  0.1  3079876  12388   ??  S    Fri04pm   0:00.41 node /usr/local/bin/flow type-at-pos --json --path /Users/project/components/standard-layout.js 8 21 --retry-if-init false --retries 0 --no-auto-start --from nuclide

jony89 · 2018-04-23T06:22:29Z

@Zdend

Ignoring node_modules results in breaking every import statement which is not a solution either.

Not the best solution but u may specify in flow that will not read all files, only those with // @flow on top of them. this way u will be able to include the node_modules. but again, u will have to add this `// @flow

configuration info https://flow.org/en/docs/config/options/#toc-all-boolean

either way this should be fixed.

jamsch · 2018-04-23T22:05:59Z

Seems to be a Flow issue and it still persists on 0.70.0. Really considering migrating a large project over to TypeScript because this is now a daily occurrence for us.

cglacet · 2018-05-03T07:57:04Z

Hmm same problem here, my computer is about to explode (flow:0.64.0 on osx 10.12.6 (16G29)).

PID    %CPU  %MEM    VSZ    RSS   TT  STAT   STARTED   TIME   COMMAND
10753  31.6  2.6  3069848 428772   ??  R     9:33AM   5:05.18 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10867  31.0  2.9  3069848 493084   ??  R     9:33AM   5:02.03 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
12121  30.7  1.2  2726192 205376   ??  R     9:41AM   1:22.20 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
12275  30.4  1.3  2726192 211672   ??  R     9:42AM   1:05.40 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11215  30.4  2.2  2882340 368396   ??  R     9:35AM   4:04.83 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11694  30.4  1.4  2756976 241348   ??  R     9:39AM   1:59.13 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11830  30.3  1.3  2726192 211368   ??  R     9:40AM   1:50.06 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11139  30.3  3.3  3069876 546052   ??  R     9:34AM   4:27.16 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11269  30.3  2.2  2882340 368328   ??  R     9:35AM   4:02.10 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11890  30.0  1.2  2726204 206416   ??  R     9:40AM   1:47.24 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11516  29.8  1.4  2756976 237712   ??  R     9:38AM   2:35.54 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10908  29.7  2.9  3069848 493216   ??  R     9:34AM   5:01.59 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11960  29.4  1.3  2726192 211068   ??  R     9:40AM   1:39.47 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11325  29.4  2.2  2882340 368012   ??  R     9:35AM   4:00.12 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11046  29.2  3.3  3069848 554068   ??  R     9:34AM   4:59.10 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11603  29.1  1.4  2756976 238100   ??  R     9:38AM   2:18.07 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11753  28.9  1.4  2757088 241244   ??  R     9:39AM   1:56.87 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10755  28.8  2.5  3069848 427476   ??  R     9:33AM   5:04.83 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10707  28.8  2.5  3069848 415876   ??  R     9:32AM   5:05.49 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
12081  28.8  1.1  2757088 181324   ??  R     9:41AM   1:28.94 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10982  28.5  3.3  3069848 553400   ??  R     9:34AM   4:59.86 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
12194  28.4  1.2  2726192 209436   ??  R     9:41AM   1:15.54 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
10800  28.1  2.5  3069848 427332   ??  R     9:33AM   5:03.34 flow-bin/flow-osx-v0.64.0/flow status --json some_other.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
11441  28.0  1.9  2835376 323400   ??  R     9:37AM   2:45.16 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide
12002  28.0  1.2  2726192 209076   ??  R     9:40AM   1:38.26 flow-bin/flow-osx-v0.64.0/flow status --json places.js --retry-if-init false --retries 0 --no-auto-start --from nuclide

... More processes but with way less CPU usage.

If I kill these processes, at some point some similar will show up again (involving the same files).

for id in $(ps aux | grep flow | awk '{if ($3 > 2) print $2}'); do kill -9 $id; done

I don't know if this is related but one of my .jsfile is pretty big (not one of those listed above).

Edit: It looks like if I turn flow off for this big file (~6.5M), everything works just fine. This big file only contains a dictionary (something that you would normally load from a .json or from a database, here it's just a plain .js file with a flow typed dictionary declaration). I guess the size of the file is what causes the problem because it is extremely simple, it just looks like this:

const unsortedCityNames:Map<string, CityDetails> = new Map([ ..., 
	["bordeaux", {asciiname : "bordeaux",alternatenames : "bod,bordeaux,bordele,bordeos,bordeu,bordeus,bordo,bordox,bordozo,bordèu,bordéus,bordò,bordôx,bornto,bourdel,burdeos,burdeus,burdigala,gorad bardo,bo er duo,boleudo,borado,bordo,bordu,borudo,bwrdw,bwrڈw,bxr do,porto,μπορντό,бордо,горад бардо,բորդո,בארדא,בורדו,بوردو,بورڈو,बोर्दू,बोर्दो,ਬੋਰਦੋ,பொர்தோ,บอร์โด,ບອກໂດ,ბორდო,ቦርዶ,ボルドー,波尔多,波爾多,보르도",latitude : 44.84044,longitude : -0.5805,alpha2 : "fr",population : 231844,}],
	...,
	// ~24k cities later
]);
export type CityDetails = {
	geonameid?: number,
	asciiname: string,
	alternatenames: string,
	latitude : number,
	longitude : number,
	feature_class? : string,
	feature_code? : string,
	alpha2 : string,
	cc2? : string,
	admin1_code? : string,
	admin2_code? : string,
	admin3_code? : string,
	admin4_code? : string,
	population : number,
	elevation : number,
	dem? : number,
	timezone? : string,
	modification_date? : string
};

I'll try to load that from a .json, turn flow on, and see if the CPU problem still shows up.

arnarthor mentioned this issue Sep 30, 2015

100%+ CPU usage in Nuclide 0.0.32 #199

Closed

nmote added the more-information-needed label Oct 6, 2015

mostafaeweda closed this as completed Jan 14, 2016

mostafaeweda assigned nmote Jan 27, 2016

mostafaeweda reopened this Jan 27, 2016

mostafaeweda closed this as completed Nov 17, 2017

jony89 mentioned this issue Feb 7, 2018

Hundrets of flow processes flow/flow-for-vscode#172

Closed

facebook-github-bot unassigned nmote Dec 12, 2018

%200 CPU usage with tons of flow processes #201

%200 CPU usage with tons of flow processes #201

Comments

oguzbilgic commented Sep 18, 2015

pinguinjkeke commented Sep 20, 2015

h3l commented Sep 21, 2015

nmote commented Sep 21, 2015

h3l commented Sep 22, 2015

nmote commented Sep 22, 2015

oguzbilgic commented Sep 22, 2015

h3l commented Sep 23, 2015

nmote commented Sep 23, 2015

kagha commented Oct 6, 2015

nmote commented Oct 6, 2015

kagha commented Oct 6, 2015

mostafaeweda commented Jan 14, 2016

alvaromb commented Jan 26, 2016

viatsko commented Jan 26, 2016

alvaromb commented Jan 26, 2016

alvaromb commented Jan 26, 2016

alvaromb commented Jan 27, 2016

nmote commented Jan 27, 2016

alex-wilmer commented Oct 19, 2016

nmote commented Oct 20, 2016

iddan commented Feb 6, 2017

Arual90 commented Feb 23, 2017

MihaiDamian commented Mar 15, 2017 • edited Loading

rajat1saxena commented Oct 16, 2017

csardinha commented Oct 19, 2017

byoigres commented Oct 24, 2017

abourget commented Nov 3, 2017

mostafaeweda commented Nov 3, 2017

mostafaeweda commented Nov 17, 2017

Dakuan commented Nov 27, 2017

jamsch commented Jan 16, 2018 • edited Loading

theweekendgeek commented Feb 4, 2018 • edited Loading

jony89 commented Feb 7, 2018 • edited Loading

Zdend commented Apr 23, 2018 • edited Loading

jony89 commented Apr 23, 2018 • edited Loading

jamsch commented Apr 23, 2018

cglacet commented May 3, 2018 • edited Loading

MihaiDamian commented Mar 15, 2017 •

edited

Loading

jamsch commented Jan 16, 2018 •

edited

Loading

theweekendgeek commented Feb 4, 2018 •

edited

Loading

jony89 commented Feb 7, 2018 •

edited

Loading

Zdend commented Apr 23, 2018 •

edited

Loading

jony89 commented Apr 23, 2018 •

edited

Loading

cglacet commented May 3, 2018 •

edited

Loading