Fixed out of memory issue in extractFromEntries

[chuntley]

When dealing with a large data set of 200k large documents, Gatsby runs out of memory in the extractFromEntries function in data-tree-utlis.js

Before the fix:

success source and transform nodes — 261.101 s
error UNHANDLED REJECTION

  RangeError: Maximum call stack size exceeded

  - data-tree-utils.js:140 extractFromEntries
    [website]/[gatsby]/dist/schema/data-tree-utils.js:140:34

  - data-tree-utils.js:239 extractFieldExamples
    [website]/[gatsby]/dist/schema/data-tree-utils.js:239:19

  - data-tree-utils.js:281 getExampleValues
    [website]/[gatsby]/dist/schema/data-tree-utils.js:281:26

  - infer-graphql-input-fields.js:310 inferInputObjectStructureFromNodes
    [website]/[gatsby]/dist/schema/infer-graphql-input-fields.js:310:20

  - build-node-types.js:186
    [website]/[gatsby]/dist/schema/build-node-types.js:186:36

  - Generator.next

  - next_tick.js:131 _combinedTickCallback
    internal/process/next_tick.js:131:7

  - next_tick.js:180 process._tickCallback
    internal/process/next_tick.js:180:9

By replacing [].concat(...values) with _.flatten(values), it will use far less memory and get past this point in the build process, while achieving the exact same result.

Another OOM issue happens much further down the line after the info bootstrap finished - 353.691 s message, but that is likely a memory leak to track down in another ticket.

[KyleAMathews]

Deploy preview for gatsbyjs failed.

Built with commit 44a5483

https://app.netlify.com/sites/gatsbyjs/deploys/5b5ab7fd82d3f157fba06050

[KyleAMathews]

Deploy preview for using-postcss-sass failed.

Built with commit 44a5483

https://app.netlify.com/sites/using-postcss-sass/deploys/5b5ab7fe82d3f157fba06058

[gatsbybot]

Deploy preview for using-drupal ready!

Built with commit 44a5483

https://deploy-preview-6797--using-drupal.netlify.com

[gatsbybot]

Deploy preview for gatsbygram ready!

Built with commit 44a5483

https://deploy-preview-6797--gatsbygram.netlify.com

[pieh]

LGTM

[pieh]

Another OOM issue happens much further down the line after the info bootstrap finished - 353.691 s message, but that is likely a memory leak to track down in another ticket.

In this PR #6636 we moved saving state to after bootstrap is complete, so my first guess this is same issue that was reported in #6611 (JSON.stringify for large objects will eat a lot of memory). To verify this, You could try commenting out lines:

gatsby/packages/gatsby/src/redux/index.js

Line 96 in a8a8829

`nodes`,

and

gatsby/packages/gatsby/src/redux/index.js

Line 108 in a8a8829

pickedState.nodes = mapToObject(pickedState.nodes)

If this would allow build to complete (or even go further) then this is potential solution to this - #6656