-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(gatsby): do not call and iterate getAllNodes(File) for each file #28891
Conversation
… this is what we ended up with and I do not wish to discuss this any further
…ed set of arrays containing nodes, not a flat array. Then me and TS had another argument and we seem cool now.
type nestedListOfStrings = Array<string | nestedListOfStrings> | ||
type nestedListOfNodes = Array<IGatsbyNode | nestedListOfNodes> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Funsies. But turns out the resulting nesting structure must match the input nesting structure. So if you have [a, [b, [c, d]]]
as input, then your output should be the same except the input strings are replaced with nodes.
This is a tad annoying since it means we can't "just" pass on a flat array of promises to Promise.all()
but so be it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I realise this is already merged, but types should be PascalCased
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should change your middle name to Makes it faster
Awesome work, @pvdz |
Before it would call
context.nodeModel.getAllNodes({ type: `File` })
(already a non-trivial operation on its own) and then filter that down to the one node it was looking for; the one that represents the file. This approach is not cached, super slow, and simply does not scale.This PR changes that approach to leverage the fast filters, which will generate a O(1) lookup cache on the first call. The rest of the calls will be O(1) lookups.
Tested this on a modification of
benchmarks/gabe-fs-markdown
which adds one image per page (randomly generated before the benchmark).This effect scales as the number of nodes goes up. This should apply to most things that relate to local files, and maybe even beyond that.
This is a massive win for local file usage
just sayin'