Are results non-deterministic?

[ozum]

Hi,

I looked in docs and issues, but could not find an answer. Order of returned files of globby with same directory structure and and same glob patterns change from execution to execution.

In one of my projects, some tests depend on order of returned files, and those tests fail:

90% of time (just observation, not instrumented)

a/a
b/b1
b/b2

vs.

10% of time

b/b1
b/b2
a/a

Is this expected behavior?

Thanks,

[sindresorhus]

Yeah, order is not guaranteed as the underlying package used for globbing fast-glob doesn't guarantee the order:

meanwhile results are returned in arbitrary order. Quick, simple, effective. - https://github.com/mrmlnc/fast-glob

We could maybe sort the results, but it's not clear what downsides that might have.

@mrmlnc Is there a reason you're not just sorting the resulting array before returning it?

[mrmlnc]

Is there a reason you're not just sorting the resulting array before returning it?

The reason for the non-deterministic order is parallelism when working with the file system. The fast-glob package uses a width-first search algorithm (we trying to read all the directories in the directory in parallel). The node-glob package uses a depth-first search algorithm.

I have several reasons for this:

0. Sorting is not a quick process (the goal of my package is to provide results as quickly as possible).
1. The fast-glob has a Stream API and we can't control the order there.

You can just sort the results if required without any problems.

[ozum]

Thanks for the clarification. @sindresorhus, it may be helpful (for others) to write in README that results are non-deterministic. I assumed (my mistake) results are ordered until some of my tests failed very lately.

As you indicated in your answer fast-glob states this situation, but usually people don't read docs of dependencies of dependencies.

By the way, also @mrmlnc many thanks for fast-glob and @sindresorhus for globby.

[thorn0]

The reason for the non-deterministic order is parallelism when working with the file system.

@mrmlnc Does that apply only to the async API? Or does the return value of fastGlob.sync(...) have non-deterministic order too?

[mrmlnc]

The synchronous API must return results in a deterministic order. I know of only one case where order can be broken:

• fs.readdir — fs.readdir and fs.readdirSync file order is always sorted on Linux (but not Windows) nodejs/node#3232

JFYI: https://github.com/nodelib/nodelib/blob/8e0d8b889e282690bb5e2bfe2442f2f099438e22/packages/fs/fs.walk/src/readers/sync.ts#L7