Heuristics of ignore::WalkParallel's threads field: hardcoded to 2?

[alexpovel]

Please tick this box to confirm you have reviewed the above.

• I have a different issue.

What version of ripgrep are you using?

ignore = "0.4.22"

How did you install ripgrep?

n/a

What operating system are you using ripgrep on?

n/a

Describe your bug.

I'm using WalkParallel for, well, parallel, recursive directory traversal. There's a threads parameter which I left unspecified (using WalkBuilder):

ripgrep/crates/ignore/src/walk.rs

Line 1196 in 71d71d2

threads: usize,

The docs say if left unspecified, aka at 0, the number of threads will be picked based on heuristics:

ripgrep/crates/ignore/src/walk.rs

Lines 643 to 644 in 71d71d2

	/// The default setting is `0`, which chooses the number of threads
	/// automatically using heuristics.

What are the steps to reproduce the behavior?

Execute

ignore::WalkBuilder::new(&some_path)
        .build_parallel()
        .run(todo!())

What is the actual behavior?

Using these settings, I noticed my application using 200% CPU, where more cores would have been available.

What is the expected behavior?

Processing to use more threads, as determined by the heuristics.

It turns out, threads is hard-coded to 2 if unspecified:

ripgrep/crates/ignore/src/walk.rs

Lines 1307 to 1308 in 71d71d2

	if self.threads == 0 {
	2

where for the 'heuristic', I expected something along the lines of num_cpus. I can confirm setting threads to num_cpus::get(), 12 in my case, yields results as expected.

Could/should this be used as the heuristic? It is a substantial, free win in my case.

If not, then the docs should probably be adjusted.

[BurntSushi]

It is still technically a heuristic, although not a very good one. :-)

I think I'd be okay if you copied ripgrep's heuristic to ignore:

ripgrep/crates/core/flags/hiargs.rs

Line 172 in 71d71d2

std::thread::available_parallelism().map_or(1, |n| n.get()).min(12)

And use std, not an extra crate.

[alexpovel]

Thanks for the quick reply and pointer to ripgrep's implementation! I'll get onto that.