A better way to compute the number of render threads?

[devernay]

By default, Natron uses as as many threads for rendering as there are available cores.

The problem is that Natron is also memory-intensive, so we should probably have an automatic setting that also depends on the available memory.

We get many bug reports of Natron running slow, and lowering the number of threads is often part of the solution.

We should probably have a formula like min(num_cores, memory/3.5GB), so that would be 4 threads max for a 16GB machine.

Any thoughts? Other ideas to compute the number of threads?

[Songtech-0912]

@devernay I did some experimentation into the ideal numbers of threads to use, based on Natron running the spaceship test scene. Since our goal is maximum performance, the ideal number of threads would be the number that is the middle-ground between system resource usage (which includes both memory and CPU usage) and render speed.

In the course of the experiment, I made sure to use the GUI version of Natron to render, rather than NatronRender, as most users reporting slow rendering are using the GUI version of Natron. Additionally, I made sure to have quite a few applications on during the rendering process, including two resource-intensive web browsers, because most people aren't going to be running Natron exclusively on their operating system. Last, I only rendered 48 frames from each project, because otherwise it would've taken forever to do.

Experimental values were derived from the following:

• Number of threads is the number of thread set in the Natron settings. As my machine only has 8 threads that number will vary from 1 to 8. This is the independent variable.
• User performance is measured from a scale of 1 to 10, with 10 being lightning-fast and 1 being completely unresponsive. This does not constitute an accurate representation of actual performance, but rather a reflection of how users may perceive faster/slower rendering (e.g. if Natron freezes midway but still renders quickly).
• Memory usage is the output of the command pid=$(pidof Natron) && top -b -n 5 -d 2 -p $pid | tail -1 | awk '{printf "%.0f%\n", $10}' (this basically gets the process ID of Natron, feeds it into top, which does the rest)
• CPU usage is the output of the command pid=$(pidof Natron) && top -b -n 5 -d 2 -p $pid | tail -1 | awk '{printf "%.0f%\n", $9}', which checks CPU usage every 2 seconds, and measures the average over 5 times (10 seconds)
• Render time is measured using, well, a stopwatch, for lack of any better method. For greatest consistency, all renders were directed to ~/Renders/render_# where #=[i for i in range(0, 8)]
• The performance score is defined as the geometric mean of user performance, the inverse reciprocal of memory usage, and the inverse reciprocal of CPU usage - that is, the cube root of the product of the three values. Higher performance scores are better.
• The speed score is defined as an index between 0 and 1 using the equation . Higher values are better as well.

Here is the raw data of the experiment (favored values in bold):

Number of threads	User performance (/10)	Memory usage (%, average amount)	CPU usage (%)**	Render time (s)	Performance Score	Speed score
1	5	15% (2.4 GB)	74%	100	9.61	0.07
2	6	16-17% (2.64 GB)	84%	101	11.0	0.00
3	6	16% (2.56 GB)	64%	95	9.94	0.43
4	4	16% (2.56 GB)	83%	94	9.47	0.50
5	5	14-16% (2.4 GB)	74%	101	9.61	0.00
6	4	14% (2.24 GB)	88%	92	9.23	0.64
7	6	9-10% (1.52 GB)	93%	99	9.46	0.14
8	8	9-12% (1.68 GB)	> 100%*	87	11.0	1.00

*: CPU usage exceeded amount measurable by top

**: It was almost always the case that CPU would start at 40% or so, jump to 90% during the middle of the render, and then only to fall back again, so these numbers are rough averages between the minimum and maximum

Each setting of number of threads are either more fast (in terms of render time) or more performant (in terms of using less system resources). We want to find each score's relative average between performance and speed, and to do so I took the mean of the performance score and the speed score. I call this the usability index (higher values are better). Additionally, I also took the mean of the absolute difference of the usability index to the performance score and to the speed score, which I call the skew index (lower values are better). Below are the results:

Number of threads	Usability index	Skew index
1	4.84	4.77
2	5.50	5.50
3	5.185	4.76
4	4.99	4.49
5	4.81	4.81
6	4.94	4.30
7	4.81	4.66
8	5.50	5.50

If we finally find the dividend (by subtracting the skew index from 50% of the usability index), we arrive at a final experience score, which measures the value that offers the best of performance and system resources while not sacrificing either too heavily. Here are the experience scores for all thread numbers:

Number of threads	Experience score
1	2.46
2	2.75
3	2.80
4	2.75
5	2.40
6	2.79
7	2.48
8	2.75

Based on these results, the ideal number of threads to use on my computer, at least, would be 6 threads. Your equation would give a result of just 4 threads, so perhaps a better equation would be min(num_cores, memory/2.5)

So, based on this research, I have 3 suggestions:

• Run some sort of benchmark right after Natron runs for the first time after installation, which automatically finds the ideal number of threads based on the experimental process I did
• Use the other equation I proposed
• Run mass benchmarks via a community effort, then use that to find a default based on community benchmark data

I would like to know your thoughts on this.

[devernay]

Nice!
did you activate "turbo" mode from the Gui to do your renders?

Does your CPU have 4 cores / 8 threads?
It seems like using 4 threads gives slightly slower render time (94s vs 92s), so it seems like there is a sweet spot there too, what do you think?

[Songtech-0912]

Did you activate "turbo" mode from the Gui to do your renders?

Nope, in fact I've mostly ignored the button's existence until you've brought that up! There will definitely be a difference that way.

Does your CPU have 4 cores / 8 threads?

My CPU has 8 cores, so max of 8 threads.

It seems like using 4 threads gives slightly slower render time (94s vs 92s), so it seems like there is a sweet spot there too, what do you think?

I agree, at least based on raw stats. There does need some more testing, because the rendering behavior is somewhat erratic, and render times were unfortunately counted by the age-old method of using a stopwatch...needless to say the reliability of these results were doubtful. If you know of a way to accurately benchmark render times in Natron, it would be great.

[rodlie]

4 threads max for a 16GB machine

I run 8 threads on 16GB and have no problems. I also have a 12 thread computer with 16GB, not seen any issues there either.

[Songtech-0912]

4 threads max for a 16GB machine

I run 8 threads on 16GB and have no problems. I also have a 12 thread computer with 16GB, not seen any issues there either.

@rodlie That may very well be true and it's great that it works for you. However, I can very much guarantee that isn't true for me. Users will always have the ability to tweak it up, but our job is to set sensible defaults, ones that make it hard (if not impossible) for users to run into issues without intentionally tampering with the settings.

[devernay]

@rodlie it really depends on the graph complexity. Try something like Baymax or Spaceship in 4K (that's the standard now), and I'm pretty sure it won't hold 12 threads or even 8 on a 16Gb machine. Each thread will need memory for all inputs and output of the effect.

[rodlie]

• Spaceship maxed out at 15GB, no issues (also tested 4k)
• Baymax maxed out at 14,5GB, no issues (also tested 4k)

i7-4770k (4c/8t), 16GB RAM on macOS 10.13. This is on Natron 2.4, no settings changed.

[devernay]

What we want is a default setting that gives smooth performance and safety of execution. We should avoid using the disk cache as much as possible.
If you use many threads, it will result in disk caching, and when you say "maxed out at 15GB" it most certainly means it uses disk caching, thus less CPU, thus you think you can add more threads, but what you really get in the end is lower performance.
Users still have the ability to play with the maxThreads setting, but users should understand that if you want to get more performance, adding threads don't work unless you also add RAM.

[rodlie]

it most certainly means it uses disk caching

Just tested at work, and no. No disk cache is used.

I'm not fighting against this change, just want to document what works or not, and for me in general no issues (else I would have done something or said something a long time ago).

[devernay]

OK, I noticed that with the conservative approach implemented here, CPU ramps up more slowly than when using all logical cores, because of dependencies in the node graph. So I'm OK to raise it a bit, but I still think that there is a minimum RAM we should allocate to each thread, especially on high-end workstation, which may have 128 logical cores per CPU:

• A 4K frame takes 64MB
• There are around 30 nodes in a typical graph (more when using pyplugs), that makes 2GB to store all the intermediate frames.
  We could change the formula to use 2GB instead of 3.5GB, would that seem right?