Maximum number of iterations limited by type

[FObersteiner]

Hi there, I was wondering if there was a specific reason that the iterations counter is of type u16;

pub const Config = struct {
    /// Number of iterations the benchmark has been run. Initialized to 0.
    /// If 0 then zBench will calculate an value.
    iterations: u16 = 0,

I was playing around with some small functions the other day, and found that benchmark results using zbench became much more stable at numbers of iterations being significantly higher than u16 max. So, would it hurt to make the counter a u32?

[hendriknielaender]

Hey @FObersteiner, the u16 was chosen because it’s sufficient for most benchmarks and keeps the configuration lightweight.

It could make sense to set higher iteration counts (beyond 65,535) for microbenchmarks. For me switching to a u32 makes sense, and there is no inherent harm in changing the iterations counter to a u32.

Feel free to open a PR that updates the iterations counter and any related configuration fields (like max_iterations).

[FObersteiner]

OK I'll make a PR tonight. 2 more bytes here and there won't hurt I guess ;-) Micro-benchmarks might not be the aim of zbench, however I think we can still achieve stable results for small functions, at least that's my experience. They might not be totally accurate (see baseline discussion), but still serve well for relative comparisons.

If a user doesn't specify a number of iterations, zbench will still limit them, either by Config.max_iterations or Config.time_budget_ns - while I'm at it, I'll check the limiting mechanism.