introduce scoped logging & revise debug print

[FObersteiner]

1. I noticed that there are debug prints in the build script; those can mix in with other console output if zbench is an indirect dependency (e.g. my check_filetimes demo for zdt got a "Failed to open directory x" since zbench is used in zdt). Scoped logging should avoid this. See also : https://ziggit.dev/t/best-practices-writing-to-stdout/2760

2. Also, there seems to be mix of printing to stdout and stderr (std.debug.print). IMHO, this should be stdout only since we do not want to signal errors when running benchmarks (stderr output can be interpreted as exit(1)).

[FObersteiner]

@Bryysen you might want to check this PR

[Bryysen]

I don't think we need to call Benchmark.report anymore now that Benchmark.prettyPrint displays the number of runs performed in the pretty-table

[FObersteiner]

Right, good point. Should we rename prettyPrint --> report then ?

[Bryysen]

I suppose? I don't have strong opinions on the naming either way.

[FObersteiner]

implemented via 4a8462b

[Bryysen]

I'm still getting the "failed to open directory x" 😢

[****@**** index_map]$ zig build bench
warning(zbench_build): failed to open directory: util
benchmark                 runs     time (avg ± σ)         (min ............. max)      p75        p99        p995      
---------------------------------------------------------------------------------------------------------------------
...

Looks good otherwise!

[FObersteiner]

I'm still getting the "failed to open directory x" 😢

lol now at least you know where that message is coming from ;-) I think the problem is cwd, which in this case is the directory of your current project I assume, not zBench's.

[Bryysen]

Yeah the error/warning is a lot more descriptive now at least. Should we then first detect whether std.fs.cwd() actually is "zBench" before starting to iterate through the test directiories? That seems like a brittle solution.. it's odd to me that we are descending into the test-setup part of build.zig when zBench is being compiled as a dependency in the first place :/

[FObersteiner]

it's odd to me that we are descending into the test-setup part of build.zig when zBench is being compiled as a dependency in the first place :/

Not sure why this was introduced, maybe zig didn't detect tests in all files? In general, I think a straight-forward solution for such a case would be to do that same as is currently being done for the examples. That way, you don't miss a file plus you don't need to obtain the cwd 'manually'. Described it here in more detail.

Besides, the build.zig will need a major revision for zig 0.12 anyway, a lot of changes seem to be made to the build-system right now.

[hendriknielaender]

it's odd to me that we are descending into the test-setup part of build.zig when zBench is being compiled as a dependency in the first place :/

Not sure why this was introduced, maybe zig didn't detect tests in all files? In general, I think a straight-forward solution for such a case would be to do that same as is currently being done for the examples. That way, you don't miss a file plus you don't need to obtain the cwd 'manually'. Described it here in more detail.

Besides, the build.zig will need a major revision for zig 0.12 anyway, a lot of changes seem to be made to the build-system right now.

Yeah I wanted to have a more dynamic solution for the test setup, since you can have tests next to the code. And not having only dedicated test files like: format_test.zig. But since this lib is not that huge so far, i also think we could revert this and use the same approach as mentioned.

I created a tracking issue for 0.12, so that we can already start migrating the codebase to 0.12.
#35

[hendriknielaender]

@Bryysen @FObersteiner please have a look into this PR, this should simplify the test setup.

#36

[hendriknielaender]

Thanks for bumping the version! 👍