Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API redesign #61

Merged
merged 1 commit into from
Mar 17, 2024
Merged

Conversation

bens
Copy link
Collaborator

@bens bens commented Mar 12, 2024

Taking a stab at #52, I had the exact same API in mind before I saw that issue. Currently it's just a basic draft but it works for the most basic case.

util/format.zig Outdated Show resolved Hide resolved
zbench.zig Outdated Show resolved Hide resolved
@bens bens marked this pull request as draft March 12, 2024 15:49
@bens bens force-pushed the api-redesign branch 2 times, most recently from 6abb394 to a4e61f9 Compare March 13, 2024 04:32
@bens
Copy link
Collaborator Author

bens commented Mar 15, 2024

AFAICT I've implemented all the features of the current library, and added a couple of things which could be moved into separate PRs to simplify review. Those are

  • the incremental running API which is shown in the progress.zig example test,
  • the getSystemInfo change which makes it use only stack (4KB though) and static memory so no need for an allocator,
  • and the writeJSON functions.

I made a Runner abstraction which does the job of the old run() function but its advantage is it removes all I/O from it, making it functional and much more easily testable, you can just pass in whatever nanosecond values you want in tests and the runner's responses will be deterministic. I think this is worthwhile but it could be removed and just stick with the one-shot run function which does its own time measurement I/O.

I can take out those three changes for now but I might hold off on making PRs for them to avoid needless conflicts if some version of this PR gets merged.

I'd love a thorough review, there's obviously a lot changed but I wasn't able to make small changes to get to the desired API. Reviewing the diffs probably won't be very helpful, best just to check out the branch and play around.

@bens bens marked this pull request as ready for review March 15, 2024 05:28
@bens
Copy link
Collaborator Author

bens commented Mar 15, 2024

Ok, I've moved those three features to separate branches based on this api-redesign branch, so they can wait for later I think. The diff is a little less onerous now...

Copy link
Owner

@hendriknielaender hendriknielaender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @bens, first of all really cool PR. Really appreciate that you tackled this 💪. This API looks already a lot cleaner.

For the upcoming PR, especially for the json output, i opened an issue please have a look if you agree with the feature request. #63

Maybe we can also go first with your already prepared PR 👍 and then tackle the issue.

examples/basic.zig Show resolved Hide resolved
util/runner/auto.zig Outdated Show resolved Hide resolved
zbench.zig Outdated Show resolved Hide resolved
Copy link
Collaborator

@FObersteiner FObersteiner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not much more to say except I like it ^^ I'm pretty much inactive at the moment when it comes to Zig 0.11, only using the 0.12-dev version for my little projects. Once 0.12 is released as "stable", I'll re-work some zBench benchmarks and will probably have more to contribute here ;-)

@@ -0,0 +1,5 @@
const std = @import("std");

pub const Error = std.mem.Allocator.Error;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would call this "MemErr" or something along that line, to differentiate between other, "normal" errors

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was it's any error that can be thrown from a runner, but currently that's only allocation errors.

@@ -30,6 +56,7 @@ pub fn linux(allocator: std.mem.Allocator) !OsInfo {
const memory = try lnx.getTotalMemory();

return OsInfo{
.allocator = allocator,
Copy link
Collaborator

@FObersteiner FObersteiner Mar 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we're "fixing" the allocator now in OsInfo, what about making getSystemInfo a method of this struct? The platform-specific implementations could also be non-public.

Copy link
Collaborator Author

@bens bens Mar 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had some other ideas around the OsInfo thing but moved them to another branch, this was a small change to make the structure usable as not just an internal thing. See bens@59f8d7a for the other ideas, basically I wanted to avoid needing an allocator, only use stack while fetching the info and then cache it in static memory. Then it never needs to be deinited.

Currently the whole API is in the zbench module, moving getSystemInfo to zbench.util.platform.OsInfo is a little more obscure, otherwise I don't mind.

util/optional.zig Show resolved Hide resolved
@hendriknielaender
Copy link
Owner

One last request, can you update the README. Since the config changes a bit 👍

zbench.zig Show resolved Hide resolved
- Make the Benchmark type manage any number of benchmark functions and run them
  all with a single call to `run()`.

- Provide all recorded timing information in `Results`, so any desired analysis
  and output formatting is available to the library user.

- Made the running logic more testable by converting it to an iterator type
  interface - it doesn't actually run the code under test itself now.

- Avoid needing an allocator when getting system info, it's only fetched once
  and then cached in static memory now.
@hendriknielaender hendriknielaender merged commit 5cc5e90 into hendriknielaender:main Mar 17, 2024
5 checks passed
@bens bens deleted the api-redesign branch March 17, 2024 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants