Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a UI to fuzzing to report progress, code coverage, interesting inputs, and other stats #20812

Closed
andrewrk opened this issue Jul 26, 2024 · 0 comments · Fixed by #20958
Closed
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing
Milestone

Comments

@andrewrk
Copy link
Member

Extracted from #20773.

While fuzzing, it's important to get a sense of what is happening. Fuzzing is an interactive process that involves tweaking, often to the fuzzing algorithm itself. Zig is well-positioned for this since one can make edits to fuzzer.zig and have them picked up immediately. However, both users of the fuzzing features and contributors to the fuzzer need a way to understand intuitively what is happening.

This is an open-ended issue but here are some requirements for a fuzzing UI:

  • It should be live-updating so that you can watch the state change without manual inputs
  • It should report code coverage (support code coverage when testing #352) as the fuzzer explores more of the codebase
  • It should report stats such as how many iterations, iterations per second, unique inputs found, number of bugs found, which fuzzing strategies in particular have been effective, etc.
  • There should be a way to gain access to the interesting inputs while the fuzzing process is still ongoing.

I personally think the terminal is a terrible place for a UI, so I will be exploring using a web-based client and server to explore and interact with the information. This will allow for some potentially interesting interactions such as browsing code coverage by line, and placing "fuzz points" (think breakpoints but for making the fuzzer try to hit a particular line) by clicking in the gutter.

This strategy also leaves room for fuzzing across multiple machines, aggregating the data and serving the results across a network.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing labels Jul 26, 2024
@andrewrk andrewrk added this to the 0.14.0 milestone Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. fuzzing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant