Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial support for integrated fuzzing #20725

Merged
merged 13 commits into from
Jul 23, 2024
Merged

initial support for integrated fuzzing #20725

merged 13 commits into from
Jul 23, 2024

Conversation

andrewrk
Copy link
Member

@andrewrk andrewrk commented Jul 22, 2024

Initial implementation of #20702.

  • Add the -ffuzz and -fno-fuzz CLI arguments.
  • Detect fuzz testing flags from zig cc.
  • Set the correct clang flags when fuzz testing is requested. It can be
    combined with TSAN and UBSAN.
  • Compilation: build fuzzer library when needed which is currently an
    empty zig file.
  • Add optforfuzzing to every function in the llvm backend for modules
    that have requested fuzzing.
  • In ZigLLVMTargetMachineEmitToFile, add the optimization passes for
    sanitizer coverage.
  • std.mem.eql uses a naive implementation optimized for fuzzing when
    builtin.fuzz is true.
  • Add the build system API for enabling fuzzing.
  • Add @disableInstrumentation builtin and use it in start code to prevent using threadlocal variables before threadlocal storage is initialized.
  • Makes the LLVM backend emit sancov calls.

Current status:

$ stage4/bin/zig build-exe ../test/standalone/simple/hello_world/hello.zig -ffuzz 
$ ./hello 
0x105a113: comparison of 0 and 9
0x105a183: switch on value 6 (32 bits) with 1 cases
0x105a113: comparison of 1 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 2 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 3 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 4 and 9
0x105a183: switch on value 1 (32 bits) with 1 cases
0x105a113: comparison of 5 and 9
0x105a183: switch on value 7 (32 bits) with 1 cases
0x105a113: comparison of 6 and 9
0x105a183: switch on value 1685382482 (32 bits) with 1 cases
0x105a113: comparison of 7 and 9
0x105a183: switch on value 1685382480 (32 bits) with 1 cases
0x105a113: comparison of 8 and 9
0x105a183: switch on value 1685382481 (32 bits) with 1 cases
0x105a1d2: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105afe1: switch on value 0 (16 bits) with 3 cases
0x105a20f: comparison of 0 and 0
0x105a25f: comparison of 16777216 and 8388608
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105b1b9: switch on value 0 (16 bits) with 4 cases
0x105a2e4: comparison of 0 and 0
0x109eeac: comparison of 1 and 11
0x109eecf: comparison of 9 and 11
0x109eef2: comparison of 19 and 11
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ecbc: comparison of 0 and 0
0x109eeac: comparison of 1 and 4
0x109eecf: comparison of 9 and 4
0x109eef2: comparison of 19 and 4
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ed0d: comparison of 0 and 0
0x109eeac: comparison of 1 and 7
0x109eecf: comparison of 9 and 7
0x109eef2: comparison of 19 and 7
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109ed5e: comparison of 0 and 0
0x109eeac: comparison of 1 and 8
0x109eecf: comparison of 9 and 8
0x109eef2: comparison of 19 and 8
0x109f041: comparison of 0 and 4
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x109edaf: comparison of 0 and 0
0x105b9e8: comparison of 0 and 0
0x109eeac: comparison of 1 and 13
0x109eecf: comparison of 9 and 13
0x109eef2: comparison of 19 and 13
0x109f041: comparison of 0 and 0
0x10b39f8: comparison of 18446744073709547520 and 0
0x10b3a3a: comparison of 0 and 0
0x109f273: comparison of 0 and 0
0x109e4c8: comparison of 18446744073709547520 and 0
0x109e50a: comparison of 0 and 0
0x105bafc: switch on value 0 (16 bits) with 3 cases
0x105a421: comparison of 0 and 0
0x1059c73: comparison of 0 and 14
0x1059ce7: comparison of 0 and 14
0x1059d24: comparison of 14 and 14
0x1059d6a: comparison of 0 and 14
0x109f7c9: comparison of 0 and 14
Hello, World!
0x109e4c8: comparison of 18446744073709547520 and 14
0x109e50a: comparison of 0 and 14
0x109f87b: switch on value 0 (16 bits) with 15 cases
0x105bc4f: comparison of 0 and 0
0x1059dd4: comparison of 0 and 0
0x1059c73: comparison of 14 and 14
0x1059b24: comparison of 0 and 0

I'm putting this up because I think it will already benefit zig-afl-kit mainly because of making std.mem.eql optimize for fuzzing when -ffuzz is selected. Probably also because of the optforfuzzing attribute on all the functions in the LLVM backend.

Closes #5484. See #20702 for follow-up issues.

Next steps:

  • Implement libfuzzer in zig
  • Build system and test runner integration

@andrewrk andrewrk force-pushed the fuzz branch 2 times, most recently from a2fdc16 to 25fc237 Compare July 22, 2024 07:27
src/link/MachO.zig Outdated Show resolved Hide resolved
src/link/MachO.zig Show resolved Hide resolved
* Add the `-ffuzz` and `-fno-fuzz` CLI arguments.
* Detect fuzz testing flags from zig cc.
* Set the correct clang flags when fuzz testing is requested. It can be
  combined with TSAN and UBSAN.
* Compilation: build fuzzer library when needed which is currently an
  empty zig file.
* Add optforfuzzing to every function in the llvm backend for modules
  that have requested fuzzing.
* In ZigLLVMTargetMachineEmitToFile, add the optimization passes for
  sanitizer coverage.
* std.mem.eql uses a naive implementation optimized for fuzzing when
  builtin.fuzz is true.

Tracked by #20702
This is needed to ensure that start code does not try to access thread
local storage before it has set up thread local storage.
@ProkopRandacek
Copy link
Contributor

Next steps:

  • Implement libfuzzer in zig
  • Build system and test runner integration

I have committed to implementing this next month this as part of my term project. Is there a space for collaboration? I have to implement this either way and I would love to contribute my implementation upstream.

@andrewrk
Copy link
Member Author

I have committed to implementing this next month this as part of my term project. Is there a space for collaboration? I have to implement this either way and I would love to contribute my implementation upstream.

Sure. I'll have the system wired together shortly, so you can see how it is all supposed to work.

Then it is a matter of improving the genetic algorithm. We can look at afl-fuzz.c for inspiration.

This prevents it from trying to access thread local storage before it
has set up thread local storage, particularly when code coverage
instrumentation is enabled.
`-fno-sanitize=function` must come after `-fsanitize=undefined` or it
has no effect.
@@ -263,6 +264,14 @@ pub const list = list: {
.illegal_outside_function = true,
},
},
.{
"@disableInstrumentation",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this built-in to the langref?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it makes sense to leave out of the langref but include in the spec

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this built-in to the langref?

Yes of course

i think it makes sense to leave out of the langref but include in the spec

This not Meghan's Opinion Tracker. Provide technical arguments, or keep your opinion to yourself.

@andrewrk andrewrk merged commit 6f3e993 into master Jul 23, 2024
10 checks passed
@andrewrk andrewrk deleted the fuzz branch July 23, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

-fsanitize=fuzzer support
5 participants