Add Windows/macOS/Linux and x86/arm64 fuzzing CI #255
Conversation
|
Windows is broken due to janestreet/jst-config#1 making cstruct-async uninstallable |
I just used |
.drone.yml
Outdated
| - opam install -y . | ||
| - opam install -y crowbar fmt | ||
| - opam exec -- dune build fuzz/fuzz.exe | ||
| - mkdir out && mkdir in && echo bactrian > in/test |
There was a problem hiding this comment.
If you're using bun you don't need to do this anymore - bun should take care of it for you.
There was a problem hiding this comment.
i cant think of a pun involving bun but i pushed the change for fun and now i have to run
.drone.yml
Outdated
| - opam exec -- dune build fuzz/fuzz.exe | ||
| - mkdir out && mkdir in && echo bactrian > in/test | ||
| - opam pin add -y bun https://github.com/yomimono/ocaml-bun.git | ||
| - opam exec -- bun -i in -o out -- ./_build/default/fuzz/fuzz.exe |
There was a problem hiding this comment.
Here's where I'd add a timeout invocation. It's done via dune runtest in this version, but https://github.com/yomimono/ocaml-test-stdlib/blob/primary/test/jbuild#L29 has the basic shape of how this was done in earlier bun invocations.
|
It says: |
|
Hm, this is now passing but the fuzzer appears to terminate too soon despite a 30 minute timeout. https://cloud.drone.io/mirage/ocaml-cstruct/17/1/2 -- any ideas whats up here? |
|
No idea. The summary is certainly disappointing: |
|
@kayceesrk reported a similar bug - ocurrent/bun#6 . I'm trying to figure out what process-handling mistake I'm making in |
|
Hm, there have also been multiple |
|
I'm not sure I understand how it's supposed to work. From a quick look, it seems that The main application finishes with: Printf.printf "Fuzzers launched. Waiting %d seconds for the first status update...\n%!" delay;
ignore @@ Unix.alarm delay;
Ok (Unix.pause ())Does that mean that if any signal handler ever returns then the program exits? |
To be honest I don't really remember what was going on here. I'm working on something else this afternoon but I'll try and figure out (1) the original intent (2) whether it ever worked outside of the testing scenario where it demonstrably worked for some test code. |
|
@yomimono : if you're busy with other things at the moment, I can take a look at the bun process logic. |
This was slightly complex since it needs to avoid `system` calls, so I modified the driver to never fail, but this requires a small hack to maintain compat with OCaml 4.06 The gitattributes matter as otherwise line numbers change on windows and the expect tests fail
|
And now it gets a lot of fuzzing done, hurrah! @yomimono @talex5 |
…uct-sexp and cstruct-lwt (5.0.0) CHANGES: **Security**: This release tightens bounds checks to ensure that data outside a given view (but still inside the underlying buffer) cannot be accessed. - `sub` does more checks (mirage/ocaml-cstruct#244 mirage/ocaml-cstruct#245 @hannesm @talex5 review by @dinosaure) - `add_len` and `set_len` are now deprecated and will be removed in a future release. (mirage/ocaml-cstruct#251 @hannesm) - do not add user-provided data for bounds checks (mirage/ocaml-cstruct#253 @hannesm, report and review by @talex5) - improve CI to add fuzzing (mirage/ocaml-cstruct#255 mirage/ocaml-cstruct#252 @avsm @yomimono @talex5) **Remove Unix dependency**: cstruct now uses the new `bigarray-compat` library instead of Bigarray directly, to avoid a dependency on Unix when using OCaml compilers less than 4.06.0. This will break downstream libraries that do not have a direct dependency on `Bigarray`. Simply fix it in your library by adding a `bigarray` dependency in your dune file. (mirage/ocaml-cstruct#247 @TheLortex) **Capability module**: To improve the safety of future code with stronger type checking, this release introduces a new `Cstruct_cap` module which makes the underlying Cstruct an abstract type instead of a record. In return for this extra abstraction, the module can enforce read-only, write only, and read/write buffers by tracking them as phantom type variables. Although this library shares an implementation internally with classic `Cstruct`, it is a significant revision and so we will be gradually migrating to it. Feedback on it is welcome! (mirage/ocaml-cstruct#237 @dinosaure and many excited reviewers) **Ppx compare functions**: A new `compare_X` function is generated for `cenum` declarations. This respects custom ids supplied in the cenum declaration and so is more robust than polymorphic compare (mirage/ocaml-cstruct#248 @emillon) The CI has also been switched over to both Azure Pipelines and Drone in addition to Travis, and as a result the tests all run on Windows, macOS, various Linux distributions, on x86 and arm64 machines, and runs AFL fuzz tests on the Drone cloud (mirage/ocaml-cstruct#255 @avsm).
Azure Pipelines and cloud.drone.io
For the fuzz tests, this fixes #252, but how do we tell
afl-fuzzhow long to run for? (See .drone.yml, tweaks welcome to the command line). It's running atm on https://cloud.drone.io/mirage/ocaml-cstruct/6/1/2 which I hope is a public URL.