Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve tracing #656

Merged
merged 1 commit into from
Dec 20, 2023
Merged

Improve tracing #656

merged 1 commit into from
Dec 20, 2023

Conversation

talex5
Copy link
Collaborator

@talex5 talex5 commented Dec 10, 2023 •
edited
Loading

This adds tracing of cancellation contexts and OS operations, and
documents and cleans up the API a bit.

When reading events, instead of the consumer providing one function per
event, we now use a single function taking a variant. This makes the API
easier to use and lets the caller decide whether they want to handle all
events (checked by the compiler) or just some of them.

Note: eio_posix yields before each operation to avoid starvation.
However, if it needed to block then it did another yield afterwards when
retrying, which isn't necessary and clutters up the traces. This commit
also removes that.

(based on #554)

@talex5 talex5 added enhancement New feature or request api API design decision labels Dec 10, 2023
@talex5 talex5 force-pushed the trace-cc branch 8 times, most recently from 636558e to fe6e5f7 Compare December 19, 2023 15:19
@talex5 @TheLortex
Improve tracing
69b1150 
This adds tracing of cancellation contexts and OS operations, and
documents and cleans up the API a bit.

When reading events, instead of the consumer providing one function per
event, we now use a single function taking a variant. This makes the API
easier to use and lets the caller decide whether they want to handle all
events (checked by the compiler) or just some of them.

Note: eio_posix yields before each operation to avoid starvation.
However, if it needed to block then it did another yield afterwards when
retrying, which isn't necessary and clutters up the traces. This commit
also removes that.

Co-authored-by: Lucas Pluvinage <lucas@tarides.com>
@talex5 talex5 marked this pull request as ready for review December 19, 2023 15:28
@talex5
Copy link
Collaborator Author

talex5 commented Dec 19, 2023

Using a prototype visualiser, this is what examples/net looks like with these events:

trace

@talex5 talex5 merged commit 4db993d into ocaml-multicore:main Dec 20, 2023
4 of 5 checks passed
@talex5 talex5 deleted the trace-cc branch December 20, 2023 09:32
@c-cube
Copy link
Contributor

c-cube commented Dec 20, 2023

are the green zones representing times where a fiber is actually running?

@talex5
Copy link
Collaborator Author

talex5 commented Dec 20, 2023

Yes, that's right.

@c-cube
Copy link
Contributor

c-cube commented Dec 20, 2023

That's fairly cool, I don't think I've seen that in any "classic" trace visualizer (perfetto, tracy, etc.). It means you emit a lot of events though, don't you?

@talex5
Copy link
Collaborator Author

talex5 commented Dec 21, 2023

We only switch fibers when a fiber suspends, and we need to log why it's suspending anyway, so it's not a lot more events.

This was referenced Dec 21, 2023
Closed
Closed
talex5 added a commit to talex5/opam-repository that referenced this pull request Jan 17, 2024
@talex5
[new release] eio (5 packages) (0.14)
5340789 
CHANGES:

New features / API changes:

- Add `Eio.Executor_pool` (@SGrondin ocaml-multicore/eio#639, reviewed by @talex5).
  Provides an easy way to distribute jobs across domains.

- Add `Fiber.first ~combine` and `Fiber.n_any` (@SGrondin @talex5 ocaml-multicore/eio#587).
  Allows keeping both results in the case where multiple fibers succeed.

- Add `Eio_mock.Backend.run_full` with auto-advancing mock clock (@talex5 ocaml-multicore/eio#644, reviewed by @SGrondin).
  Simplifies testing of code using clocks.

- Add `Buf_write.printf` (@SGrondin @talex5 ocaml-multicore/eio#655).

- Add `Net.listening_addr` (@mefyl ocaml-multicore/eio#555, reviewed by @patricoferris @talex5).
  Useful to get the socket's address if the OS assigns it.

- Add `Promise.try_resolve` (@talex5 ocaml-multicore/eio#646).

- Remove `Cancel_hook_failed` exception (@talex5 ocaml-multicore/eio#640).
  Didn't seem to be used and broke dscheck.

Tracing:

- Improve tracing (@TheLortex @patricoferris @talex5 ocaml-multicore/eio#656).
  Trace cancellation contexts and OS operations, and simplify API.

- Add labels to switches (@talex5 ocaml-multicore/eio#661, reviewed by @SGrondin).

- `Fiber.all`: use the parent fiber (@talex5 ocaml-multicore/eio#665, reviewed by @SGrondin).
  Cleans up the traces a bit.

Performance:

- Faster and simpler `Lf_queue` (@talex5 ocaml-multicore/eio#647, based on work by @polytypic).

- Optimise `Flow.copy` with `Buf_read.as_flow` (@talex5 ocaml-multicore/eio#663, reviewed by @SGrondin, reported by @leostera).

Bug fixes:

- Fix handling of very long IO vectors (@talex5 ocaml-multicore/eio#653, reported by @Cjen1).

- eio_posix: use `caml_enter_blocking_section` in more places (@talex5 ocaml-multicore/eio#654, reviewed by @SGrondin).

- eio_posix: work around `caml_unix_alloc_sockaddr` bug (@talex5 ocaml-multicore/eio#651).

- Remove default backtrace from `Switch.fail` (@talex5 ocaml-multicore/eio#664).

Documentation:

- Organise eio.mli better (@talex5 ocaml-multicore/eio#667).

- Fix quoting of quotes in process error messages (@talex5 ocaml-multicore/eio#666, reviewed by @SGrondin).

- Mention Path module in File and Fs documentation (@talex5 ocaml-multicore/eio#659, requested by @clecat).

- Minor documentation updates (@SGrondin @talex5 ocaml-multicore/eio#670).

Build / internals:

- Allow closing synchronous streams (@talex5 ocaml-multicore/eio#641, reviewed by @SGrondin).
  This isn't currently exposed in the public interface.

- Fix non-idempotent tests (@SGrondin ocaml-multicore/eio#662).

- eio_windows: add explicit fmt dependency (@talex5 ocaml-multicore/eio#643).
@talex5 talex5 mentioned this pull request Jan 17, 2024
Merged
talex5 added a commit to talex5/opam-repository that referenced this pull request Jan 17, 2024
@talex5
[new release] eio (5 packages) (0.14)
6eb34f3 
CHANGES:

New features / API changes:

- Add `Eio.Executor_pool` (@SGrondin ocaml-multicore/eio#639, reviewed by @talex5).
  Provides an easy way to distribute jobs across domains.

- Add `Fiber.first ~combine` and `Fiber.n_any` (@SGrondin @talex5 ocaml-multicore/eio#587).
  Allows keeping both results in the case where multiple fibers succeed.

- Add `Eio_mock.Backend.run_full` with auto-advancing mock clock (@talex5 ocaml-multicore/eio#644, reviewed by @SGrondin).
  Simplifies testing of code using clocks.

- Add `Buf_write.printf` (@SGrondin @talex5 ocaml-multicore/eio#655).

- Add `Net.listening_addr` (@mefyl ocaml-multicore/eio#555, reviewed by @patricoferris @talex5).
  Useful to get the socket's address if the OS assigns it.

- Add `Promise.try_resolve` (@talex5 ocaml-multicore/eio#646).

- Remove `Cancel_hook_failed` exception (@talex5 ocaml-multicore/eio#640).
  Didn't seem to be used and broke dscheck.

Tracing:

- Improve tracing (@TheLortex @patricoferris @talex5 ocaml-multicore/eio#656).
  Trace cancellation contexts and OS operations, and simplify API.

- Add labels to switches (@talex5 ocaml-multicore/eio#661, reviewed by @SGrondin).

- `Fiber.all`: use the parent fiber (@talex5 ocaml-multicore/eio#665, reviewed by @SGrondin).
  Cleans up the traces a bit.

Performance:

- Faster and simpler `Lf_queue` (@talex5 ocaml-multicore/eio#647, based on work by @polytypic).

- Optimise `Flow.copy` with `Buf_read.as_flow` (@talex5 ocaml-multicore/eio#663, reviewed by @SGrondin, reported by @leostera).

Bug fixes:

- Fix handling of very long IO vectors (@talex5 ocaml-multicore/eio#653, reported by @Cjen1).

- eio_posix: use `caml_enter_blocking_section` in more places (@talex5 ocaml-multicore/eio#654, reviewed by @SGrondin).

- eio_posix: work around `caml_unix_alloc_sockaddr` bug (@talex5 ocaml-multicore/eio#651).

- Remove default backtrace from `Switch.fail` (@talex5 ocaml-multicore/eio#664).

Documentation:

- Organise eio.mli better (@talex5 ocaml-multicore/eio#667).

- Fix quoting of quotes in process error messages (@talex5 ocaml-multicore/eio#666, reviewed by @SGrondin).

- Mention Path module in File and Fs documentation (@talex5 ocaml-multicore/eio#659, requested by @clecat).

- Minor documentation updates (@SGrondin @talex5 ocaml-multicore/eio#670).

Build / internals:

- Allow closing synchronous streams (@talex5 ocaml-multicore/eio#641, reviewed by @SGrondin).
  This isn't currently exposed in the public interface.

- Fix non-idempotent tests (@SGrondin ocaml-multicore/eio#662).

- eio_windows: add explicit fmt dependency (@talex5 ocaml-multicore/eio#643).
nberth pushed a commit to nberth/opam-repository that referenced this pull request Jun 18, 2024
@talex5 @nberth
[new release] eio (5 packages) (0.14)
c39875a 
CHANGES:

New features / API changes:

- Add `Eio.Executor_pool` (@SGrondin ocaml-multicore/eio#639, reviewed by @talex5).
  Provides an easy way to distribute jobs across domains.

- Add `Fiber.first ~combine` and `Fiber.n_any` (@SGrondin @talex5 ocaml-multicore/eio#587).
  Allows keeping both results in the case where multiple fibers succeed.

- Add `Eio_mock.Backend.run_full` with auto-advancing mock clock (@talex5 ocaml-multicore/eio#644, reviewed by @SGrondin).
  Simplifies testing of code using clocks.

- Add `Buf_write.printf` (@SGrondin @talex5 ocaml-multicore/eio#655).

- Add `Net.listening_addr` (@mefyl ocaml-multicore/eio#555, reviewed by @patricoferris @talex5).
  Useful to get the socket's address if the OS assigns it.

- Add `Promise.try_resolve` (@talex5 ocaml-multicore/eio#646).

- Remove `Cancel_hook_failed` exception (@talex5 ocaml-multicore/eio#640).
  Didn't seem to be used and broke dscheck.

Tracing:

- Improve tracing (@TheLortex @patricoferris @talex5 ocaml-multicore/eio#656).
  Trace cancellation contexts and OS operations, and simplify API.

- Add labels to switches (@talex5 ocaml-multicore/eio#661, reviewed by @SGrondin).

- `Fiber.all`: use the parent fiber (@talex5 ocaml-multicore/eio#665, reviewed by @SGrondin).
  Cleans up the traces a bit.

Performance:

- Faster and simpler `Lf_queue` (@talex5 ocaml-multicore/eio#647, based on work by @polytypic).

- Optimise `Flow.copy` with `Buf_read.as_flow` (@talex5 ocaml-multicore/eio#663, reviewed by @SGrondin, reported by @leostera).

Bug fixes:

- Fix handling of very long IO vectors (@talex5 ocaml-multicore/eio#653, reported by @Cjen1).

- eio_posix: use `caml_enter_blocking_section` in more places (@talex5 ocaml-multicore/eio#654, reviewed by @SGrondin).

- eio_posix: work around `caml_unix_alloc_sockaddr` bug (@talex5 ocaml-multicore/eio#651).

- Remove default backtrace from `Switch.fail` (@talex5 ocaml-multicore/eio#664).

Documentation:

- Organise eio.mli better (@talex5 ocaml-multicore/eio#667).

- Fix quoting of quotes in process error messages (@talex5 ocaml-multicore/eio#666, reviewed by @SGrondin).

- Mention Path module in File and Fs documentation (@talex5 ocaml-multicore/eio#659, requested by @clecat).

- Minor documentation updates (@SGrondin @talex5 ocaml-multicore/eio#670).

Build / internals:

- Allow closing synchronous streams (@talex5 ocaml-multicore/eio#641, reviewed by @SGrondin).
  This isn't currently exposed in the public interface.

- Fix non-idempotent tests (@SGrondin ocaml-multicore/eio#662).

- eio_windows: add explicit fmt dependency (@talex5 ocaml-multicore/eio#643).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api API design decision enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@talex5 @c-cube