Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added sync mode #21

Merged
merged 4 commits into from
Dec 8, 2018
Merged

Added sync mode #21

merged 4 commits into from
Dec 8, 2018

Conversation

mcollina
Copy link
Member

@mcollina mcollina commented Dec 7, 2018

This is mainly to solve pinojs/pino#542, however I found out that it gives us a 20% speed bump when logging things with sync mode in pino benchmarks.

Benchmarks in pino with sonic-boom master:

$ node benchmarks/basic.bench.js
benchPino*10000: 162.391ms
benchPinoExtreme*10000: 108.874ms
benchPinoNodeStream*10000: 245.955ms

Benchmarks in pino with this PR, enabling sync mode for both destination and extreme:

$ node benchmarks/basic.bench.js
benchPino*10000: 147.822ms
benchPinoExtreme*10000: 83.608ms
benchPinoNodeStream*10000: 243.117ms

cc @jsumners @davidmarkclements

@@ -4,13 +4,14 @@ Extremely fast utf8-only stream implementation to write to files and
file descriptors.

This implementation is partial, but support backpressure and `.pipe()` in is here.
However, it is 20x faster than Node Core `fs.createWriteStream()`:
However, it is 2-3x faster than Node Core `fs.createWriteStream()`:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has the number dropped because Node's gotten faster?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! 🚀

I also adjusted the benchmark to be more realistic

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@davidmarkclements
Copy link
Member

is this just a case of optimizing for benchmarks though?

Does it show improvement in pino-http / hapi-pino benchmarks?

David Mark Clements and others added 2 commits December 7, 2018 14:05
Co-Authored-By: mcollina <matteo.collina@gmail.com>
Co-Authored-By: mcollina <matteo.collina@gmail.com>
@mcollina
Copy link
Member Author

mcollina commented Dec 8, 2018

is this just a case of optimizing for benchmarks though?

It seems so, yes. In fact, we are slowing things down with pino-http for example by 30% when using sync mode. It's still 20% faster than a standard Node.js stream.

Does it show improvement in pino-http / hapi-pino benchmarks?

Not at all.


TL;DR

I will assemble a PR to fix pinojs/pino#542 when using the standard destination. We will lose throughput, but I think that type of behavior should be opt-in. Extreme mode will stay as is, and user will be given an option.

@mcollina
Copy link
Member Author

mcollina commented Dec 8, 2018

My plan for pino is to have:

  1. sync mode if the destination is stdout, this solves Pino 5.8.1 fails to flush log on process exit pino#542.
  2. retain extreme mode as is.
  3. provide an option object to pino.destination() and pino.extreme(), so that users could in theory pass in sync: true or  sync: false. I plan to make the default for pino.destination() synchronous for safety reasons.

This should match the behavior of Node core as much as possible.

cc @jsumners @davidmarkclements

@davidmarkclements
Copy link
Member

  1. 👍
  2. 👍
  3. by definition, destination should be maximum safety, extreme should be maximum performance. From an API perspective I don't see much benefit to allowing sync true|false

@mcollina mcollina merged commit 8208677 into master Dec 8, 2018
@mcollina mcollina deleted the sync-mode branch December 8, 2018 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants