trace_event.h status?

[Qard]

What is the status of the trace_event.h work happening with V8? Is that in a state yet where we can start integrating it with node?

If anyone working on V8 is available to attend the collaborator summit, I'd love to work together to get the development effort of this kicked off.

cc: @ofrobots @natduca

[ofrobots]

Thanks for poking on this. trace-event.h hasn't merged into V8 yet (@fmeawad is hoping to get it into 4.5) however, I don't think we need to wait for it to land before building upon it.

I did manage to get some time to play with it a little bit. Here are the next steps in getting trace-event into node, as I see them. This is my understanding of trace-event, so @fmeawad and @natduca can keep me honest here.

1. trace_event.h – This is a bunch of C macros that provide a low-overhead (single test) mechanism to add traces around your code. It is a bunch of pre-processor macro tricks to make this cheap. This file is going to be merged into V8, but in the interim nothing is stopping us from copying this into io.js and prototyping. That's how I was playing with this.
2. A mechanism to define, enable and disable Categories and Category Filters. These filters control which traces are active. This code is present in chromium in base/trace_event/trace_event_impl.[h|cc]. We can copy it from there.
3. A buffer that gathers traces and then asynchronously flushes them a sink. This is implemented by TraceBuffer and its subclasses in chromium. We can copy the code from there.
4. A sink. File, Network, etc. V8 sends these to v8.log presently. We (node) can figure out what makes sense here.

Beyond this, we will need to do a few things in JavaScript:

1. Expose the trace-event API in JavaScript so that node-core and user-space can start adding traces to their code.
2. Re-implement debug() logging on top of the trace-event API.

The great thing, once this is done is that we'll get traces from the whole stack (V8, node-core, user-space) consolidated into the same place.

I started hacking on branch with the above a while back but got side-tracked. I can create the branch on github if there are people who want to collaborate and hack together at this.

[trevnorris]

To clarify, the stats are buffered and delivered async. Is the processing of those stats expected to be done off the main thread to prevent blocking? If so, how would this work with exposing a JS API that would force blocking the main thread?

[ofrobots]

Can you elaborate what you mean by 'stats'? How are those different from a 'trace'? I know 'trace' can mean different things to different people and in different context, but this is what I mean in this context:

var debug = util.debuglog('net');
//...
debug('createConnection', args);

Or this:

code-creation,LazyCompile,0,0x6e7cbf21420,436,"BIT_OR native runtime.js:157:16",0x195af1c3c0,

The JS Trace API needs to do minimal work to create the packet of information and then call trace-event to deposit it to the buffer. This would be synchronous.

I do expect the TraceBuffer is would be implementing the flushing as a separate thread, or some other concurrency mechanism that is off the main thread.

BTW, the util.debuglog is a decent API for one-shot events (trace-event calls these 'INSTANT' events). We will need more to describe other types of traces, e.g. matched being/end, async flows, etc.

[trevnorris]

Sorry. By "stats" I was talking about the "packet of information" collected and stored in the buffer to be processed asynchronously.

[ofrobots]

I think the tracing API should be synchronous. It needs to be near-zero cost (single test) when the trace-category is not enabled. It needs to be very cheap when trace category is enabled. All processing should be done off the main thread.

[natduca]

I've gotten a bit swamped so am not able to keep pace here right now, but @fmeawad indicated he's planning a reply for this bug shortly :)

[fmeawad]

I would like to apologize that trace_event.h has not landed yet in V8. We still think that this is the right direction. We found that we needed more support on the chrome side to support the same things the v8.log currently supports. Therefore it got lowered in priority for a while, but we are still trying to get it landed in 4.5.

[trevnorris]

Does it need to be included in V8 for the macros to be used in node?

[fmeawad]

I think you need another copy of it in node to be able to use it. And it does not conflict. @ofrobots can comment better since I believe has a prototype.

[trevnorris]

What I was getting at is, if the interface is stable could we start using it in node before it's been integrated into V8?

[fmeawad]

The interface is very stable, and it has been used in chrome and blink among other places for a long time now.

[trevnorris]

Then let's begin discussions on how it should be integrated into node. How would we like to proceed?

[ofrobots]

I think the file can be immediately copied into node. We can remove it once it lands in v8 and start including it from there. Until that happens, we won't be getting traces from V8, but that should be fine.

[trevnorris]

Sounds good. That will give us time to work on the JS interface. I'd like to explore integration with AsyncWrap.