Making socket api access transparent

[uttampawar]

I've been working to improve Node.js performance by various ways. One of the way is to use userspace TCP/IP stack with explicit network driver like "dpdk (http://dpdk.org)", instead of kernel stack. I've seen gain in the range of 2x for a typical Node.js application (with DB server) to 4x for small application (micro-services but no DB server) with small flow size (data transfer). But in order to do that, I'd to make socket api transparent so that it can be intercepted appropriately. This has been done by patching linux-syscall.c file for "*nix" platform.
I would like to submit the PR so that it can be pulled/integrated into Node.js project. I also wanted to have a discussion of what the community thinks of this contribution.

[bnoordhuis]

Hard to say because it's not clear to me what "make socket api transparent" means. Any ABI- or API-breaking changes are certainly out of the question for v1.x but up for discussion with master.

[uttampawar]

@bnoordhuis What I mean is that current socket API (accept4, eventfd, epoll_create, etc.) is wrapped under 'syscall()' interface with specific number which makes it harder if not impossible to override or trap a particular system call. As an example if libuv were to call 'epoll_create()' directly instead of via syscall() then it would be easier to trap it in the userspace stack. Here is a snapshot of the change as an example,

int uv__epoll_create(int size) {
#if defined(__NR_epoll_create)
#if defined(MTCP_DPDK)
return epoll_create(size);
#else
return syscall(__NR_epoll_create, size);
#endif
#else
return errno = ENOSYS, -1;
#endif
}

Change is under #MTCP_DPDK

There are many userspace stacks available (such as mTCP, seastar, F-stack, LKL, etc.) Currently I've integrated libuv with userspace stack called 'mTCP' which is the closest BSD compatible stack among them with best performance so far.

[bnoordhuis]

See #1372, we'll probably get rid of the wrappers in due time. They exist to deal with old kernel/libc versions but will be phased out over time.

On a tangent:

[..] 'syscall()' interface with specific number which makes it harder if not impossible to override or trap a particular system call.

I did just that in StrongLoop's strong-agent tool, by interposing syscall. There's no reason it can't work.

[uttampawar]

My changes won't require, when nodejs/node#13306 lands.

[richardlau]

All currently supported versions of Node.js contain a later version of libuv than 1.12.0 in the referenced nodejs/node#13306.

[uttampawar]

@richardlau If I understand correctly, this change has not yet been merged or back ported in libuv v1.x and hence in Node.js. In this case I prefer to implement syscall() interface as @bnoordhuis suggested, i.e. no changes to libuv. Could you please share your thoughts?

[bnoordhuis]

In case you're curious what strong-agent did: https://github.com/strongloop/strong-agent/blob/1f5a1aae2eaff0f6833e4d587cbcdf0e0aa646b8/src/watchdog.h#L105-L204

Caveat emptor: proprietary code.

[uttampawar]

@bnoordhuis Thanks.

[richardlau]

@uttampawar I'm afraid I'm not following you. You stated that:

My changes won't require, when nodejs/node#13306 lands.

But nodejs/node#13306 is the upgrade in Node.js to libuv 1.12.0. All current releases of Node.js have later versions of libuv. If the change you require isn't in libuv v1.x then I'm confused as to your earlier statement.

[uttampawar]

@richardlau I misunderstood the whole story. I thought that syscall wrapper change is already merged into libuv v1.12.0 (part of #1372) and into Node.js by PR nodejs/node#13306 . Then last night I realized my mistake. Sorry about the confusion.

[uttampawar]

@bnoordhuis @richardlau Just curious, do you have time estimate when #1372 will merge into v1.x?

[bnoordhuis]

No.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[unicomp21]

Can someone clue me in on the current "state of the art" with respect to libuv, nodejs, and userspace tcp?