Introducing wsnic, a fresh take at the websockproxy server

[chschnell]

wsnic is a WebSocket to TAP device proxy server for linux intended as a fully compatible alternative to benjamincburns/websockproxy (and its derivates). The service provided is identical but it is implemented using different methods.

Benchmark comparison for a 374 MB download from a local HTTP server using curl (websockproxy is unthrottled):

Maximum throughput:

Proxy	Buildroot	Debian 12
websockproxy	909 kB/s	1,752 kB/s
wsnic	32,768 kB/s	15,360 kB/s

Under Buildroot, wsnic is around 36 times faster than websockproxy, and under Debian around 9 times faster.

Of course, these benchmarks vary widely from system to system, but they do provide reproducible results for each system.

[spetterman66]

Well yeah thats great looking forward for when its implemented in v86 :D

[chschnell]

Well yeah thats great looking forward for when its implemented in v86 :D

wsnic uses the same NetworkAdapter in V86 as websockproxy, I run all tests against the current V86 master.

So it works with an unmodified V86, but it is still work in progress so I wouldn't recommend using it yet.

[copy]

Thanks, this looks quite promising. Feel free to send a PR to point docs/networking.md to this implementation.

[chschnell]

I found a very nice solution for what to use as the TLS termination proxy in wsnic.

So far I had used Apache2 for this task which was very efficient but really quite inconvenient when it comes to documenting its setup procedure.

I had tested socat to see if it could replace Apache2, it works but is reaching only about a third of the throughput that Apache2 delivers.

However, today I came across stunnel, it fits my use case perfectly and is performing just as good as Apache2! It was really easy to embed and now serves as a built-in method to support secured wss:// connections in wsnic out of the box, only a certificate is required. For documentation see wsnic's README at https://github.com/chschnell/wsnic. In all, the project has matured quite a bit.

I've noticed that many other proxy servers also rely on an external TLS termination proxy to support wss:// connections, I think stunnel could be a highly recommended tool for this task.

[chschnell]

I've split wsnic internally into a WebSocket frontend and an abstract network backend. I want to try out the many different technologies the Linux kernel has to offer for virtual networking, besides the one I learned from the original websockproxy, hence this approach.

So far I've implemented 4 different backends and I think I got my first alternate network backend to work. That's exciting to me because I hadn't really ever worked with the virtual networking components in Linux before, and this involved quite a bit of learning.

The 4 backends are:

1. tapdev -- a single TAP device shared by all WebSocket clients with MAC-based packet routing in wsnic
2. brtap -- a single virtual bridge and one exclusive TAP device per WebSocket client
3. brveth -- a single virtual bridge and one veth pair per WebSocket client
4. macvtap -- one macvtap device per WebSocket client

The first backend uses the same virtual network architecture as the original websockproxy. Since it uses only one shared TAP device for all WebSocket connections it must be able to forward each incoming packet from the TAP device to its proper WebSocket connection, it does so based on the MAC addresses in ethernet packets. It also forwards multicast and broadcast packets from the TAP device to all WebSockets, and does a few other things in this context.

The second one is the one I got to work today. Its advantage over tapdev is that it doesn't have to look at each packet for routing purposes, packets in both directions are just blindly passed through. Another advantage is that I have a clear strategy against the backpressure problem that generally arises in proxy servers (when a sender is pushing data faster than it can be forwarded to the receiver causing the data to balloon in wsnic), yet that's still todo. wsnic's job here is reduced to network setup and WebSocket handling (handshake, ping/pong, and the binary frames just contain plain ethernet frames).

The third one almost works and should perform a bit better than the second one. One end of the veth pair is connected to the bridge, and wsnic connects to the other end using a packet socket (AF_PACKET).

The fourth one is a special case, from what I understood it should behave like another NIC of the host (fullly integrated into the host's network, for example automatically using the same DHCP server as the host), but requires support in your external router hardware (so-called "hairpinning"), it's unclear to me how well supported that is. I tested it in two very different physical networks, macvtap doesn't work in my home setup (my router is a FritzBox) but kinda works in the other. So that one is still work-in-progress.

A few other notes:

• A bit to my surprise, the performance in Firefox and Chrome is indistinguishable, both max out at around 35Mb/s.
• The guest's TCP/IP stack matters! While brtap works with buildroot it does not with FreeDOS (have to look into that).
• EDIT: For some reason, FreeDOS does not reply to pings, even if I ping one FreeDOS instance from another (everything else works).
• Things I test with the backends: vm-to-network communication (ARP, DHCP), vm-to-vm (ping, socat), vm-to-host, vm-to-intranet and vm-to-internet
• DHCP/DNS is provided by dnsmasq or by an wsnic-internal handler (DHCP only).
• I still want to look into traffic shaping in the Linux kernel to see if I can throttle connections this way
• I also plan to release this as a Docker image

So there's still some work left to do, can't say how long it will take.

Any input, specifically about virtual networking in Linux in this context, is very welcome. Useful documentation about this is hard to come by it seems. Like, on Google I found half a dozen or so similar sounding blog articles but with the exact same shell instructions, and then I found the issue on stackoverflow from which all of them seem to have been autogenerated, Yikes!