in-kernel PM: listen socket: support "behind a NAT" use case #337
Comments
|
Hi @shockx2 Do you know if your GST server close the listening socket after having accepted the first connection? (e.g. do you have to relaunch the server after the client got disconnected?) For MPTCP, we need to have a socket listening to accept more subflows. If it is the case and if you cannot modify your app, a way to work around this is to have another app doing a import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_MPTCP)
s.bind(("10.0.2.2", 40000))
s.listen(1)(Mmh, but that's the same IP as the initial subflow, the server only has one IP :-/) |
|
strace shows that gst-launch-1.0 closes the listener socket just after accepting the first subflow. As guess by Mat, that is the root cause of the failure. An alternative workaround, beyond the already proposed ones would be adding on the server side a port-based endpoint: ip mptcp endpoint add 10.0.2.2 port 12345 signal set the 'fullmesh' flag on the client endpoints, and increases max subflow limit: ip mptcp limit set subflows 4 Side note: could you please share your routing configuration, too? e.g. the output of ip addr; ip route on both the client and the server. |
|
Very thank you. @matttbe @pabeni
I try the solution to AWS EC2 instance(Server), and my local PC(Client). But, 2nd endpoint advertising is not working. @pabeni
Thank you. |
@shockx2 you need then to announce the public IP to be reachable from the client side. Mmh yes, the in-kernel PM doesn't create a new socket with "freebind". This could be changed I suppose. While trying workarounds, can you first try to add the public IP to the loopback interface? Then add the new endpoint using the public IP? You will need to add an IP rule and route to use the right interface when the source IP is the one linked to the "second" interface https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting Something like this I suppose: But maybe you will need to has a SNAT rule to change public IP to source IP... :-) |
|
Thank you, very much @matttbe I am understanding that the "freebind" means that it makes PM can announces public IP to client. right? I think another workaround that the announcing packet modification by eBPF to send public IP. Thank you. |
It would allow the kernel to create a listening socket on an IP it doesn't own. It could be needed in some cases but in yours, that will partly help you:
I see two solutions (that could be combined) for your case (being a NAT and an app closing the listening socket after the
(@pabeni: what do you think?) In both cases, it means you will have to create the listening socket, e.g. with Python: import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 262)
s.bind(("<private IP>", <port>))
s.listen(1024)
while True:
ss = s.accept()
print(ss)
ss[0].close()
It is not easy because there is an HMAC to avoid having this address being modified by someone else (or "injected"). With the current situation, I think the best is either to:
Does it work for you? |
both option will fail ?!? if the listener socket is not created, and the server closes the port after connect, I don't see how the subflow created by the client could fail. Since this thing looks very specific to NAT, and the admin need to know the NAT details (exposed address, local address) I think we could change the in-kernel PM to listen on a different address other then the signaled one, something alike:
|
|
side note: the client routing configuration looks broken. Specifically I don't see how the client could connect from h1-eth1/10.0.1.2 towards the server, since it lacks a suitable route. e.g.
from the client should fail - while it should be successful from h1-eth0 |
|
@pabeni thank you for your reply!
Yes indeed. But it is possible to work around this issue by creating this listening socket as suggested with the Python code, no?
Indeed, it would be cleaner and clearer. I can update the ticket to switch to "feature request" |
matttbe
changed the title
Thank you @matttbe
Client's wireshark |
|
Hello,
If your app doesn't close the listening socket, you don't need the workaround that force creating a listening socket. All you need is to add the
You should keep 127.0.0.1 but add a new one.
Yes but the listening socket will only listen on the public IP. But the server will receive the packet modified by the NAT: with the private IP.
Do you have the packet trace instead? (export the trace and zip it to join it here) |
Puranjay Mohan says: ==================== arm32, bpf: add support for cpuv4 insns Changes in V2 -> V3 - Added comments at places where there could be confustion. - In the patch for DIV64, fix the if-else case that would never run. - In the same patch use a single instruction to POP caller saved regs. - Add a patch to change maintainership of ARM32 BPF JIT. Changes in V1 -> V2: - Fix coding style issues. - Don't use tmp variable for src in emit_ldsx_r() as it is redundant. - Optimize emit_ldsx_r() when offset can fit in immediate. Add the support for cpuv4 instructions for ARM32 BPF JIT. 64-bit division was not supported earlier so this series adds 64-bit DIV, SDIV, MOD, SMOD instructions as well. This series needs any one of the patches from [1] to disable zero-extension for BPF_MEMSX to support ldsx. The relevant selftests have passed expect ldsx_insn which needs fentry: Tested on BeagleBone Black (ARMv7-A): [root@alarm del]# echo 1 > /proc/sys/net/core/bpf_jit_enable [root@alarm del]# ./test_progs -a verifier_sdiv,verifier_movsx,verifier_ldsx,verifier_gotol,verifier_bswap #337/1 verifier_bswap/BSWAP, 16:OK #337/2 verifier_bswap/BSWAP, 16 @unpriv:OK #337/3 verifier_bswap/BSWAP, 32:OK #337/4 verifier_bswap/BSWAP, 32 @unpriv:OK #337/5 verifier_bswap/BSWAP, 64:OK #337/6 verifier_bswap/BSWAP, 64 @unpriv:OK #337 verifier_bswap:OK #351/1 verifier_gotol/gotol, small_imm:OK #351/2 verifier_gotol/gotol, small_imm @unpriv:OK #351 verifier_gotol:OK #359/1 verifier_ldsx/LDSX, S8:OK #359/2 verifier_ldsx/LDSX, S8 @unpriv:OK #359/3 verifier_ldsx/LDSX, S16:OK #359/4 verifier_ldsx/LDSX, S16 @unpriv:OK #359/5 verifier_ldsx/LDSX, S32:OK #359/6 verifier_ldsx/LDSX, S32 @unpriv:OK #359/7 verifier_ldsx/LDSX, S8 range checking, privileged:OK #359/8 verifier_ldsx/LDSX, S16 range checking:OK #359/9 verifier_ldsx/LDSX, S16 range checking @unpriv:OK #359/10 verifier_ldsx/LDSX, S32 range checking:OK #359/11 verifier_ldsx/LDSX, S32 range checking @unpriv:OK #359 verifier_ldsx:OK #370/1 verifier_movsx/MOV32SX, S8:OK #370/2 verifier_movsx/MOV32SX, S8 @unpriv:OK #370/3 verifier_movsx/MOV32SX, S16:OK #370/4 verifier_movsx/MOV32SX, S16 @unpriv:OK #370/5 verifier_movsx/MOV64SX, S8:OK #370/6 verifier_movsx/MOV64SX, S8 @unpriv:OK #370/7 verifier_movsx/MOV64SX, S16:OK #370/8 verifier_movsx/MOV64SX, S16 @unpriv:OK #370/9 verifier_movsx/MOV64SX, S32:OK #370/10 verifier_movsx/MOV64SX, S32 @unpriv:OK #370/11 verifier_movsx/MOV32SX, S8, range_check:OK #370/12 verifier_movsx/MOV32SX, S8, range_check @unpriv:OK #370/13 verifier_movsx/MOV32SX, S16, range_check:OK #370/14 verifier_movsx/MOV32SX, S16, range_check @unpriv:OK #370/15 verifier_movsx/MOV32SX, S16, range_check 2:OK #370/16 verifier_movsx/MOV32SX, S16, range_check 2 @unpriv:OK #370/17 verifier_movsx/MOV64SX, S8, range_check:OK #370/18 verifier_movsx/MOV64SX, S8, range_check @unpriv:OK #370/19 verifier_movsx/MOV64SX, S16, range_check:OK #370/20 verifier_movsx/MOV64SX, S16, range_check @unpriv:OK #370/21 verifier_movsx/MOV64SX, S32, range_check:OK #370/22 verifier_movsx/MOV64SX, S32, range_check @unpriv:OK #370/23 verifier_movsx/MOV64SX, S16, R10 Sign Extension:OK #370/24 verifier_movsx/MOV64SX, S16, R10 Sign Extension @unpriv:OK #370 verifier_movsx:OK #382/1 verifier_sdiv/SDIV32, non-zero imm divisor, check 1:OK #382/2 verifier_sdiv/SDIV32, non-zero imm divisor, check 1 @unpriv:OK #382/3 verifier_sdiv/SDIV32, non-zero imm divisor, check 2:OK #382/4 verifier_sdiv/SDIV32, non-zero imm divisor, check 2 @unpriv:OK #382/5 verifier_sdiv/SDIV32, non-zero imm divisor, check 3:OK #382/6 verifier_sdiv/SDIV32, non-zero imm divisor, check 3 @unpriv:OK #382/7 verifier_sdiv/SDIV32, non-zero imm divisor, check 4:OK #382/8 verifier_sdiv/SDIV32, non-zero imm divisor, check 4 @unpriv:OK #382/9 verifier_sdiv/SDIV32, non-zero imm divisor, check 5:OK #382/10 verifier_sdiv/SDIV32, non-zero imm divisor, check 5 @unpriv:OK #382/11 verifier_sdiv/SDIV32, non-zero imm divisor, check 6:OK #382/12 verifier_sdiv/SDIV32, non-zero imm divisor, check 6 @unpriv:OK #382/13 verifier_sdiv/SDIV32, non-zero imm divisor, check 7:OK #382/14 verifier_sdiv/SDIV32, non-zero imm divisor, check 7 @unpriv:OK #382/15 verifier_sdiv/SDIV32, non-zero imm divisor, check 8:OK #382/16 verifier_sdiv/SDIV32, non-zero imm divisor, check 8 @unpriv:OK #382/17 verifier_sdiv/SDIV32, non-zero reg divisor, check 1:OK #382/18 verifier_sdiv/SDIV32, non-zero reg divisor, check 1 @unpriv:OK #382/19 verifier_sdiv/SDIV32, non-zero reg divisor, check 2:OK #382/20 verifier_sdiv/SDIV32, non-zero reg divisor, check 2 @unpriv:OK #382/21 verifier_sdiv/SDIV32, non-zero reg divisor, check 3:OK #382/22 verifier_sdiv/SDIV32, non-zero reg divisor, check 3 @unpriv:OK #382/23 verifier_sdiv/SDIV32, non-zero reg divisor, check 4:OK #382/24 verifier_sdiv/SDIV32, non-zero reg divisor, check 4 @unpriv:OK #382/25 verifier_sdiv/SDIV32, non-zero reg divisor, check 5:OK #382/26 verifier_sdiv/SDIV32, non-zero reg divisor, check 5 @unpriv:OK #382/27 verifier_sdiv/SDIV32, non-zero reg divisor, check 6:OK #382/28 verifier_sdiv/SDIV32, non-zero reg divisor, check 6 @unpriv:OK #382/29 verifier_sdiv/SDIV32, non-zero reg divisor, check 7:OK #382/30 verifier_sdiv/SDIV32, non-zero reg divisor, check 7 @unpriv:OK #382/31 verifier_sdiv/SDIV32, non-zero reg divisor, check 8:OK #382/32 verifier_sdiv/SDIV32, non-zero reg divisor, check 8 @unpriv:OK #382/33 verifier_sdiv/SDIV64, non-zero imm divisor, check 1:OK #382/34 verifier_sdiv/SDIV64, non-zero imm divisor, check 1 @unpriv:OK #382/35 verifier_sdiv/SDIV64, non-zero imm divisor, check 2:OK #382/36 verifier_sdiv/SDIV64, non-zero imm divisor, check 2 @unpriv:OK #382/37 verifier_sdiv/SDIV64, non-zero imm divisor, check 3:OK #382/38 verifier_sdiv/SDIV64, non-zero imm divisor, check 3 @unpriv:OK #382/39 verifier_sdiv/SDIV64, non-zero imm divisor, check 4:OK #382/40 verifier_sdiv/SDIV64, non-zero imm divisor, check 4 @unpriv:OK #382/41 verifier_sdiv/SDIV64, non-zero imm divisor, check 5:OK #382/42 verifier_sdiv/SDIV64, non-zero imm divisor, check 5 @unpriv:OK #382/43 verifier_sdiv/SDIV64, non-zero imm divisor, check 6:OK #382/44 verifier_sdiv/SDIV64, non-zero imm divisor, check 6 @unpriv:OK #382/45 verifier_sdiv/SDIV64, non-zero reg divisor, check 1:OK #382/46 verifier_sdiv/SDIV64, non-zero reg divisor, check 1 @unpriv:OK #382/47 verifier_sdiv/SDIV64, non-zero reg divisor, check 2:OK #382/48 verifier_sdiv/SDIV64, non-zero reg divisor, check 2 @unpriv:OK #382/49 verifier_sdiv/SDIV64, non-zero reg divisor, check 3:OK #382/50 verifier_sdiv/SDIV64, non-zero reg divisor, check 3 @unpriv:OK #382/51 verifier_sdiv/SDIV64, non-zero reg divisor, check 4:OK #382/52 verifier_sdiv/SDIV64, non-zero reg divisor, check 4 @unpriv:OK #382/53 verifier_sdiv/SDIV64, non-zero reg divisor, check 5:OK #382/54 verifier_sdiv/SDIV64, non-zero reg divisor, check 5 @unpriv:OK #382/55 verifier_sdiv/SDIV64, non-zero reg divisor, check 6:OK #382/56 verifier_sdiv/SDIV64, non-zero reg divisor, check 6 @unpriv:OK #382/57 verifier_sdiv/SMOD32, non-zero imm divisor, check 1:OK #382/58 verifier_sdiv/SMOD32, non-zero imm divisor, check 1 @unpriv:OK #382/59 verifier_sdiv/SMOD32, non-zero imm divisor, check 2:OK #382/60 verifier_sdiv/SMOD32, non-zero imm divisor, check 2 @unpriv:OK #382/61 verifier_sdiv/SMOD32, non-zero imm divisor, check 3:OK #382/62 verifier_sdiv/SMOD32, non-zero imm divisor, check 3 @unpriv:OK #382/63 verifier_sdiv/SMOD32, non-zero imm divisor, check 4:OK #382/64 verifier_sdiv/SMOD32, non-zero imm divisor, check 4 @unpriv:OK #382/65 verifier_sdiv/SMOD32, non-zero imm divisor, check 5:OK #382/66 verifier_sdiv/SMOD32, non-zero imm divisor, check 5 @unpriv:OK #382/67 verifier_sdiv/SMOD32, non-zero imm divisor, check 6:OK #382/68 verifier_sdiv/SMOD32, non-zero imm divisor, check 6 @unpriv:OK #382/69 verifier_sdiv/SMOD32, non-zero reg divisor, check 1:OK #382/70 verifier_sdiv/SMOD32, non-zero reg divisor, check 1 @unpriv:OK #382/71 verifier_sdiv/SMOD32, non-zero reg divisor, check 2:OK #382/72 verifier_sdiv/SMOD32, non-zero reg divisor, check 2 @unpriv:OK #382/73 verifier_sdiv/SMOD32, non-zero reg divisor, check 3:OK #382/74 verifier_sdiv/SMOD32, non-zero reg divisor, check 3 @unpriv:OK #382/75 verifier_sdiv/SMOD32, non-zero reg divisor, check 4:OK #382/76 verifier_sdiv/SMOD32, non-zero reg divisor, check 4 @unpriv:OK #382/77 verifier_sdiv/SMOD32, non-zero reg divisor, check 5:OK #382/78 verifier_sdiv/SMOD32, non-zero reg divisor, check 5 @unpriv:OK #382/79 verifier_sdiv/SMOD32, non-zero reg divisor, check 6:OK #382/80 verifier_sdiv/SMOD32, non-zero reg divisor, check 6 @unpriv:OK #382/81 verifier_sdiv/SMOD64, non-zero imm divisor, check 1:OK #382/82 verifier_sdiv/SMOD64, non-zero imm divisor, check 1 @unpriv:OK #382/83 verifier_sdiv/SMOD64, non-zero imm divisor, check 2:OK #382/84 verifier_sdiv/SMOD64, non-zero imm divisor, check 2 @unpriv:OK #382/85 verifier_sdiv/SMOD64, non-zero imm divisor, check 3:OK #382/86 verifier_sdiv/SMOD64, non-zero imm divisor, check 3 @unpriv:OK #382/87 verifier_sdiv/SMOD64, non-zero imm divisor, check 4:OK #382/88 verifier_sdiv/SMOD64, non-zero imm divisor, check 4 @unpriv:OK #382/89 verifier_sdiv/SMOD64, non-zero imm divisor, check 5:OK #382/90 verifier_sdiv/SMOD64, non-zero imm divisor, check 5 @unpriv:OK #382/91 verifier_sdiv/SMOD64, non-zero imm divisor, check 6:OK #382/92 verifier_sdiv/SMOD64, non-zero imm divisor, check 6 @unpriv:OK #382/93 verifier_sdiv/SMOD64, non-zero imm divisor, check 7:OK #382/94 verifier_sdiv/SMOD64, non-zero imm divisor, check 7 @unpriv:OK #382/95 verifier_sdiv/SMOD64, non-zero imm divisor, check 8:OK #382/96 verifier_sdiv/SMOD64, non-zero imm divisor, check 8 @unpriv:OK #382/97 verifier_sdiv/SMOD64, non-zero reg divisor, check 1:OK #382/98 verifier_sdiv/SMOD64, non-zero reg divisor, check 1 @unpriv:OK #382/99 verifier_sdiv/SMOD64, non-zero reg divisor, check 2:OK #382/100 verifier_sdiv/SMOD64, non-zero reg divisor, check 2 @unpriv:OK #382/101 verifier_sdiv/SMOD64, non-zero reg divisor, check 3:OK #382/102 verifier_sdiv/SMOD64, non-zero reg divisor, check 3 @unpriv:OK #382/103 verifier_sdiv/SMOD64, non-zero reg divisor, check 4:OK #382/104 verifier_sdiv/SMOD64, non-zero reg divisor, check 4 @unpriv:OK #382/105 verifier_sdiv/SMOD64, non-zero reg divisor, check 5:OK #382/106 verifier_sdiv/SMOD64, non-zero reg divisor, check 5 @unpriv:OK #382/107 verifier_sdiv/SMOD64, non-zero reg divisor, check 6:OK #382/108 verifier_sdiv/SMOD64, non-zero reg divisor, check 6 @unpriv:OK #382/109 verifier_sdiv/SMOD64, non-zero reg divisor, check 7:OK #382/110 verifier_sdiv/SMOD64, non-zero reg divisor, check 7 @unpriv:OK #382/111 verifier_sdiv/SMOD64, non-zero reg divisor, check 8:OK #382/112 verifier_sdiv/SMOD64, non-zero reg divisor, check 8 @unpriv:OK #382/113 verifier_sdiv/SDIV32, zero divisor:OK #382/114 verifier_sdiv/SDIV32, zero divisor @unpriv:OK #382/115 verifier_sdiv/SDIV64, zero divisor:OK #382/116 verifier_sdiv/SDIV64, zero divisor @unpriv:OK #382/117 verifier_sdiv/SMOD32, zero divisor:OK #382/118 verifier_sdiv/SMOD32, zero divisor @unpriv:OK #382/119 verifier_sdiv/SMOD64, zero divisor:OK #382/120 verifier_sdiv/SMOD64, zero divisor @unpriv:OK #382 verifier_sdiv:OK Summary: 5/163 PASSED, 0 SKIPPED, 0 FAILED As the selftests don't compile for 32-bit architectures without modifications due to long being 32-bit, I have added new tests to lib/test_bpf.c for cpuv4 insns, all are passing: test_bpf: Summary: 1052 PASSED, 0 FAILED, [891/1040 JIT'ed] test_bpf: test_tail_calls: Summary: 10 PASSED, 0 FAILED, [10/10 JIT'ed] test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED [1] https://lore.kernel.org/all/mb61p5y4u3ptd.fsf@amazon.com/ ==================== Link: https://lore.kernel.org/r/20230907230550.1417590-1-puranjay12@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a test case which replaces an active ingress qdisc while keeping the miniq in-tact during the transition period to the new clsact qdisc. # ./vmtest.sh -- ./test_progs -t tc_link [...] ./test_progs -t tc_link [ 3.412871] bpf_testmod: loading out-of-tree module taints kernel. [ 3.413343] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel #332 tc_links_after:OK #333 tc_links_append:OK #334 tc_links_basic:OK #335 tc_links_before:OK #336 tc_links_chain_classic:OK #337 tc_links_chain_mixed:OK #338 tc_links_dev_chain0:OK #339 tc_links_dev_cleanup:OK #340 tc_links_dev_mixed:OK #341 tc_links_ingress:OK #342 tc_links_invalid:OK #343 tc_links_prepend:OK #344 tc_links_replace:OK #345 tc_links_revision:OK Summary: 14/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Cc: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20240708133130.11609-2-daniel@iogearbox.net Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Currently, if the server is behind a NAT/LB/..., adding an endpoint with a "public" (exposed) IP and a custom port would fail because the in-kernel PM will try to create a listening socket with an IP not allocated on the system:
-99: Cannot assign requested address.It is possible to work around that by adding the exposed IP on the loopback interface but the listen socket created by the in-kernel PM will be useless as it will bind on the "public" (exposed) IP, not the "private" (internal) one.
A solution could be to extend the current API to allow something like:
listenwill needport(andsignal).(
no-listenflag could also be use not to create a listening socket automatically when a port is given)Original bug report:
Hello
I'm very interested in all about MPTCP. Thank you MPTCP team.
And, I experiment to apply MPTCP for video streaming.
The simple command is proof of concept of availability of MPTCP for streaming. But, MPTCP is fallback to single TCP.
Other applications e.g. iperf3 that works very well.
Environments
Host A: MPTCP client (linux v6.2.0-rc4)
10.0.0.2
10.0.1.2
ip mptcp endpoint
10.0.0.2 id 1 subflow dev h1-eth0
10.0.1.2 id 2 subflow dev h1-eth1
Host B: MPTCP server (linux v6.2.0-rc4)
10.0.2.2
Command:
Host A(Client):
mptcpize run -d gst-launch-1.0 videotestsrc ! tcpclientsink host=10.0.2.2 port=40000Host B(Server):
mptcpize run -d gst-launch-1.0 tcpserversrc host=0.0.0.0 port=40000 ! filesink location=./outputRun Host B first, and Host A later. Then the server saves output file for receiving data.
But, MPTCP is not working. second subflow is reset.
Result: wireshark
And iperf3 result below
Command:
Host B:
mptcpize run -d iperf3 -sHost A:
mptcpize run -d iperf3 -c 10.0.2.2Result: It works well.
Thank you again!!
The text was updated successfully, but these errors were encountered: