[feature request] add --tunnelserver-addr for yurtctl convert comand

[adamzhoul]

What would you like to be added:

yurtcl convert --tunnelserver-addr xxxx

Why is this needed:

1. --tunnelserver-addr is supported at yurtcl convert edgenode
2. yurtcl convert use job to run yurtctl convert edgenode. However, job template hardcode command line without this

openyurt/pkg/yurtctl/constants/constants.go

Line 278 in 47a88ea

- "cp /usr/local/bin/yurtctl /tmp && nsenter -t 1 -m -u -n -i -- /var/tmp/yurtctl convert edgenode --yurthub-image {{.yurthub_image}} {{if .yurthub_healthcheck_timeout}}--yurthub-healthcheck-timeout {{.yurthub_healthcheck_timeout}} {{end}}--join-token {{.joinToken}} && rm /tmp/yurtctl"

if not set, tunnel-agent get nodeIp、clusterIp、127.0.0.1。and choose to use the first nodeIp as tunnel-server Ip
However, node Ip is unreachable. When we edge node and cloud node are isolated networks.

So, when we first use yurtctl convert command, we shall get everything done.

others
/kind feature

[rambohe-ch]

What would you like to be added:

yurtcl convert --tunnelserver-addr xxxx

Why is this needed:

1. --tunnelserver-addr is supported at yurtcl convert edgenode
2. yurtcl convert use job to run yurtctl convert edgenode. However, job template hardcode command line without this

openyurt/pkg/yurtctl/constants/constants.go

Line 278 in 47a88ea

- "cp /usr/local/bin/yurtctl /tmp && nsenter -t 1 -m -u -n -i -- /var/tmp/yurtctl convert edgenode --yurthub-image {{.yurthub_image}} {{if .yurthub_healthcheck_timeout}}--yurthub-healthcheck-timeout {{.yurthub_healthcheck_timeout}} {{end}}--join-token {{.joinToken}} && rm /tmp/yurtctl"

if not set, tunnel-agent get nodeIp、clusterIp、127.0.0.1。and choose to use the first nodeIp as tunnel-server Ip
However, node Ip is unreachable. When we edge node and cloud node are isolated networks.

So, when we first use yurtctl convert command, we shall get everything done.

others
/kind feature

@adamzhoul Thanks for raising issue. I think that --tunnelserver-addr is used by yurt-tunnel-agent component, but yurt-tunnel-agent will get address of yurt-tunnel-server when startup, user do not need to configure the address for tunnel-agent. so it's not need to add the --tunnelserver-addr parameter for yurtctl.

[adamzhoul]

thanks @rambohe-ch

As I mentioned here:

if not set, tunnel-agent get nodeIp、clusterIp、127.0.0.1。and choose to use the first nodeIP as tunnel-server Ip
However, node Ip is unreachable. When we edge node and cloud node are isolated networks.

the IP I get is node IP, which is unreachable.
so, My advice, for now, is to pass an IP that can be reached from an isolated network.

And this is only for discussion, to solve the isolated network situation.
Another better solution is very welcome.

[rambohe-ch]

thanks @rambohe-ch

As I mentioned here:

if not set, tunnel-agent get nodeIp、clusterIp、127.0.0.1。and choose to use the first nodeIP as tunnel-server Ip
However, node Ip is unreachable. When we edge node and cloud node are isolated networks.

the IP I get is node IP, which is unreachable.
so, My advice, for now, is to pass an IP that can be reached from an isolated network.

And this is only for discussion, to solve the isolated network situation.
Another better solution is very welcome.

@adamzhoul Thanks for your feedback.

Now, I have understood your scenario, and yurt-tunnel-agent can support configure tunnel-server address, It's reasonable to add an --tunnelserver-addr parameter for yurtctl for configuring address.

[adamzhoul]

great. I can work on this. @rambohe-ch

[rambohe-ch]

/assign @adamzhoul

[adamzhoul]

hi @rambohe-ch
I meet a problem here.

Background:

1. Cloud-Network is isolated with Edge-Network
2. tunnel-agent can't connect to the tunnel-server node directly. 
3. So we apply a public IP and bind it to the master node. (with 6443 port accessible. so visiting APIserver from the edge is ok.)

I want to use yurtctl convert —tunnelserver-addr ip:port. so tunnel-agent can visit the public IP.
However, the tunnel-server svc type is nodePort.
so, the problems are:

1. I know the IP, but the port is not. 
2. public IP can not open all ports accessible, must be limited to some port.

what do you think about all this？
thanks.

[adamzhoul]

PS:
this leads to an important question. How do we manage traffics between edge and cloud?

 1. now yurtHub connects to APIServer directly
 2. yurt-tunnel is used for APIServer to connect kubelet… . more like manager traffic control.
 3. what’s the underlay traffic policy? how pod to pod, pod to node, svc, etc. do we need another cni?

[rambohe-ch]

hi @rambohe-ch
I meet a problem here.

Background:

1. Cloud-Network is isolated with Edge-Network
2. tunnel-agent can't connect to the tunnel-server node directly. 
3. So we apply a public IP and bind it to the master node. (with 6443 port accessible. so visiting APIserver from the edge is ok.)

I want to use yurtctl convert —tunnelserver-addr ip:port. so tunnel-agent can visit the public IP.
However, the tunnel-server svc type is nodePort.
so, the problems are:

1. I know the IP, but the port is not. 
2. public IP can not open all ports accessible, must be limited to some port.

what do you think about all this？
thanks.

@adamzhoul It's a good question. In my opinion, we need to define the custom nodePort for x-tunnel-server-svc service when x-tunnel-server-svc is a NodePort service. of course, we need to define nodePort in the range: 30000~32767:

spec:
    type: NodePort
    ports:
        - port: 10262
          nodePort: 31008
          targetPort: 10262
          name: tcp

so we can open the fixed nodePort on the public IP.

[rambohe-ch]

PS:
this leads to an important question. How do we manage traffics between edge and cloud?

 1. now yurtHub connects to APIServer directly
 2. yurt-tunnel is used for APIServer to connect kubelet… . more like manager traffic control.
 3. what’s the underlay traffic policy? how pod to pod, pod to node, svc, etc. do we need another cni?

@adamzhoul At present, yurt-tunnel is used to proxy monitoring traffic like kubectl logs/exec or prometheus pull metrics at edge.

we have no solution to manage business pods traffic between cloud and edge. but now we have some ideas to manage business pods traffic between different regions. for example: cloud-edge, or edge-edge, maybe we will discuss these ideas in the later OpenYurt community meeting.

[adamzhoul]

using fixed nodePort is great.

besides:
I read the data-filtering-framework proposal. It’s a great idea that considers yurthub as a sidecar.
so comes with some points:

1. use yurthub to redirect traffic to the right IP and port in the future?
2. if yurthub is a sidecar, we say data plane. Then, we can create a control plane, to control it. 
     like how to redirect IP ? how to filter data?

[rambohe-ch]

using fixed nodePort is great.

besides:
I read the data-filtering-framework proposal. It’s a great idea that considers yurthub as a sidecar.
so comes with some points:

1. use yurthub to redirect traffic to the right IP and port in the future?
2. if yurthub is a sidecar, we say data plane. Then, we can create a control plane, to control it. 
     like how to redirect IP ? how to filter data?

@adamzhoul It's a good idea to add control plane for node sidecar, maybe we can discuss these issues at the OpenYurt community meeting.