Support for mixed windows and linux k8s clusters

[campbelldgunn]

I have searched through the documentation for how support Istio on mixed window and linux node clusters. Right now, I will use the nodeSelector in helm chart to install initially on linux. But not what the process is for windows nodes?

Thanks in Advance.

[campbelldgunn]

@ZackButcher any hints or advice on this or someone else? Greetings from when we me at qcon 2017 too.

[rshriram]

The Istio control plane could be easily built for windows. However, Envoy will still need a linux platform. Are you looking at windows based containers or just linux containers running on windows platform? If its latter, it should work. If former, this is more of an Envoy question. @mattklein123 might have some updates on the Microsoft folks who were looking to build Envoy on windows.

[mattklein123]

NT support is tracked here: envoyproxy/envoy#129.

@vturecek should be able to provide an update.

[campbelldgunn]

Thanks to all for replying. I am surprise that someone has not asked about this earlier.

So, it is a mixed cluster. That is there are master nodes on linux, a linux pool (10 linux nodes) and then a windows pool (3 windows nodes). I would install istio on the linux pool, and I am running windows containers on the windows nodes. My question if possible to have sidecar support for the windows pods on the windows nodes.

[ZackButcher]

You'll need an Envoy build/container for windows, and if you're doing automatic sidecar injection via the initializer, you'll need to run two initializers: one for the linux side of things injecting the normal Istio Envoy image, and second for the Windows side of things injecting the Windows Envoy image.

We might need to add some more initializer config to let it control which pods it attempts to initialize more finely.

[campbelldgunn]

Thanks @ZackButcher. Yeah, hence my other issue #925 using a nodeSelector option works well with on services I put on to the mixed cluster.

So, having the following options:

1. The pilot, mixer and auth at the control plane can be controlled by the nodeSelector, and
2. Based on what you said above, is to use the nodeSelector and have the appropriate sidecar be injected, either for windows or linux.

[ZackButcher]

Yup, that'd work.

[rshriram]

Please reopen if Zack's response did not solve your issue

[iampluque]

Thanks @ZackButcher. Yeah, hence my other issue #925 using a nodeSelector option works well with on services I put on to the mixed cluster.

So, having the following options:

2. The pilot, mixer and auth at the control plane can be controlled by the nodeSelector, and
3. Based on what you said above, is to use the nodeSelector and have the appropriate sidecar be injected, either for windows or linux.

Zack, what sidecar proxy are you using for your Windows pod?

We currently have a mixed-os cluster but I am not able to figure out what I should use!

Thanks

[masaeedu]

@ZackButcher Could you elaborate on what "the initializer" refers to, and how we can run several of them? Is this something like an admission controller that we manually create a tweaked copy of?

I was following the Getting Started guide here: https://istio.io/latest/docs/setup/getting-started/ and am having trouble seeing how I would tweak kubectl label namespace default istio-injection=enabled to get the system to inject sidecars with images appropriate to each node.

[masaeedu]

@pluqueTheLuxe I haven't been able to get any of this to work, so take this with a grain of salt, but I would imagine the sidecar to use would be: https://hub.docker.com/r/envoyproxy/envoy-windows-dev (although I believe this didn't exist at the time you posted your comment).

[iampluque]

@pluqueTheLuxe I haven't been able to get any of this to work, so take this with a grain of salt, but I would imagine the sidecar to use would be: https://hub.docker.com/r/envoyproxy/envoy-windows-dev (although I believe this didn't exist at the time you posted your comment).

hahahaha @masaeedu, I no longer run in mixed mode, we switch all our platform to linux. Fixed :P

[kevinmatthews-kpmg]

I am experiencing the same issues, have a windows node pool with windows containers and a linux node pool with linux containers. THe windows node pool is required as we have some legacy .net framework applications. Ive had to remove the istio-injection label from the namespace so that I can deploy the applications.

I take it this means I lose the mTLS, any egress rules etc... it looks like the envoy proxy is in beta now (not alpha) is there anymore regarding support in istio?

[ysma500]

Is there news on this subject as of 2022?