Che 7, TLS and self signed certs

[ghost]

There are multiple problems:

1. Failed to start a Che 7 workspace if Che deployment is configured to use self signed cert

2019-02-07 18:23:32,549[aceSharedPool-1]  [WARN ] [.i.k.KubernetesInternalRuntime 249]  - 
Failed to start Kubernetes runtime of workspace workspaceq1ys0pxhpfgzgmu8. 
Cause: Plugins installation process failed. Error: Unrecoverable event occurred: 
'FailedMount', 'MountVolume.SetUp failed for volume "che-self-signed-cert" : secret 
"che-self-signed-cert" not found', 'workspaceq1ys0pxhpfgzgmu8.che-plugin-broker'

A secret with cert body isn't created but che-plugin-broker pod is configured to use it

2. Once Update DTD for GWT-module descriptors #1 is fixed I expect that che-plugin-broker isn't aware of such a cert and will fail to communicate with master using tls route/ingress.

3. Once Update DTD for GWT-module descriptors #1 and Package docker runner in Che #2 is solved, Theia server side will be the next suspect since Theia communicates with Che master to grab workspace config and other info. So, Theia should also use the cert or trust all insecure endpoints.

@slemeur @l0rd is this smth that should be taken care of for after beta Che 7 releases?

[slemeur]

That sounds like a bug/regression.
We should be looking at this and testing it for the beta.

[skabashnyuk]

@slemeur it's hard to treat this as a bug because we didn't have this feature before for Che7. All activity we did relate to TLS and self-signed certs were made for che master and GWT-based IDE.

[skabashnyuk]

CC @evidolob @benoitf @garagatyi @ibuziuk

[benoitf]

hello, would let's encrypt help ?

Because with let's encrypt I don't see anymore many ppl using self-signed certificates.
if you really want to setup che with https then you've probably your own domain. In development mode you're probably using nip.io and there is some rate limits on this highly used domain by let's encrypt

[ghost]

@benoitf so you propose not to have support for self signed certs?

Let's encrypt requires a public dns - this isn't always the case for many.

[l0rd]

@eivantsov I see it like a requirement for Che 7 GA, not beta. @benoitf self signed certs were considered critical for Che 6 because we considered that let's encrypt doesn't work for every use case.

[johnmcollier]

Just wondering if there's any updates on this? Is a fix for this targeted for the Che 7 GA? Hitting the same issue on my cluster with self-signed certs:

Error: Failed to run the workspace: "Plugins installation process failed. Error: Unrecoverable event occurred: 'FailedMount', 'MountVolume.SetUp failed for volume "che-self-signed-cert" : secret "workspacek7av6dqw93udgvtw-che-self-signed-cert" not found', 'workspacek7av6dqw93udgvtw.che-plugin-broker'"

[bbalakriz]

+1. Any updates on this would be helpful. Thanks!

[gorkem]

@l0rd Is this one part of GA plan?

[l0rd]

@gorkem it wasn't actually. I have added that to the GA list. Still wondering what we need to do to fix this. Will try to make a list here:

• wsmaster/keycloak side should be already addressed
• fix the issue faced by @johnmcollier: volume created but the secret with the self-signed cert is not Volume created but the secret with the self-signed cert is not #13433
• make sure that che-theia trusts the self-signed cert Che 7, TLS and self signed certs #12634
• make sure that che-plugin-broker trusts the self-signed cert

@skabashnyuk @benoitf @sleshchenko please review this list and comment if I am missing something

@johnmcollier can you provide more details to reproduce your problem? To setup the self-signed cert have you followed Che 6 documentation? With what stack are you testing?

[ghost]

@l0rd stack does not matter here. Just deploy Che in TLS mode with support of self signed certs. Installer script will create a secret self-signed-certificate with cert content. And in case of Che 7 secret for a workspace isn't created for plugin broker pod. As a result it cannot be scheduled.

[l0rd]

@vparfonov @skabashnyuk has one of you taken this issue in your sprint?

[skabashnyuk]

@l0rd . It's something that has to be done for che-theia and che-plugin-broker. I guessed @evidolob or @ibuziuk can better tell the status.

[l0rd]

@skabashnyuk this issue is labelled team/platform isn't it? And the error faced by @johnmcollier happens before the plugin-btoker and che-theia are involved.

[skabashnyuk]

this issue is labelled team/platform isn't it

That is correct. As well as osio and ide2 since they know about che-theia and plugin-brocker packaging and architecture more

And the error faced by @johnmcollier happens before the plugin-btoker and che-theia are involved.

The issue says that che-theia and plugin-brocker processes have to be taught to work with self-signed certificates.

Do you want us to take this task in the next sprint?

[l0rd]

@skabashnyuk I had listed 3 distincts subtasks above. The first task should be on your side. The remaining 2 should be easier to analyse / work on when the first one is fixed.

[l0rd]

Do you want us to take this task in the next sprint?

Yes please. Even if it's not a bug that's still a regression compared to Che 6 hence it's important.

[slemeur]

related to #12971

[sleshchenko]

Che Plugin broker is almost adapted to self-signed certificates but unfortunately, Theia does not work out-of-the-box. Here is a demo of state of Che with self-signed certificates with changes that will be merged soon #13565: https://youtu.be/8z8WXA82G28
For Theia part, there is a separate issue #13574
cc @l0rd @slemeur @skabashnyuk @evidolob

[sleshchenko]

I believe there should not be any issues with self-signed certs, our testing result can be found here #14035 and #13869 (comment)
Feel free to create a new issue if there still some issues