User should be able to set/retrieve secrets and see them mounted and/or as env variables

[sunix]

Is your enhancement related to a problem? Please describe.

There is already some implementation related to saving ssh keys into secrets and mount them. But the user should be able to customize and not be restricted to ssh keys. The user should be able to store as secrets github token, aws access token. The user should be able choose to mount these values to the place of his choice or make them available as environment variable from the devfile.

https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables

Describe the solution you'd like

It could be using k8s secrets but this is not necessary.
Ideally it would be good to have a UI to set the values

UPD: The scope of this issue will be ability to mount secret as an file or ENV variable, and ability to override some properties in devfile. Applying settings from a configmap will be in separate one.
UPD2: devfile overrides have separate issue #17079

[sunix]

We should try to do something similar to https://docs.openshift.com/container-platform/4.3/builds/creating-build-inputs.html#builds-adding-input-secrets-configmaps_creating-build-inputs

[sunix]

from #15518 (comment)

The way I see it, is a section in the Dashboard where you could define these configmaps or secrets. We could also have configmaps or secret per organisation. and we could have a way to set these configmap or secrets through chectl.

I definitely do not see these values/files defined in the devfile. But references to secrets and configmap.
These secrets and config maps could be used by any workspaces by just referencing them.

We could have default entries for configmaps and secrets that we use in our default devfiles. These ones could be overriden by the user.

[sunix]

This is important for setting up Airgap, for instance injecting a Maven settings.xml file that would contain:

• user/password to access to internal server (repositories)
• Several private repos and configurations
• Several mirrors

See last lines of #15503 (comment)

[sunix]

@tolusha I change that one to severity/P1
Let me know if it works for you.

[l0rd]

We should leverage labels on configmaps and secrets to automatically mount them in workspaces containers.

For example:
If the a user creates a secret/configmap in his che workspaces namespace with the label app.kubernetes.io/name: che then the secret/configmap gets automatically mounted on the workspace pod containers.

[sunix]

in my opinion for each container, somehow we need to specify where to mount. It can not be done automatically. Others secrets may need to be exposed as env variable,not mounted.

[l0rd]

@sunix right. I think annotations should be used to specify the behaviour.

[l0rd]

@skabashnyuk this issue is about being able to mount the secret as a file in a container or used as an environment variable. The default would be to mount as a file but we could control the behavior using an annotation (useSecretAsEnv ):

---
apiVersion: v1
kind: Secret
metadata:
  name: mvn-settings-secret
  labels:
    app: che                # <--- mandatory: all secrets labeled `app: che` will be mounted
    targetContainer: maven  # <--- optional, if omitted will be mounted on all containers
  annotations:
    mavenSettingsPath: /home/user/.m2/
    useSecretAsEnv: true    # <--- the default is false

[skabashnyuk]

After we add API to add secrets to users k8s namespace we will need also some dashboard UI to use these APIs.

[skabashnyuk]

this issue is about being able to mount the secret as a file in a container or used as an environment variable.

The name of the environment variable should be set with annotation too?

[sunix]

I thought we would be able to set the path where to mount from the container definition in the devfile: #15518 (comment)
or user would just need to set the right name as annotation and it would override it?

[sunix]

But I don't see how that could be generic setting an annotation name with mvn in it.
How would it look like if I would like to expose a GitHub token I have in my secret as environment variable of a specific container?
or a ~/.aws/credentials file from another secret mounted to another container

[mshaposhnik]

Second thing is to allow bind multiple ENV variables by single secret. Proposal is following:

apiVersion: v1
data:
  myfoo: bXliYXI=
  mybar: YWRtaW4=
kind: Secret
metadata:
  annotations:
    myfoo.envName: MY_FOO
    mybar.envName: MY_BAR
    useSecretAsEnv: 'true'
  labels:
    app: che

In case of single value, ENV name annotation can be either myfoo.envName: MY_FOO or simply envName: MY_FOO:

apiVersion: v1
data:
  myfoo: bXliYXI=
kind: Secret
metadata:
  annotations:
    envName: MY_FOO
    useSecretAsEnv: 'true'

[sunix]

hello @mshaposhnik this is looking nice. Would it be possible to make a first implementation as a preview feature? It means: that would be available for the user but we say that the api/format can change in the next releases: I would like to have that ASAP and test it in my flow to give you feedback.

[sunix]

@mshaposhnik eventually, I would like that we replace the way we are dealing with ssh keys, and having them stored using that mechanism. WDYT?

[mshaposhnik]

@sunix let's not mix everything into single wineglass :) good idea for separate issue

[sunix]

yes,

eventually

:) but it really makes sense

[l0rd]

@mshaposhnik makes sense. For the naming I am not satisfied by goal: mount. Looking at kubernetes recommended labels I am thinking about app.kubernetes.io/part-of: org.eclipse.che, app.kubernetes.io/component: workspace-secret.

[mshaposhnik]

@l0rd @sunix During testing, found an very annoying thing, not sure if you were aware of it...
In short, every folder that is creaded using config map or secret as a source, is readonly, no matter what we'll write in definition. Here is K8S issue: kubernetes/kubernetes#62099

So it means, for example, if we create foder .m2 with some file inside using secret mount, it will be readonly, so maven cannot create repository here.

Dunno if it's acceptable for us now?

The workaround can be pre-creating such forders in image. Or what i saw in K8S issue discussion, here is also some way with launching init containers which will create shared folders so the secret mount haven't to create them.

[sunix]

We have to change the current m2 volume to /location.user.home/.m2/repository/ ... not /location.user.home/.m2, (and renamed m2repo) would that work ?
the settings.xml should come from configuration ... not volume.

[l0rd]

@mshaposhnik #15518 (comment)

[skabashnyuk]

@l0rd here #16895 (comment) #16895 (comment) #16895 (comment) @metlos have a few suggestions about annotation format. Can you take a look and tell us what do you think?

[l0rd]

@skabashnyuk @mshaposhnik I do like @metlos proposals

[mshaposhnik]

So, with the all recent updates, here is how the secrets will looks like:

1. File mount:

---
apiVersion: v1
kind: Secret
metadata:
  name: mvn-settings-secret
  labels:
    app.kubernetes.io/part-of: org.eclipse.che   
    app.kubernetes.io/component: workspace-secret
  annotations:
    org.eclipse.che/target-container: maven  
    org.eclipse.che/mount-path: /home/user/.m2/
    org.eclipse.che/mount-as: file
data:
  settings.xml:  ...

will result in settings.xml mount in /home/user/.m2/

2. Single ENV mount:

---
apiVersion: v1
kind: Secret
metadata:
  name: mvn-settings-env
  labels:
    app.kubernetes.io/part-of: org.eclipse.che   
    app.kubernetes.io/component: workspace-secret
  annotations:
    org.eclipse.che/target-container: maven  
    org.eclipse.che/mount-as: env
    org.eclipse.che/env-name: FOO_ENV
data:
  mykey: myvalue

will result inFOO_ENV=myvalue set in container.

3. Multiple ENV mount:

---
apiVersion: v1
kind: Secret
metadata:
  name: mvn-settings-env
  labels:
    app.kubernetes.io/part-of: org.eclipse.che   
    app.kubernetes.io/component: workspace-secret
  annotations:
    org.eclipse.che/target-container: maven  
    org.eclipse.che/mount-as: env
    org.eclipse.che/mykey_env-name: MYKEY_ENV
    org.eclipse.che/somekey_env-name: SOMEKEY_ENV
data:
  mykey: myvalue
  somekey: somevalue

will result inMYKEY_ENV=myvalue and SOMEKEY_ENV=somevalue set in container.

[l0rd]

@mshaposhnik re-reading k8s labels character set documentation and che.eclipse.org/<name> sounds better than org.eclipse.che/<name>

[mshaposhnik]

So i believe this issue should be closed after #17167 merge, since all initial requirements are covered. For further improvements, we will do separate ones (there is already some AFAIK)

[sunix]

@mshaposhnik thanks a lot.
What is the best way to try that out? (preferably using Che)

[mshaposhnik]

@sunix Try secrets mount? Just write a right secret & try it :). Also my demo might help. PM if you have troubles with this.