Ability to override automount secret properties in devfile

[mshaposhnik]

Is your task related to a problem? Please describe.

Atref implementing #14680 users can have secrets mounted to theirs workspace contaibers as files on env variabes. At the same time, user may need some variabitilty, like changing environment var name or mount path without affecting the original secret in namespace. So that can de done by updating the devfile of the perticular workspace.

Describe the solution you'd like

Initial idea (copied from #14680, probably needs to be revised and refactored) :

To be able to explicitly override mountPath or envName of concrete Secret we need a pair secretName and key that we want to override + value. It can look like this in devfile
devfile.yaml

...
components:
 - type: dockerimage
   annotations:
      mvn-settings-secret: mountPath=/home/user2/.m2/
      mvn-settings-secret: envName=ENV_NAME2
   alias: maven
   image: maven:3.11
...

or

devfile.yaml

...
components:
 - type: dockerimage
   alias: maven
   image: maven:3.11
   secrets:
      mvn-settings-secret:
         mountPath=/home/user2/.m2/
...      envName=ENV_NAME2 

[mshaposhnik]

Since secrets may be applied to any container of any component in devfile, it seems to me that
overriding properties should not be described on component-s level, but more globally for whole devfile, so metadata section seems to be more appropriate place. Moreover, that matches with secrets description, so it looks more uniform for the writer.

So, thats is the first draft proposal for discussion beginning (it supposes devfile owner knows the secret names, but i'm not sure it is always true).

apiVersion: 1.0.0
metadata:
  name: petclinic-dev-environment
  generateName: petclinic-
  secrets-(annotations?)-override:
      - <mvn-settings-secret-name>:
            che.eclipse.org/mount-path: /home/user2/.m2/    ## option 1: prefixed exact annotation name override
            che.eclipse.org/target-container: maven 
     - <single-env-var-secret-name>
            env-name: ENV_NAME2                             ## option2:  unprefixed annotation override
            target-container: maven
      - <multi-env-var-secret-name>
            key1_env-name: FOO1
            bar1_env-name: BAR1
            target-container: theia
projects:
....

[mshaposhnik]

Ok, so after having some internal discussion, we see the two different approaches, with different flexibility and usage complexity. Generally, it's per-component or per-devfile overrides.

First one initially proposed by Sergii, with examples in issue description, it's per-component override.
So it will affect only component where this override is described. But user may need to know container names provided by that component, and need to write the same override definition multipe times if he wants his changes to be applied to serveral components across workspace. Also, there is a question with default components, which are not present in devfile. So to have them be re-configured, they need to be defined explicitly in devfile and overrides then written into theirs specification.

Second is proposed by me in comment above, it-s per devfile override, authomatically applying override all changes per all components, so it can be written just once. But of cource it less flexibility, not allowing to change something in just one component (but you can still override just one target component if you know it's name).

As for me, mounting same secret into ALL containers is an quite unusual exceprional case. In most usecases, thre will be just one target container where the secret shoult be applied. So, IMO, having single definition is just enough if you want to change this container and/or modify mount path or ENV name. So with 2nd case you dont even need to care in which component this container is.

So let's try to choose one that fit our needs better, to stay on right track.
@l0rd @benoitf @sunix

[l0rd]

I was looking at the automounting of a SA account token that has some analogies with this use case and here are some considerations:

Automounting

To avoid that the SA token get automatically mounted on every containers k8s API provides the attribute automountServiceAccountToken that can be set at the SA level and at the Pod level.
The value at Pod level has higher priority.

For our use case we should have a similar attribute automountWorkspaceSecrets at secret and devfile.component level instead of relying on targetContainer at secret level. And the value at devfile.component level should have higher priority.

Overriding Secrets Paths

To override SA token default mount path (/var/run/secrets/kubernetes.io/serviceaccount/token) k8s API provides SA token volume projection. The path get overriden by the containers[].volumeMounts[].mountPath.

For our use case I would get inspiration from it (again) and have something similar:

components:
 - type: dockerimage
   alias: maven
   image: maven:3.11
   volumes:
     - name: <secret-name>
       containerPath: /my/new/path

Cross component Volume Definition

I would wait devfile 2.0 for that as there is a proposal for something that would address that devfile/api#19 (comment).

[mshaposhnik]

So, to make sure i understood it correctly about automounting:

In secret, we will replace org.eclipse.che/target-container: maven annotation with org.eclipse.che/automountWorkspaceSecrets: (false|true) which has the follwing meaning:

org.eclipse.che/automountWorkspaceSecrets: true means secret is mounted by default into all devfile components, except those which have automountWorkspaceSecrets: false override in devfile.

org.eclipse.che/automountWorkspaceSecrets: false means secret is not mounted into any
devfile components, until the component have automountWorkspaceSecrets: true override, allowing to mount the secret to this particular component. Am i got it correct @l0rd ?

[l0rd]

@mshaposhnik yes. automountWorkspaceSecrets or automountWorkspaceSecret?

[mshaposhnik]

I think plural is better since several secrets can be present in namespace, wdyt ?

[l0rd]

Ok but each secret will have its annotation automountWorkspaceSecret right? The confusion is maybe that in one kubernetes secret there are multiple data entries?

[mshaposhnik]

Yes, so maybe it should be automountWorkspaceSecret in secret and automountWorkspaceSecrets in devfile (but what if we want to enable/disable mount of particular one instead of all -at -once)

[mshaposhnik]

Maybe we can try the following for the devfile component only:
automountWorkspaceSecrets: (false | true | secret-foo, secret-bar) so true mounts all, false none, or just those enumenrated my names, wdyt ?

[l0rd]

Not sure we should mix booleans with free-form strings. I would go with true/false only. And eventually adding a selector in the future if there are some use cases that require that.

[mshaposhnik]

Ok, cool. So, to summarize:

1. We replace che.eclipse.org/target-container: maven with
   che.eclipse.org/automount-workspace-secret: true | false in secrets to indicate it should be applied everywhere is the WS; (naming format is changed to be aligned with the format of rest annotations)

2. We add automountWorkspaceSecrets: true | false into devfile to override secret setting per component. Looks good for you Mario? CC @skabashnyuk

[l0rd]

@mshaposhnik nice!

[skabashnyuk]

(naming format is changed to be aligned with the format of rest annotations)

@l0rd Should we handle also the old format? or we can cover that with documentation?

[l0rd]

@l0rd Should we handle also the old format? or we can cover that with documentation?

I would only cover that with documentation. I don't think there has been adoption yet. But we should probably make sure that we release the new format for CRW 2.2.