DevWorkspace Switch - Endgame Plan

[l0rd]

Is your enhancement related to a problem? Please describe

This epic is supposed to replace the DevWorkspace STEP 3 milestone. It lists the issues required to switch from the che-server to the DevWorkspace, as well as the acceptance criteria to check .

ACCEPTANCE CRITERIA

Green field deployment and configuration

• DONE OperatorHub stable channel uses “all-namespace” install mode (one instance per cluster). Channel stable is the only channel for Che. It’s NOT possible to disable DevWorspaces. Kubernetes 1.21 (OCP 4.8) or higher is required
• DONE Latest upstream DevWorkspace Operator is deployed by Che operator
• DONE Keycloak is NOT managed anymore: on Kubernetes Dex is deployed, on OpenShift no extra auth component is deployed
• DONE chectl commands that interact with che-server or related to devfile v1 are removed
• DONE On platforms other than minikube and OpenShift an OIDC provider is a prerequisite to deploy Che (on minikube chectl deploys dex automatically, on OCP the integrated OAuth service is used)

Upgrading an existing instance

• DONE Existing Che instances are NOT upgraded automatically (even when using chectl)
• DONE Upgrading an existing Che instance is possible following some manual steps
• DONE During the upgrade all users workspaces are stopped
• DONE Backup feature is not available
• DONE After the upgrade existing devfilev1-based workspaces are disabled in Dashboard and can only be migrated

Dashboard and "Getting started"

• DONE "Getting Started" samples contain a devfile v2.1 at their root
• DONE All “Getting Started” devfiles use the same tag of the Universal Developer Image
• DONE All "Getting Started" are distributed as zipfile and with a pre-processed DevWorkspace CR using Theia as an editor (JetBrains and VS Code will be included in future releases)
• DONE “Getting Started” are labelled “community”
• DONE A workspace can be started from an devfile URL or from a GitHub/Gitlab/BitBucket repository URL
• DONE The path of the devfile, the branch, the IDE, the creation strategy are configurable using URL parameters (deployment labels/annotations and devfile metadata/attributes cannot be configured yet, will be in future releases)
• DONE It is possible to edit the devfile of an existing workspace from the Dashboard
• DONE It is possible to configure Che to use external devfile and plugin registries

Workspace startup

• DONE Factory flow: both devfile spec v2.1.0 and v1.0.0 are supported. The old v1.0.0 spec is supported through a v1 to v2 conversion on the fly
• DONE Starting a workspace from a Git repo without a devfile is supported: the default workspace uses Che-Theia with no plugins as the editor and the Universal Developer Image as the develpment/tooling container

Workspaces features

• DONE Java, Go, NodeJS, Python, .NET language server and debug adapter work as expected
• DONE Theia built-in plugins work as expected except SSH, OpenShift Authentication and GitHub authentication that will be fixed in future releases
• Installing VS Code extension from Theia work as expected - in progress
• DONE When upgrading Che the version of the default editor of a workspace is updated after a restart
• DONE Workspace Idling works for Theia based workspaces (idling for JetBrains and VS Code IDEs will be supported in future releases)
• DONE The 3 storage types persistent, ephemeral and asynchronous are supported

Enterprise features

• DONE Starting a workspace from a git repo behind proxy works and the project is cloned successfully
• DONE Starting a workspace from a git repo using an untrusted certificate is supported and the project is cloned successfully
• DONE Airgap scenarios are supported (installation, update, offline devfile sample projects from zip, plugins)
• DONE It is possible to configure Che to use an external OpenID Connect provider
• DONE It is possible to configure Che to use an external Postgres database
• DONE Che integrates with the DevConsole through the creation of a ConsoleLink that enables Che URLs in topology view
• DONE OpenShift trusted certificates bundle is propagation to Che workspaces

Telemetry

• DONE Telemetry Plugin support in DevWorkspaces (workspaces started with the DevWorkspace engine)

Monitoring

• DONE Minimal monitoring support with relevant documentation for configuring Prometheus / Grafana dashboards.

Documentation, website, and blog

TASKS

Deployment

• OperatorHub stable channel must use "all-namespace" install mode #20866
• chectl commands in the devworkspace land #20671
• Simplify authentication and authorization with a more flexible and lightweight approach #19182
• Use latest DevWorkspace Operator release when using stable channel #20908

Upgrading an existing instance

• As an Admin, I want to have clear instructions for switching to Dev Workspace engine #20614
• Don't upgrade existing Che instances to a new version with DevWorkspace engine enabled #20865

DevWorkspace

• Include ssh keys and tls certs into gitconfig devfile/devworkspace-operator#613
• Check that DWO is compatible with network-isolated clusters devfile/devworkspace-operator#668
• Support Pod tolerations for DevWorkspace Pods devfile/devworkspace-operator#614
• Enable proxy support for DevWorkspaces devfile/devworkspace-operator#678
• Workspace is not failing if requirement is not satisfied devfile/devworkspace-operator#685
• DWO should track workspaces that go from Running to Starting devfile/devworkspace-operator#618
• PROJECT_SOURCE env variable is invalid devfile/devworkspace-operator#677
• Create a test plan for DWO devfile/devworkspace-operator#666
• Support user name and email in gitconfig devfile/devworkspace-operator#694
  • (milestone 3 part is done) Add automount subpaths option devfile/devworkspace-operator#710

Dashboard

• Hide devfile v1 workspaces if the DevWorkspace engine is enabled #20900
• As a user, I want to start a DevWorkspace from a factory link for repositories with devfile v1 #20784
• When DevWorkspace is enabled and Che is upgraded to a new version, existing workspaces should pick the new editors #20193
• Dashboard should better handle devworkspace startup failure #19684
• Dashboard should be be able to group samples in dedicated groups #19633
• Use generated DevWorkspaceTemplates from the devfile registry to create workspaces #20829
• [UD] The ability to update a running devworkspace is broken. #20737
• Github action happy path tests are failing on workspace readiness #20734

Monitoring

• As an Admin, I want to have a granularity for the DevWorkspace startup failures exposed as a Prometheus metric #20261
• Provide documentation for enabling DevWorkspace monitoring (Prometheus / Grafana) #20662
  • procedures: Dev Workspace monitoring documentation che-docs#2161

Plugins

• Changes to Che-Theia preferences are persisted and loaded at every workspace start using a ConfigMap #20622
• Che-Theia Plugins view support for DevWorkspace based workspaces #20638
• Adapt che-theia plugins to avoid requesting che-server API #20788
  • Automatic connection to the OpenShift cluster when using the OpenShift connector (no need to login for the user anymore) #20857
  • [devWorkspace] Adapt ssh-plugin to avoid requesting che-server API  #20859
  • [devWorkspace] Adapt github-auth plugin to avoid requesting che-server API #20860
• Resources Monitor Plugin doesn't work when devWorkspace engine is enabled #20800

Devfiles

• Create v2 devfiles for Getting Started samples #19341
• Devfile registry support for Devfile v2 zip files  #20720

Che-server

• On prem private Gitlab repositores support with OAuth2 flow (without Keycloak) #20831
• Private Github factories with OAuth2 providers without keycloak #20497
• Support custom certificates for git hosts for devworkspaces #20528
• Provision SSH keys that are stored in the PostgreSQL database to the user's namespace in a known to devworkspace-controller format #20832
• Configure devworkspace namespace with user name and email in gitconfig #20871

OUT OF SCOPE (ADDRESSED IN FUTURE RELEASES / ORDERED BY PRIORITY)

• When a devfile is not present Che should include a UDI based dev container in the workspace #21340
• Telemetry Plugin support in DevWorkspaces (workspaces started with the DevWorkspace engine) #20090
• Provide documentation for enabling DevWorkspace telemetry and update existing docs #20820
• Improve handling for trusted certificates devfile/devworkspace-operator#665
• Build and publish multi-platform developer images #20862
• Allow to specify annotation/labels for the Workspace devfile/devworkspace-operator#698
• Add support for devfile kubernetes and openshift components #17894
• Persistent Volume strategies: add “per workspace” #21185
• Extend the DevWorkspaceTemplate spec with containers contributions devfile/devworkspace-operator#656
• Simplify Eclipse Che configuration format (CheCluster API v2) #20758
• Workspace creation metrics are available through DevWorkspace CR metrics in Telemeter
• Che operator option to install the image puller
• Che-Theia plugin that recommends the installation of VS Code extensions
• WON’T FIX IN FUTURE RELEASES Che operator backup and restore capability
• WON’T FIX IN FUTURE RELEASES Che4z “Getting Started”
• WON’T FIX IN FUTURE RELEASES Configure workspaces ServiceAccount (the DW controller can specify the default SA SCC instead)

[l0rd]

@skabashnyuk @azatsarynnyy @svor @tolusha if you haven't yet please update the TASKS with your milestone 3 issues
@dmytro-ndp please review the acceptance criteria

[tolusha]

done

[dmytro-ndp]

Hello Mario,

A couple of notices to consider:

1. and Theia with no plugins and no looks incomplete
2. Will Eclipse Che 7.41.0 support DevWorkspace disabled mode?
3. What is the target infra to deploy? Is it minikube (using helm), openshift 4.x using chectl or OperatorHub, ocp 3.11 ?

Next capabilities look also critical / blocker, IMHO:

1. Section "Green field deployment and configuration":

• Cascaded user removal

2. Section "Upgrading an existing instance":

• backup-restore feature
• rolling back update via CLI (https://issues.redhat.com/browse/CRW-1497)

3. Section "Dashboard and "Getting started" features":

• external & internal devfile registries
• "editor" factory parameter

4. Section "Workspace startup feature":

• Starting a workspace from zipped project stored in devfile registry
• 3 storage types: persistent, ephemeral, asynchronous

5. Section "Workspaces features":

• Custom .vsix plugins support
• User secrets in workspace
• OpenShift connector plugin support

6. Section "Enterprise feature":

• External database / SSO server support
• DevConsole integration
• Git with self sign certificates
• Propagation of the trusted bundle CA certificates to the workspaces
• multi-platform support (Build and publish multi-platform developer images #20862)
• Intellij IDEA editor support
• Image puller

[l0rd]

Thank you @dmytro-ndp, this is great.

[dmytro-ndp]

@l0rd: it's clear that we are going to support conversion of workspaces with devfile v1, linked to github projects, to devfile v2 (an issue #20784).

How about conversion of workspaces with devfiles v1, which have been obtained code from zipped project, keeping source code in persistent storage and devfile content in Che PostgreSQL database ?
AFAIK, we are going to stop such workspaces (an issue #20631).

What will user be needed to do in order to start such devfile v1 workspace again after DevWorkspace switch?

[nickboldt]

I don't think we need to worry about workspaces loaded from sample project code (dashboard devfile tiles).

But we DO need a story for these two possible paths when updating to 2.15 / DW enabled:

• make it 💯 clear that the update to 2.15 will break access to existing workspaces (so SAVE NOW or LOSE IT!), and/or
• provide a way to get data out of the postgres db where dirty workspace changes are persisted

If we can't warn users / admins clearly enough that turning on DW will prevent access to unsaved workspace changes... then we need that second option.

[benoitf]

How about conversion of workspaces with devfiles v1, which have been obtained code from zipped project, keeping source code in persistent storage and devfile content in Che PostgreSQL database ?

Converting che-server workspaces should use the 'stored definition/model' so there is no specific 'zip project case'

[dmytro-ndp]

@benoitf: I meant zipped project case like:

projects:
  - name: jboss-eap-quickstart
    source:
      location: 'https://devfile-registry-crw-crwctl.apps.ocp49.crw-qe.com/resources/jboss-eap-bootable-jboss-eap-quickstart-xp-2.0.x.zip'
      type: zip

[benoitf]

@dmytro-ndp I still don't see why it's a particular case.

[dmytro-ndp]

@benoitf: it's about non-factory flow, when devfile content is created in User Dashboard and stored into Che PostreSQL database.

About zip project type https://www.eclipse.org/che/docs/che-7/end-user-guide/authoring-devfiles-version-1/#adding-projects-to-a-devfile_che

git
Projects with sources in Git. The location points to a clone link.

github
Same as git but for projects hosted on GitHub only. Use git for projects that do not use GitHub-specific features.

zip
Projects with sources in a .zip archive. Location points to a .zip file.

AFAIK, converter v1 > v2 works with factory flow only, when user starting workspace using factory URL, and with project type github in devfile.

I am curious about how conversion v1 > v2 will work for workspaces without project, or with project without devfile.yml in source code.

[benoitf]

AFAIK, converter v1 > v2 works with factory flow only, when user starting workspace using factory URL, and with project >type github in devfile.

I am curious about how conversion v1 > v2 will work for workspaces without project, or with project without devfile.yml in source code.

The converter library takes a devfile v1 object as input so you can convert a devfile of an existing workspace (no matter of git, github or zip)

[nickboldt]

Maybe he's asking about dirty workspaces w/ code that was previously imported from the v1 devfile?

I assume for devfile samples listed in the Dashboard, there's no expectation that dirty workspace changes to a sample project will be persisted / supported after conversion to v2.1.

And for fresh workspaces created from new samples... well, they'll use the v2.1 devfiles instead, and will import the project from whatever is specified (git, zip, etc.)

But I suppose the UX challenge here is that if an ADMINISTRATOR performs the steps to manually update from CRW 2.14 / Che 7.40 (devworkspace disabled) to CRW 2.15 / Che 7.42 (devworkspace enabled) there's no way to communicate to USERS that all their old workspaces (be they personal, custom, or from a sample in the devfile registry) will be inaccessible after the switch is flipped to enable DW, and no way for users after the fact to access any unsaved changes.

So maybe we can mitigate this with a big warning in the CSV text of the 7.42/2.15 operator reminding admins that there's no way to go backwards and re-open workspaces that have been cut off after the switch to enable devworkspace?

[dmytro-ndp]

The converter library takes a devfile v1 object as input so you can convert a devfile of an existing workspace (no matter of git, github or zip)

@benoitf: let's say I have workspace v1 with zip project.
How should I apply converter library to convert it to v2?

[benoitf]

@dmytro-ndp #20845 ?

[dmytro-ndp]

@l0rd: will Eclipse Che continue supporting DevWorkspace disabled mode (with devfile v1, and back conversion v2 > v1) from 7.41.0?

[l0rd]

@dmytro-ndp no it won't be supported anymore.

[dmytro-ndp]

@l0rd: could you, please, say, if there will by multi-host support in DevWorkspace, or there will be single-host support only?

[l0rd]

@dmytro-ndp single-host only

[dmytro-ndp]

@l0rd: will DevWorkspace Step 3 support analytics plugins for java, NodeJs and Python?

[l0rd]

@l0rd: will DevWorkspace Step 3 support analytics plugins for java, NodeJs and Python?

@dmytro-ndp that should works as it doesn't use che-server API. @svor can you confirm?

[svor]

@l0rd: will DevWorkspace Step 3 support analytics plugins for java, NodeJs and Python?

@dmytro-ndp that should works as it doesn't use che-server API. @svor can you confirm?

It should work

[dmytro-ndp]

@l0rd, @svor: thanks for the answer!

@SkorikSergey: cc ^^

[nickboldt]

Starting a workspace from a Git repo without a devfile is supported: the default workspace uses Che-Theia with no plugins as the editor and the Universal Developer Image as the develpment/tooling container

Attempted this with Che dogfood cluster + https://che-dogfooding.apps.che-dev.x6e0.p1.openshiftapps.com/#https://github.com/redhat-developer/devspaces-images

Does not seem like the UDI image is available on workspace load -- I only get one choice for a terminal, and it doesn't know what java is. Also, it's an Alpine container, not a UBI8 based UDI image.

bash-5.1$ java -version
bash: java: command not found

bash-5.1$ cat /etc/motd
Welcome to Alpine!
The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.
You can setup the system with the command: setup-alpine
You may change this message by editing /etc/motd.
bash-5.1$ 

[l0rd]

Closing this issue. The switch to DevWorkspace operator happened in Che v7.42 release.