Is there any way to persist the home directory?

[dclong]

Similar to the /workspace directory?
Most user configuration files are in the home directory, so it's quite important.

[svenefftinge]

Currently, it is not backed up. We are considering snapshoting the state so you would have it after restarting the same workspace, but I guess your requests goes further in that you as a usr would like to see the same configuration files in every Gitpod workspace?

[jankeromnes]

I like the idea of persisting the home directory, but it could also have unfortunate side-effects: For example, if we install a new workspace tool that requires a change to ~/.bashrc, then restarting older workspaces + home would effectively overwrite ~/.bashrc and break the new tool. We recently did such a thing by installing SDKMAN to set up Java, Gradle and Maven, and it would break with older ~/.bashrcs.

Another option could be to allow Gitpod users to edit various configuration files in a central location, e.g. under a URL like https://gitpod.io/settings/ or https://gitpod.io/configurations/, and then these config files could be deployed to every workspace. We do something similar in Janitor (see the config files templates).

[dclong]

@svenefftinge It would be super to see the same configuration files in every Gitpod workspace. However, persisting each workspace's state is probably good enough as users typically don't have that many workspaces.

[dclong]

@jankeromnes Thank you very much for pointing out the side effects of persisting the home directory. I love the idea of central configuration location.

[PirateBread]

I'm playing with GitPod now and I think it's a great tool but I think the option for a "persistent" user directory being super useful.

For example I'm used to saving my ZSH configuration + history to my personal home directory which I like to port with me on projects that I am working on.

I think GitPod should consider two use cases : totally ephemeral project workspaces which are strongly bound to a given project and a more persistent workspace which is strongly bound to an individual and their own configuration.

I've only spent 3 hours in GitPod so far so maybe I'm coming at this from the wrong perspective but I see both being highly valuable.

[jankeromnes]

@PirateBread Agreed, and in fact we have already implemented a "full workspace backup" mechanism that removes the /workspace mountpoint and instead captures every change made in your workspace, including in the HOME directory. (So, when you stop and restart a workspace, the entire file system with the exception of /tmp will be just as you left it -- you'll no longer need to worry about what's under /workspace and what's not.)

We did this because it's a common expectation that changes in the HOME directory can be persisted, and reconfiguring every tool to make it save its valuable data under the non-natural /workspace directory was a loosing battle.

Full workspace backup is still experimental and kept behind a feature flag, but several team members have been using it without issues for several weeks now so we're thinking about rolling it out to more users soon.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[apolopena]

Could we get an update on this?
A persistant ~/ directory is pretty important to an overall good user experience. Without it we cannot customize the server much beyond writing code in ~/.bashrc from .gitpod.Dockerfile

[jankeromnes]

Thanks for the ping here @apolopena!

I've posted a longer-form update about Gitpod's upcoming Full Workspace Backup feature in https://community.gitpod.io/t/home-gitpod-directory-gets-reset-on-every-restart-of-the-workspace/2705/2

TL;DR -- It's already implemented and coming soon, but it's not part of Feature Preview yet. (If you'd like to try it already, we can technically enable it for your account, but warning: this is very experimental.)

[apolopena]

Thank you @jankeromnes for listing the current options and future plans.

I ended up implementing a workaround that copies files that need to be persisted outside of /workspace into a $GITPOD_REPO_ROOT--store directory for that project and then copies those files back to their original locations on each subsequent start of the workspace.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jankeromnes]

This is (tentatively) on the roadmap for Q2 2021 (which begins in 2 weeks).

[dclong]

It would great if the HOME directory can be used for storing configuration files (instead of using /workspace) as it is more consistent with the common setup and makes things much easier to run CICD pipelines.

[asabi]

Yep, will be great to see this feature live. I did not realize that it is not persistent, got everything running, was super happy only to lose everything in the home directory 😢.

Any update on this?

[loujaybee]

For example I'm used to saving my ZSH configuration + history to my personal home directory which I like to port with me on projects that I am working on.

As it is not yet mentioned on this thread, for those of you who are looking to persist personal configurations, with non-sensitive data, such as bash or zsh configs, you can use dotfiles.

For sensitive information, an option (outside of introducing your own secret store, such as vault) is to leverage: environment variables, and consider encoding e.g. with base64. See blog post.

[kokobd]

I found a generic method to persist some folders outside of /workspace using mount --bind

Put this in your before script. You can modify CACHE_DIRS to the list of folders you want to preserve between workspace restarts & prebuilds.

    before: |
      # Make sure some folders not in /workspace persist between worksapce restarts.
      # You may add additional directories to this list.
      declare -a CACHE_DIRS=(
        $HOME/.local
        $HOME/.cabal
        $HOME/.stack
        $HOME/.ghcup
        /nix
      )
      for DIR in "${CACHE_DIRS[@]}"; do
        mkdir -p $(dirname /workspace/cache$DIR)
        mkdir -p $DIR # in case $DIR doesn't already exist
        # On a fresh start with no prebuilds, we move existing directory
        # to /workspace. 'sudo mv' fails with 'no permission', I don't know why
        if [ ! -d /workspace/cache$DIR ]; then
          sudo cp -rp $DIR /workspace/cache$DIR
          sudo rm -rf $DIR/*
        fi
        mkdir -p /workspace/cache$DIR # make sure it exists even if cp fails
        # Now /workspace/cache$DIR exists.
        # Use bind mount to make $DIR backed by /workspace/cache$DIR
        sudo mount --bind /workspace/cache$DIR $DIR
      done

I'd like to know if this is fair use.

[raphaeltm]

Just want to weigh in and point out that this is causing me problems at the moment.

I'm working on a project that stores configuration in the user's home directory. I'm doing it this way in part because other tools that we rely on have the same developer experience. I realized things weren't working properly because the home directory didn't persist.

I'm not sure I understand the reasoning in #340 (comment)

The way we work with GitPod (which is incentivized by GitPod's UX), a good chunk of the value comes from the fact that workspaces are almost treated as ephemeral: you work on a feature in one, and toss it and work on a different feature in a different workspace. Given that, I wouldn't expect (or want) a workspace to receive updates or be changed in any way after being provisioned. I wouldn't use each workspace for a long time, but I would expect them to remain consistent for the duration that I use them.

Though perhaps I'm misunderstanding something about how GitPod manages images for prebuilds, etc.

[axonasif]

Thanks for sharing your feedback @raphaeltm.
At the moment, you could pre-create (dummy) files under /workspace that you intend to persist and then symlink/mount them back to their original path under $HOME using .gitpod.yml tasks, so all writes to them are redirected to /workspace and thus restorable between reboots.

[avelops]

Any news regarding this?
i'm having this issue with cypress using the home folder
i get "No version of Cypress is installed in: /home/gitpod/.cache/Cypress/13.6.0/Cypress" after a mere restart to the workspace

[avelops]

Make sure some folders not in /workspace persist between worksapce restarts.

  # You may add additional directories to this list.
  declare -a CACHE_DIRS=(
    $HOME/.local
    $HOME/.cabal
    $HOME/.stack
    $HOME/.ghcup
    /nix
  )
  for DIR in "${CACHE_DIRS[@]}"; do
    mkdir -p $(dirname /workspace/cache$DIR)
    mkdir -p $DIR # in case $DIR doesn't already exist
    # On a fresh start with no prebuilds, we move existing directory
    # to /workspace. 'sudo mv' fails with 'no permission', I don't know why
    if [ ! -d /workspace/cache$DIR ]; then
      sudo cp -rp $DIR /workspace/cache$DIR
      sudo rm -rf $DIR/*
    fi
    mkdir -p /workspace/cache$DIR # make sure it exists even if cp fails
    # Now /workspace/cache$DIR exists.
    # Use bind mount to make $DIR backed by /workspace/cache$DIR
    sudo mount --bind /workspace/cache$DIR $DIR
  done

not working with the following folder

$HOME/.cache/Cypress

[axonasif]

@avelops you can install cypress from a custom dockerfile and it'll remain.