Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

corruptedEnvironmentError showing up when updating conda requirements #3337

Open
VasLem opened this issue Oct 9, 2024 · 0 comments
Open

Comments

@VasLem
Copy link

VasLem commented Oct 9, 2024

Let's say I have this example Dockerfile:

FROM ubuntu:22.04
ENV R_VERSION=4.3.2
ENV PYTHON_VERSION=3.10

ARG DEBIAN_FRONTEND=noninteractive

SHELL ["/bin/bash", "-c"]
USER root
ENV HOME=/home/root
WORKDIR $HOME
ENV PATH="$PATH:$HOME/miniforge3/bin"
RUN conda config --add channels bioconda && \
    conda config --add channels conda-forge && \
    conda config --set channel_priority flexible
RUN conda update -y -n base -c conda-forge conda 
RUN conda install -y conda-libmamba-solver python=$PYTHON_VERSION r-base=$R_VERSION 
RUN conda install -y bedops r-r.utils 

and this cloudbuild.yaml :

steps:
- name: 'gcr.io/kaniko-project/executor:latest'
  args:
  - --destination=europe-west1-docker.pkg.dev/MY_REPO/MY_IMAGE:latest
  - --cache=true
  - --compressed-caching=false
  - --cache-copy-layers=true

which needs all of those flags for a variety of stupid reasons (see this).

Then, I use this command

gcloud builds submit --async --region=europe-west1 --project=MY_PROJECT --default-buckets-behavior REGIONAL_USER_OWNED_BUCKET --machine-type=e2-highcpu-8

to make a build, successfully. If afterwards, I add/modify any line that has to do with conda, for example by appending this line:

RUN conda install samtools -y

and retry to build, I am always getting something like this error:

CorruptedEnvironmentError: The target environment has been corrupted. Corrupted environments most commonly
occur when the conda process is force-terminated while in an unlink-link
transaction.
  environment location: /home/root/miniforge3
  corrupted file: /home/root/miniforge3/conda-meta/.wh.ruamel.yaml-0.18.6-py312h98912ed_0.json

error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1

(The corrupted package could be any package installed in the previous build.)

which forces me to delete the cache and retry installation. Of course this runs alright, however kaniko completely loses its purpose, and is only meaningful when I change other things in the Dockerfile. Of course, changing miniconda requirements is the most frequent operation I do, as I am using Python and R, and any new dependency coming up needs to have the image updated.

Description Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@VasLem and others