Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

htcondorcern: commands are not run in provided unpacked images #471

Open
mdonadoni opened this issue Oct 4, 2024 · 0 comments · Fixed by #474
Open

htcondorcern: commands are not run in provided unpacked images #471

mdonadoni opened this issue Oct 4, 2024 · 0 comments · Fixed by #474

Comments

@mdonadoni
Copy link
Member

Consider the following changes to reana-demo-helloworld:

--- a/workflow/snakemake/Snakefile
+++ b/workflow/snakemake/Snakefile
@@ -25,10 +25,14 @@ rule helloworld:
         sleeptime=config["sleeptime"]
     output:
         "results/greetings.txt"
-    container:
-        "docker://docker.io/library/python:2.7-slim"
+    resources:
+        kerberos=True,
+        compute_backend="htcondorcern",
+        htcondor_max_runtime="espresso",
+        unpacked_img=True
+    container: "/cvmfs/unpacked.cern.ch/registry.hub.docker.com/rootproject/root:6.26.00-ubuntu20.04"
     shell:
-        "python {input.helloworld} "
-        "--inputfile {input.inputfile} "
-        "--outputfile {output} "
-        "--sleeptime {params.sleeptime}"
+        """
+echo $SINGULARITY_NAME ;
+python --version
+        """

When executing this job, reana-job-controller will create the following job_singularity_wrapper.sh wrapper to run the user-provided commands on HTCondor:

#!/bin/bash 
singularity exec --home $PWD:/srv --bind $PWD:/srv --bind /cvmfs --bind /eos /cvmfs/unpacked.cern.ch/registry.hub.docker.com/rootproject/root:6.26.00-ubuntu20.04  echo ZWNobyAkU0lOR1VMQVJJVFlfTkFNRSA7IHB5dGhvbiAtLXZlcnNpb24=|base64 -d | bash

Note that:

$  echo ZWNobyAkU0lOR1VMQVJJVFlfTkFNRSA7IHB5dGhvbiAtLXZlcnNpb24=|base64 -d
echo $SINGULARITY_NAME ; python --version

However, if we execute this script manually on lxplus, this is the output:

$ bash job_singularity_wrapper.sh 

bash: line 1: python: command not found

The | base64 -d | bash part of the script is not executed in the singularity container, but in the outer bash shell, thus the user-provided commands are not executed in the singularity container.

@mdonadoni mdonadoni added this to 0.95.0 Oct 4, 2024
@mdonadoni mdonadoni removed this from 0.95.0 Nov 20, 2024
@mdonadoni mdonadoni added this to 0.9.4 Nov 20, 2024
@mdonadoni mdonadoni moved this to Ready for work in 0.9.4 Nov 20, 2024
@mdonadoni mdonadoni moved this from Ready for work to In work in 0.9.4 Nov 20, 2024
mdonadoni added a commit to mdonadoni/reana-job-controller that referenced this issue Nov 20, 2024
mdonadoni added a commit to mdonadoni/reana-job-controller that referenced this issue Nov 21, 2024
mdonadoni added a commit to mdonadoni/reana-job-controller that referenced this issue Nov 21, 2024
@mdonadoni mdonadoni linked a pull request Nov 21, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant