User Interface Question

[vsoch]

heyo runc! I am getting the hang of using the client, and am at the step of creating a rootless container:

runc --root /tmp/runc run snakmake.runc

and then I shell inside, and this is good ;) My questions are

1. what would be the equivalent of doing the above, but with a custom bind?
2. how about the same thing, but with an exec (the entire thing from outside the container)

I think that should be enough to accomplish what I want - when I follow the same instruction provided for the (non rootless) containers, I can't seem to get it working, eg:

# This works, I shell inside!
runc --root /tmp/runc run snakmake.runc
#

# Maybe this is the issue?
runc --root /tmp/runc create snakmake.runc
cannot allocate tty if runc will detach without setting console socket

# Makes sense if create didn't work
$ runc --root /tmp/runc start snakmake.runc
container "snakmake.runc" does not exist

# So lonely! 
$ runc list
ID          PID         STATUS      BUNDLE      CREATED     OWNER

Is there a subset of commands that must be root? (e.g., creation and then can run without root?) or are the rootless containers limited in some way? Also, if there is a docs page that explains all this, please point me there - I went through the main repo readme and then couldn't find the answer so I came here. Thank you!

[vsoch]

I'm having a lot of trouble with this - I can't get basic commands to do binds to work. Could you please help? It's stopping me from progressing with runc.

[cyphar]

Your issue isn't related to rootless containers or permissions (your setup looks okay at first glance in that respect). You can run effectively all commands as an unprivileged user (assuming you have a config.json that is valid to work as an unprivileged user).

The issue you're hitting is related to terminals. #1730 is working on adding some documentation about this incredibly common pitfall. The short version is "if you're using terminal:true then running detached is harder". The main reason for not using runc run is that you want to do some sort of container setup between runc create and runc start, or that you don't want runc code to be running after the container is set up. If neither of those things interest you at the moment, you can use runc run and it'll be much simpler to work with.

[cyphar]

• what would be the equivalent of doing the above, but with a custom bind?
• how about the same thing, but with an exec (the entire thing from outside the container)

I'm not sure what you mean by "custom bind" or "with an exec", can you clarify what those mean?

[vsoch]

I'll have to read more carefully though the terminal documentation (my brain starts to shut down at this time) but I can quickly answer your second questions! I am coming at runc from the perspective of a docker or singularity user, for which a container looks like an executable, and a "run" command is simply executing some special script in the container. I'll do my best to put this into the way I (possibly wrongly) understand runc so far!

So I'm starting with a project directory, and in it I've generated a config.json that is --rootless and I've also extracted a bunch of .tar.gz for a complete operating system and my special scripts into a subfolder. It follows the example in the docs, and looks like this:

project/
    config.json
    rootfs/
        /bin
        ...
        /lost+found

What I can do pretty easily is cd into the project folder, and then use runc run to "shell" inside:

$ project> runc --root /tmp/project run containerid
#

This is great! Now what I've lost in dumping these layers is the custom logic for some entry point. It's ok, I could call the entry point script directly from here:

# /opt/conda/bin/snakemake all

but oops, when I was running this command in other container places, there was an expectation about a particular directory (on my local machine) being bound as a volume to my container. So I would have wanted to run the container and do something like:

$ project> runc --root /tmp/project run --bind /home/vanessa/data:/scif/data containerid
# ls /scif/data
   hereis  thedata.txt

And I would want to be able to tell anyone using this container "just bind your data, wherever it is on your host, to this folder in the container and it will run as you want!" (This is the custom bind) But then there is this other custom configuration file that must be in the present working directory when the run starts, so I would want to set that too:

$ project> runc --root /tmp/project run --bind /home/vanessa/data:/scif/data --pwd /scif/apps/project containerid

for Singularity this is --pwd and for Charliecloud it is --cd, so something like that.

To answer part II of the question, the exec is just doing all that, but without the interaction. Not ever "entering" the container. so instead it would look like this:

$ project> runc --root /tmp/project exec --bind /home/vanessa/data:/scif/data --pwd /scif/apps/project containerid /opt/conda/bin/snakemake

So generally of the form runc exec

I apologize for mis-ordering of the above - but I hope this illustrates my example! Thank you so much for your help, I'm totally new to runc and hoping to pick it up!

[crosbymichael]

@vsoch have you tried editing the config.json? I think it will make more sense if you haven't opened up the file yet. All the process args, mounts, cwd(pwd) is configured in that file.

[vsoch]

derp, nope haven't looked much at it... \facepalm let me do that now :)

[vsoch]

okay, so possibly some tiny progress. I thought it would be easier to move all the files I need in my context to be in the same root where I have the rootfs and config.json. I then added a bind to one of those folders (relative is ok?) it looks like this:

{
			"destination": "/scif/data",
			"type": "none",
			"source": "data/"
		},

and I'm getting an error there is no such device:

runc --root $PWD run snakmake.runc
container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"data/\\\" to rootfs \\\"/home/vanessa/Documents/Dropbox/Code/scif/examples/snakemake.scif/rootfs\\\" at \\\"/home/vanessa/Documents/Dropbox/Code/scif/examples/snakemake.scif/rootfs/scif/data\\\" caused \\\"no such device\\\"\""

I had seen this before and I just needed to create the /scif/data folder, but now I definitely have it. I am thinking - something to do with the type of bind, or permission, or using a relative path or enviroinment variable for the --root? Also I was trying some of the options like --bundle (to designate the folder with rootfs and config.json I think) and I can't figure it out:

$ runc --bundle /tmp/runc/ run snakmake.runc
... [ usage ]
ERRO[0000] flag provided but not defined: -bundle       
flag provided but not defined: -bundle

OPTIONS:
   --bundle value, -b value  path to the root of the bundle directory, defaults to the current directory
   --console-socket value    path to an AF_UNIX socket which will receive a file descriptor referencing the master end of the console's pseudoterminal
   --detach, -d              detach from the container's process
   --pid-file value          specify the file to write the process id to
   --no-subreaper            disable the use of the subreaper used to reap reparented processes
   --no-pivot                do not use pivot root to jail process inside rootfs.  This should be used whenever the rootfs is on top of a ramdisk
   --no-new-keyring          do not create a new session keyring for the container.  This will cause the container to inherit the calling processes session key
   --preserve-fds value      Pass N additional file descriptors to the container (stdio + $LISTEN_FDS + N in total) (default: 0)

[cyphar]

@vsoch You need to make the mount look more like this (ENODEV is the error that mount(2) gives you if you're trying to mount something in a way that doesn't really make sense.):

{ "destination": "/scif/data", "source": "/absolute/path/to/source/data", "type": "bind", "options": ["bind"] }

(Note the "type" and "options" fields. I know that that the "type" being "bind" doesn't make much sense from a Unix perspective -- but this is currently a long-standing bug in runc. You can also use rbind for the options if that is what you want.)

As for --bundle, it needs to come after the runc run (because global options go before the subcommand, and subcommand options go after it -- this is a "feature" of the flag parsing library we use).

[vsoch]

oh that's so helpful! Woohoo #1, we have data!

runc --root /tmp/runc run --bundle /tmp/runc snakmake.runc
# ls /scif/data
calls	 db	    genome.fa.amb  genome.fa.bwt  genome.fa.pac  images        report.html  sorted_reads
dag.svg  genome.fa  genome.fa.ann  genome.fa.fai  genome.fa.sa	 mapped_reads  samples

And now I see the sections in the config for environment, entrypoint, and directory. Wow this is really powerful! I can control the entire container but also the runtime specifics just by way of different config files! If It's ok with you I'd like to leave this open for one more day while I test different interactive (and commands run from outside) the container. Thank you for the help so far!!

[cyphar]

No problem. If you like a more in-depth explanation of precisely what knobs are available in config.json (and what they do) you can take a look at the specification (in particular, config.md and config-linux.md).

[vsoch]

Update! I was able to get every command working for runc --> https://github.com/sci-f/snakemake.scif/blob/container-friends/README.md#run-the-workflow and once the general workflow clicked it was really quite intuitive. This is awesome! Thank you again so much. Closing issue.