Container manipulated via multiple commands

[duglin]

I was trying to think about how I would go about creating an image builder program using runc/OCI. I was envisioning something similar to Docker's builder/Dockerfile file processing but with one key difference, I wanted the long-sought-after 'squash' feature. I wanted the process used by this builder to be:
1 - create the container
2 - perform a set of actions to manipulate the container - cp files, run commands, etc...
3 - create an OCI compliant image from the resulting container

Even outside of this "builder" use case this would seem to be a fairly trivial use case that might be useful to lots of people.

But, I'm having trouble imagining how I would do this with today's spec and runc. The issues all seem to be around the process management side of things. In particular:

• I can't create the container w/o also running a command
• I can't run a command in a container w/o having an existing command already running

In #225 (and in other places) there's the start of a discussion around splitting create from start and I'd like to explore that some more. While I can certainly understand that in certain environments the authors of that environment might want to be very opinionated about the lifecycle of the processes in the containers (e.g. Docker goes out of its way to have the notion of a 'main' process which when it dies it kills all other processes), but as runc/OCI is a basic building block I don't see the value in requiring this view on everyone.

As mentioned in #225 (comment) I'm wondering why the ops we have aren't just:
1 - create container
2 - run a cmd in a container (irrespective of whether another cmd is already running)
3 - delete a container

Then 2 can be run as many times as needed by the tooling in order to do what the end user wants.

There are ways we could still support the opinionated path if we wanted. For example, if on the create() we passed in a cmd then let it do as we do today, but if there is no cmd then we do the flow above.

[wking]

On Tue, Oct 13, 2015 at 11:24:05AM -0700, Doug Davis wrote:

• I can't create the container w/o also running a command

+1 towards separating create and run a much as possible, but I don't
think it's possible to separate out the PID namespace (e.g. you can't
have one without also having a running command to keep it open).
Still, “an optional PID namespace” is a much lighter-weight idea than
“the full container”, so I think it would be useful to have:

• a “create the container except for the PID namespace” command and
• a separate “launch a process joining an existing container
  (potentially including a PID namespace) and optionally
  creating/configuring a new PID namespace”.

More notes on how you could set this up in 1.

[wking]

On Tue, Oct 13, 2015 at 11:32:16AM -0700, W. Trevor King wrote:

• a “create the container except for the PID namespace” command and
• a separate “launch a process joining an existing container
  (potentially including a PID namespace) and optionally
  creating/configuring a new PID namespace”.

I wrote a wrapper 1 around ccon 2 that does this (example in 3),
in case folks want to play around with it. ccon-ced appends a
namespace-bind-mounting hook in the pre-start hook and invokes ccon.
When “process” isn't defined (it's optional in ccon 4, but maybe not
in an OCI spec [5,6]?), ccon's container process exits instead of
blocking (this is new since ccon 0.2.0 7), so we can just leave
“process” out of create.json 8. exec.json creates the PID namespace
and the PID-namespace-specific /proc mount, but is otherwise just a
process definition 9. Users can then do things like:

manage-cgroups setup

ccon-ced create --id abc

for CONFIG in exec-*.json; do ccon-ced exec --id abc --config "$CONFIG"; done

oci-snapshot --id abc

ccon-ced delete --id abc

manage-cgroups delete

to handle things like cgroup manipulation, bundle snapshot creation,
etc.

If we want to enable this sort of functionality in OCI-compliant
specs, I think we want:

a. An explicitly-optional “process” (for ‘create’), so we don't have
to require true(1) in the container.
b. An explicitly-optional almost-everything-except “process” (for
‘exec’), so exec configs aren't too wordy.
c. A way to pass configs in as command line arguments (ccon's
--config-string) or named pipes (ccon's ‘--config /proc/self/fd/3’),
otherwise you'd have to distribute a separate bundle for each exec
call. This is probably dependent on collapsing back to a single
config file 10.

Until there is a clearer opt-out mechanism for cgroups (e.g. #237),
folks can set linux.cgroupsPath to their desired control group (‘/’ if
they want to stay in the root cgroup?) and leave off linux.resources
to get “join but not alter” semantics (see #247).

That's for all for Linux namespaces. I agree with @crosbymichael that
this may all be platform specific 11, and feedback about whether/how
separate create/exec calls are possible on other platforms would be
useful.

And to collect implementation refrences here, @duglin also floated a
runC patch to add batch-like execution [12,13].

  Subject: Single, unified config file (i.e. rolling back specs#88)
  Date: Wed, 4 Nov 2015 09:53:20 -0800
  Message-ID: <20151104175320.GC24652@odin.tremily.us>

[philips]

Hello Doug-

I see building container images as a pipeline. There is an "in volume"
where the user source goes in and an "out volume" where the compiled image
comes out. The container you are running that is processing the source and
generating the output would be something like acbuild with a build
environment. You can find some examples here:
https://github.com/appc/acbuild/tree/master/examples

The only really interesting thing that can happen in a Dockefile is this
"snapshot and cache" semantic but that seems like a property for whatever
is powering the "out volume", not the container image builder.

Does that make sense?

Cheers,

Brandon

On Tue, Oct 13, 2015 at 11:24 AM Doug Davis notifications@github.com
wrote:

I was trying to think about how I would go about creating an image builder
program using runc/OCI. I was envisioning something similar to Docker's
builder/Dockerfile file processing but with one key difference, I wanted
the long-sought-after 'squash' feature. I wanted the process used by this
builder to be:
1 - create the container
2 - perform a set of actions to manipulate the container - cp files, run
commands, etc...
3 - create an OCI compliant image from the resulting container

Even outside of this "builder" use case this would seem to be a fairly
trivial use case that might be useful to lots of people.

But, I'm having trouble imagining how I would do this with today's spec
and runc. The issues all seem to be around the process management side of
things. In particular:

• I can't create the container w/o also running a command
• I can't run a command in a container w/o having an existing command
  already running

In #225 #225 (and in other
places) there's the start of a discussion around splitting create from
start and I'd like to explore that some more. While I can certainly
understand that in certain environments the authors of that environment
might want to be very opinionated about the lifecycle of the processes in
the containers (e.g. Docker goes out of its way to have the notion of a
'main' process which when it dies it kills all other processes), but as
runc/OCI is a basic building block I don't see the value in requiring this
view on everyone.

As mentioned in #225 (comment)
#225 (comment)
I'm wondering why the ops we have aren't just:
1 - create container
2 - run a cmd in a container (irrespective of whether another cmd is
already running)
3 - delete a container

Then 2 can be run as many times as needed by the tooling in order to do
what the end user wants.

There are ways we could still support the opinionated path if we wanted.
For example, if on the create() we passed in a cmd then let it do as we do
today, but if there is no cmd then we do the flow above.

—
Reply to this email directly or view it on GitHub
#226.

[duglin]

See opencontainers/runc#465 for an alternative

[duglin]

This will most likely be answered/solved by the split of create/start so we can close this. If we don't do the split then we'll reopen it.