volume: add support for non-volatile `upperdir`,`workdir` for overlay volumes #12712

flouthoc · 2021-12-28T10:40:08Z

Often users want their overlayed volumes to be non-volatile in nature
that means that same upper dir can be re-used by one or more
containers but overall of nature of volumes still have to be overlay
so work done is still on a overlay not on the actual volume.

Following PR adds support for more advanced options i.e custom workdir
and upperdir for overlayed volumes. So that users can re-use workdir
and upperdir across new containers as well.

Usage

$ podman run -it -v myvol:/data:O,upperdir=/path/persistant/upper,workdir=/path/persistant/work alpine sh

This PR is draft till vendor is updated:

Vendor c/common
Vendor c/buidlah

Closes: #12150

flouthoc · 2021-12-28T10:41:30Z

/hold

flouthoc · 2021-12-28T10:42:51Z

@giuseppe @rhatdan I still need to move vendor to their respective repos apart from that this should be good for review for first pass.

flouthoc · 2021-12-28T10:44:13Z

I have also added a test to verify persistent nature of custom upperdir, workdir

rhatdan · 2021-12-28T10:47:20Z

Should this be done on the --mount option and not dirty up the --volume option?

flouthoc · 2021-12-28T10:55:14Z

Should this be done on the --mount option and not dirty up the --volume option?

@rhatdan That would be better but I don't think the :O construct is applicable with --mount , afaik the concept of overlay with named volumes only work with -v or --volumes as specified here https://docs.podman.io/en/latest/markdown/podman-run.1.html. In fact i don't see overlay at all with --mount in the docs.

But I'll confirm thanks.

giuseppe · 2022-01-03T09:26:37Z

libpod/container_internal_linux.go

-			overlayMount, err := overlay.Mount(contentDir, mountPoint, namedVol.Dest, c.RootUID(), c.RootGID(), c.runtime.store.GraphOptions())
+
+			overlayOpts = nil
+			if upperDir != "" && workDir != "" {


I think we need to check that either both upperdir and workdir are specified, or none of them.

We cannot have just one of upperdir or workdir

giuseppe · 2022-01-03T09:26:51Z

libpod/container_internal_linux.go

@@ -2781,6 +2803,7 @@ func (c *Container) copyTimezoneFile(zonePath string) (string, error) {
 }

 func (c *Container) cleanupOverlayMounts() error {
+	//overlay.CleanupContent("/tmp/exp-overlay")


should this be dropped?

rhatdan · 2022-01-03T13:40:46Z

If the uppderdir and workdir are on different file systems, do you get a decent error, or should we be checking that?

flouthoc · 2022-01-04T08:47:36Z

If the uppderdir and workdir are on different file systems, do you get a decent error, or should we be checking that?

@rhatdan We get an error from overlay with invalid argument. Its not very pretty but I guess it should work since its expected.

Error: OCI runtime error: runc create failed: unable to start container process: error during container init: error mounting "/home/flouthoc/.local/share/containers/storage/overlay-containers/07d4e523d4a83459c7ce4187dd2d563caf74e110a2ba1bc769cb55d095fade5e/userdata/overlay/969251762/merge" to rootfs at "/data": mount /home/flouthoc/.local/share/containers/storage/overlay-containers/07d4e523d4a83459c7ce4187dd2d563caf74e110a2ba1bc769cb55d095fade5e/userdata/overlay/969251762/merge:/data (via /proc/self/fd/7), data: lowerdir=/home/flouthoc/.local/share/containers/storage/volumes/myvol/_data,upperdir=/tmp/exp-overlay/upper,workdir=/home/flouthoc/tmp/work,userxattr,context="system_u:object_r:container_file_t:s0:c267,c630": invalid argument

rhatdan · 2022-01-10T19:02:56Z

@flouthoc Rebase so we can continue working on this.

flouthoc · 2022-01-10T19:05:40Z

@rhatdan Yes will do. I just need to update man pages for buildah changes and bump vendor again.

flouthoc · 2022-01-12T09:22:30Z

Upstream buildah has some breaking change so need few PR's so i can vendor buildah in: #12822

Edit: Also c/common has some regression so wait for #12821

flouthoc · 2022-01-14T12:47:17Z

Since we vendor-ed buildah here: #12642 I'll start re-basing this again.

rhatdan · 2022-01-27T17:04:47Z

libpod/container_internal_linux.go

+			if overlayFlag && strings.Contains(o, "upperdir") {
+				splitOpt := strings.SplitN(o, "=", 2)
+				if len(splitOpt) > 1 {
+					upperDir = splitOpt[1]


Should you check for duplicates?

if upperDir != "" { return ERROR }

We ignore in underlying library if upperdir is empty. But I agree user should get early error so added here.

libpod/container_internal_linux.go

rhatdan · 2022-01-27T17:08:50Z

test/e2e/run_volume_test.go

+		Expect(session).Should(Exit(0))
+
+		// volume should contain only `test` not `overlay`
+		session = podmanTest.Podman([]string{"run", "--volume", volName + ":/data:O," + overlayOpts, ALPINE, "sh", "-c", "ls /data"})


Can you test two volume mounts, one without overlayOpts and one without, and make sure the one without is not perment

Yes added another mount with just overlay and no custom upper/workdir which tests that content is not persisted.

TomSweeneyRedHat

Any man page changes?

…umes Often users want their overlayed volumes to be `non-volatile` in nature that means that same `upper` dir can be re-used by one or more containers but overall of nature of volumes still have to be `overlay` so work done is still on a overlay not on the actual volume. Following PR adds support for more advanced options i.e custom `workdir` and `upperdir` for overlayed volumes. So that users can re-use `workdir` and `upperdir` across new containers as well. Usage ```console $ podman run -it -v myvol:/data:O,upperdir=/path/persistant/upper,workdir=/path/persistant/work alpine sh ``` Signed-off-by: Aditya R <arajan@redhat.com>

flouthoc · 2022-01-28T09:34:25Z

Any man page changes?

@TomSweeneyRedHat thanks added a section to docs as well.

rhatdan · 2022-01-28T11:49:18Z

LGTM
@containers/podman-maintainers PTAL

giuseppe

/lgtm

openshift-ci · 2022-01-28T11:57:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: flouthoc, giuseppe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [flouthoc,giuseppe]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…onymous volumes Similar feature was added for named overlay volumes here: containers#12712 Following PR just mimics similar feature for anonymous volumes. Often users want their anonymous overlayed volumes to be `non-volatile` in nature that means that same `upper` dir can be re-used by one or more containers but overall of nature of volumes still have to be overlay so work done is still on a overlay not on the actual volume. Following PR adds support for more advanced options i.e custom `workdir` and `upperdir` for overlayed volumes. So that users can re-use `workdir` and `upperdir` across new containers as well. Usage ```console podman run -it -v /some/path:/data:O,upperdir=/path/persistant/upper,workdir=/path/persistant/work alpine sh ``` Signed-off-by: Aditya R <arajan@redhat.com>

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 28, 2021

flouthoc marked this pull request as draft December 28, 2021 10:41

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 28, 2021

flouthoc marked this pull request as ready for review December 28, 2021 10:41

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 28, 2021

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 28, 2021

flouthoc marked this pull request as draft December 28, 2021 10:55

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 28, 2021

giuseppe reviewed Jan 3, 2022

View reviewed changes

flouthoc force-pushed the volume_overlay_advanced branch from d678b25 to ff133c1 Compare January 3, 2022 09:42

flouthoc force-pushed the volume_overlay_advanced branch 2 times, most recently from 9e603f9 to 5d2652c Compare January 4, 2022 10:34

flouthoc mentioned this pull request Jan 7, 2022

Mount docker.sock #6015

Closed

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 11, 2022

flouthoc force-pushed the volume_overlay_advanced branch from 5d2652c to 90f5de5 Compare January 12, 2022 09:27

openshift-ci bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jan 12, 2022

flouthoc force-pushed the volume_overlay_advanced branch from 2efb5e6 to 641b09a Compare January 17, 2022 11:32

flouthoc force-pushed the volume_overlay_advanced branch from 4a1774f to 96bd3c0 Compare January 27, 2022 15:14

flouthoc requested review from rhatdan and giuseppe January 27, 2022 17:01

rhatdan reviewed Jan 27, 2022

View reviewed changes

libpod/container_internal_linux.go Show resolved Hide resolved

rhatdan reviewed Jan 27, 2022

View reviewed changes

TomSweeneyRedHat reviewed Jan 28, 2022

View reviewed changes

flouthoc force-pushed the volume_overlay_advanced branch from 96bd3c0 to e64e650 Compare January 28, 2022 07:40

giuseppe approved these changes Jan 28, 2022

View reviewed changes

openshift-ci bot assigned giuseppe Jan 28, 2022

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 28, 2022

flouthoc removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 28, 2022

openshift-merge-robot merged commit 1b544b7 into containers:main Jan 28, 2022

flouthoc mentioned this pull request Jun 3, 2022

overlay-volumes: add support for non-volatile upperdir,workdir for overlay volumes #14474

Merged

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

volume: add support for non-volatile `upperdir`,`workdir` for overlay volumes #12712

volume: add support for non-volatile `upperdir`,`workdir` for overlay volumes #12712

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

rhatdan commented Dec 28, 2021

flouthoc commented Dec 28, 2021

giuseppe Jan 3, 2022

flouthoc Jan 3, 2022

giuseppe Jan 3, 2022

flouthoc Jan 3, 2022

rhatdan commented Jan 3, 2022

flouthoc commented Jan 4, 2022

rhatdan commented Jan 10, 2022

flouthoc commented Jan 10, 2022

flouthoc commented Jan 12, 2022 •

edited

Loading

flouthoc commented Jan 14, 2022

rhatdan Jan 27, 2022 •

edited

Loading

flouthoc Jan 28, 2022

rhatdan Jan 27, 2022

flouthoc Jan 28, 2022

TomSweeneyRedHat left a comment

flouthoc commented Jan 28, 2022

rhatdan commented Jan 28, 2022

giuseppe left a comment

openshift-ci bot commented Jan 28, 2022

volume: add support for non-volatile upperdir,workdir for overlay volumes #12712

volume: add support for non-volatile upperdir,workdir for overlay volumes #12712

Conversation

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

flouthoc commented Dec 28, 2021

rhatdan commented Dec 28, 2021

flouthoc commented Dec 28, 2021

giuseppe Jan 3, 2022

Choose a reason for hiding this comment

flouthoc Jan 3, 2022

Choose a reason for hiding this comment

giuseppe Jan 3, 2022

Choose a reason for hiding this comment

flouthoc Jan 3, 2022

Choose a reason for hiding this comment

rhatdan commented Jan 3, 2022

flouthoc commented Jan 4, 2022

rhatdan commented Jan 10, 2022

flouthoc commented Jan 10, 2022

flouthoc commented Jan 12, 2022 • edited Loading

flouthoc commented Jan 14, 2022

rhatdan Jan 27, 2022 • edited Loading

Choose a reason for hiding this comment

flouthoc Jan 28, 2022

Choose a reason for hiding this comment

rhatdan Jan 27, 2022

Choose a reason for hiding this comment

flouthoc Jan 28, 2022

Choose a reason for hiding this comment

TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

flouthoc commented Jan 28, 2022

rhatdan commented Jan 28, 2022

giuseppe left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Jan 28, 2022

volume: add support for non-volatile `upperdir`,`workdir` for overlay volumes #12712

volume: add support for non-volatile `upperdir`,`workdir` for overlay volumes #12712

flouthoc commented Jan 12, 2022 •

edited

Loading

rhatdan Jan 27, 2022 •

edited

Loading