layer: revamp for readability #255

vbatts · 2016-09-04T12:33:28Z

When adding verbiage for restriction on duplicate entries in the tar
archive, it was not immediately clear where this information would be
organized.

This revamp seeks to make the document on the whole more suitable for
implementors and be interpretable for compliance.

Signed-off-by: Vincent Batts vbatts@hashbangbash.com

vbatts · 2016-09-04T12:35:44Z

Fixes #196

@opencontainers/image-spec-maintainers PTAL

For convenience, the view or the document is https://github.com/vbatts/oci-image-spec/blob/clarify_tar_entry_order/layer.md

coolljt0725 · 2016-09-06T12:31:51Z

layer.md

-NOTE: a copy-on-write or union filesystem can make this very efficient:
+### Populate a Comparison Filesystem
+
+Create a new directory and initialize it with an identical copy or snapshot of the prior root filesystem.


Should we make an identical copy more clear here or add a example ? if we just cp -r rootfs-c9d-v1 rootfs-c9d-v1.s1, this is not an identical copy ,the timestamp is changed and I think the timestamp is important to generate diff. To make a copy identical use cp, we should along with -p flag.

I wondered about that. Something like cp -pav <src> <dst> or rsync -avPHS <src> <dst> or so on. These are all very Linux centric, which may be fine for starters, but this line is more of the rule, rather than the example.
Perhaps, having a statement of the files, their contents, and all available attributes (timestamps, mode, owner, group, xattrs, etc)?

Yes, identical should be more thoroughly defined

@jonboulle so, then provide the reader with example commands to accomplish this?
I reckon "identical" could get into inode discussions and otherwise. Though just a couple lines down the attributes are mentioned. Perhaps this ought to itemize the important attributes when making a copy?

If we are defining tar as the distributable format, then can't we lean on that?

@jonboulle fair, though most tar usages are through the golang library and up to an implementer.
Something like a tar command would be tar --xattrs --selinux --acls -c . where selinux is supported and step back from there?

updated @coolljt0725 @jonboulle how it that?

jonboulle · 2016-09-06T18:01:16Z

layer.md

+### Populate a Comparison Filesystem
+
+Create a new directory and initialize it with an identical copy or snapshot of the prior root filesystem.
+Any changes to the snapshot MUST not change or affect the directory it was copied from.


ah yeah. got it

jonboulle · 2016-09-06T18:04:28Z

layer.md


-The resulting Tar archive for `f60c56784b83` has the following entries:
+- Added and modified files and directories in their entirety
+- Deleted files or directory marked with a [whiteout file](#whiteouts)


directories

stevvooe · 2016-09-06T23:40:37Z

layer.md


-An example of creating an Image Filesystem Changeset follows.
+A layer changeset is the distributable difference between filesystems.


Maybe something like this:

A layer encodes a distributable changeset between filesystems.

wking · 2016-09-07T22:31:27Z

layer.md

+
+### File Attributes
+
+File attributes that are provided for Additions and Modifications include:


We should replace “are provided” with something testable. I expect at least:

Implementations MUST support the following file attributes for Additions and Modifications:

Although I'd still rather get out of the file-attribute-requiring business or cite a tar-like spec such as pax.

@wking your chain of links and footnotes kill me.
The options your proposing are getting in the weeds. The requirement are known and have been worked with up to this point. If there are features needed to be added to tools, I'm certain folks will do what they need to do.

On Wed, Sep 07, 2016 at 06:32:48PM -0700, Vincent Batts wrote:

+File attributes that are provided for Additions and Modifications include:

@wking your chain of links and footnotes kill me.

Sorry, it's hard to preserve the continuity of a sub-thread when
GitHub keeps hiding it on us :p. Links and footnotes are the best I
can do to make the older conversation discoverable without rehashing
all of it.

The options your proposing are getting in the weeds. The requirement
are known and have been worked with up to this point. If there are
features needed to be added to tools, I'm certain folks will do what
they need to do.

That sounds like (a) (Punt the specifics to users) from 1, and I'm
ok with that. However, in that case, I see no particular use to call
out specific file types or attributes. If a user wants to put some
oddball filetype/attribute into the tarball, more power to them. If
an implementer wants to only support regular files, directories, and
mode (ignoring or erroring on symlinks, owner, mtime, etc.), more
power to them as well. Sitting in the middle with a half-done spec
(e.g. requiring ‘xattrs’ but not defining it) doesn't seem helpful.

vbatts · 2016-09-08T02:09:20Z

@philips broken link? what was missed in the rebase?

coolljt0725 · 2016-09-08T10:32:42Z

layer.md

+See [Change Types](#change-types) for more details on changes.
+
+For example, add a directory at `/etc/my-app.d` containing a default config file, removing the existing config file.
+Also a change (be it change in attribute or file content) to `./bin/my-app-tools` binary to handle the config layout change.


be it change ? typo here? I don't quite understand this from my english

It may be ok as it stands, but I think it would read more clearly if we dropped "be it change".

When adding verbiage for restriction on duplicate entries in the tar archive, it was not immediately clear where this information would be organized. This revamp seeks to make the document on the whole more suitable for implementors and be interpretable for compliance. Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

philips · 2016-09-08T16:38:39Z

LGTM

@vbatts the link was to a comment that is now hidden because you fixed the issue. Thanks!

philips · 2016-09-08T16:55:32Z

@opencontainers/image-spec-maintainers please take a look and LGTM

stevvooe · 2016-09-08T20:06:32Z

layer.md

+
+### File Types
+
+Throughout this document section, the use of word "files" includes:


"or entries" maybe?

I just submitted a PR to your branch using the edit feature... give that a try?

ah yeah. Now that this is merge'able, can I carry that to a new PR?

stevvooe · 2016-09-08T20:11:44Z

LGTM

Nits can be taken care of in later PRs.

Thanks for doing this. This was greatly needed.

This didn't make it into 17c1f00 (*: rename serialization.md -> config.md, 2016-09-13, opencontainers#305), but I think we still want it [1,2,3]. While I was touching headers, I shifted the informative example section down under the normative property definitions, and move the normative whitespace rule up out of the example section. [1]: opencontainers#255 (comment) [2]: opencontainers#305 (comment) [3]: opencontainers#305 (comment) Signed-off-by: W. Trevor King <wking@tremily.us>

This didn't make it into 17c1f000 (*: rename serialization.md -> config.md, 2016-09-13, #305), but I think we still want it [1,2,3]. While I was touching headers, I shifted the informative example section down under the normative property definitions, and move the normative whitespace rule up out of the example section. [1]: opencontainers/image-spec#255 (comment) [2]: opencontainers/image-spec#305 (comment) [3]: opencontainers/image-spec#305 (comment) Signed-off-by: W. Trevor King <wking@tremily.us>

vbatts force-pushed the clarify_tar_entry_order branch from b7e2fec to 6c8bf26 Compare September 4, 2016 12:37

vbatts added this to the v0.5.0 milestone Sep 4, 2016

coolljt0725 reviewed Sep 6, 2016
View reviewed changes

vbatts force-pushed the clarify_tar_entry_order branch from 6c8bf26 to c375847 Compare September 6, 2016 18:00

jonboulle reviewed Sep 6, 2016
View reviewed changes

vbatts force-pushed the clarify_tar_entry_order branch from c375847 to 3cae2d8 Compare September 6, 2016 18:03

jonboulle reviewed Sep 6, 2016
View reviewed changes

jonboulle mentioned this pull request Sep 6, 2016

Fix id_desc link in layer.md #237

Closed

vbatts force-pushed the clarify_tar_entry_order branch 7 times, most recently from 5d71bad to 728e772 Compare September 6, 2016 21:25

stevvooe reviewed Sep 6, 2016
View reviewed changes

wking reviewed Sep 7, 2016
View reviewed changes

vbatts force-pushed the clarify_tar_entry_order branch from 4b40467 to 62565b3 Compare September 8, 2016 02:07

coolljt0725 reviewed Sep 8, 2016
View reviewed changes

vbatts force-pushed the clarify_tar_entry_order branch from 62565b3 to f5a91ff Compare September 8, 2016 15:56

philips mentioned this pull request Sep 8, 2016

serialization: clarify order of application of tar entries #196

Closed

stevvooe reviewed Sep 8, 2016
View reviewed changes

vbatts merged commit cddf362 into opencontainers:master Sep 8, 2016

vbatts deleted the clarify_tar_entry_order branch September 8, 2016 21:18

wking mentioned this pull request Sep 13, 2016

*: rename serialization.md -> config.md #305

Merged

This was referenced Sep 14, 2016

config: Fix "Serialization" -> "Configuration" in the title #315

Merged

Replace 'base image' with 'base layer' #312

Closed

wking mentioned this pull request Sep 26, 2016

layer: Require ustar (originally from IEEE Std 1003.1-1988 but linking IEEE Std 1003.1-2013) #342

Closed

wking mentioned this pull request Oct 20, 2016

manifest: Explicitly base manifest.layers on an empty directory #318

Closed

wking mentioned this pull request Sep 21, 2017

Reconsider "Sparse files SHOULD NOT be used" restriction? #733

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

layer: revamp for readability #255

layer: revamp for readability #255

vbatts commented Sep 4, 2016 •

edited

Loading

vbatts commented Sep 4, 2016

coolljt0725 Sep 6, 2016

vbatts Sep 6, 2016

jonboulle Sep 6, 2016

vbatts Sep 6, 2016

jonboulle Sep 6, 2016

vbatts Sep 6, 2016

vbatts Sep 6, 2016

jonboulle Sep 6, 2016

vbatts Sep 6, 2016

jonboulle Sep 6, 2016

stevvooe Sep 6, 2016

vbatts Sep 7, 2016

wking Sep 7, 2016

vbatts Sep 8, 2016

wking Sep 8, 2016

vbatts commented Sep 8, 2016

coolljt0725 Sep 8, 2016

wking Sep 8, 2016

philips commented Sep 8, 2016 •

edited by caniszczyk

Loading

philips commented Sep 8, 2016

stevvooe Sep 8, 2016

stevvooe Sep 8, 2016

vbatts Sep 8, 2016

stevvooe commented Sep 8, 2016 •

edited by caniszczyk

Loading


		An example of creating an Image Filesystem Changeset follows.
		A layer changeset is the distributable difference between filesystems.


		### File Attributes

		File attributes that are provided for Additions and Modifications include:


		### File Types

		Throughout this document section, the use of word "files" includes:

layer: revamp for readability #255

layer: revamp for readability #255

Conversation

vbatts commented Sep 4, 2016 • edited Loading

vbatts commented Sep 4, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vbatts commented Sep 8, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philips commented Sep 8, 2016 • edited by caniszczyk Loading

philips commented Sep 8, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevvooe commented Sep 8, 2016 • edited by caniszczyk Loading

vbatts commented Sep 4, 2016 •

edited

Loading

philips commented Sep 8, 2016 •

edited by caniszczyk

Loading

stevvooe commented Sep 8, 2016 •

edited by caniszczyk

Loading