Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add correct test for 1515 #5364

Merged
merged 1 commit into from
Aug 13, 2018
Merged

add correct test for 1515 #5364

merged 1 commit into from
Aug 13, 2018

Conversation

Stebalien
Copy link
Member

@Stebalien Stebalien commented Aug 10, 2018

When checking to see if GC fully reverses an ipfs add, we should check the size of the actual files, not the directory sizes. A bunch of empty directories won't use that much space and really shouldn't count against GC.

closes #1515

I'm not sure why this has been sitting around for so long but I never seem to have created this PR.

@Stebalien Stebalien requested a review from Kubuxu as a code owner August 10, 2018 00:25
@ghost ghost assigned Stebalien Aug 10, 2018
@ghost ghost added the status/in-progress In progress label Aug 10, 2018
@@ -394,6 +394,27 @@ file_size() {
$_STAT "$1"
}

directory_size() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just something like du -sb --apparent-size?

Copy link
Member Author

@Stebalien Stebalien Aug 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Didn't know about it.
  2. MacOS 😢.

@Stebalien Stebalien requested a review from magik6k August 11, 2018 01:52
@Stebalien Stebalien added need/review Needs a review and removed status/in-progress In progress labels Aug 11, 2018
Copy link
Member

@magik6k magik6k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 nit, then lgtm

@@ -409,3 +430,9 @@ convert_tcp_maddr() {
port_from_maddr() {
echo $1 | awk -F'/' '{ print $NF }'
}


fail() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is test_fsh - https://github.com/ipfs/go-ipfs/blob/1054826ac49f66d329d3088143f1a7f2b9930dcc/test/ipfs-test-lib.sh#L22-L27 and it's used all over sharness tests - like test_cmp expected unique_hash || test_fsh cat add_output

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nice!

When checking to see if GC fully reverses an `ipfs add`, we should check the
size of the actual files, not the directory sizes. A bunch of empty directories
won't use *that* much space and really shouldn't count against GC.

closes #1515

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>
@Stebalien Stebalien added RFM and removed status/in-progress In progress need/review Needs a review labels Aug 13, 2018
@Stebalien Stebalien merged commit 0e5ed59 into master Aug 13, 2018
@Stebalien Stebalien deleted the fix/1515 branch August 13, 2018 17:02
@a4d442663580
Copy link

A bunch of empty directories won't use that much space and really shouldn't count against GC.

Then the process is still irreversible. Accumulating empty dirs doesn't sound sustainable either. While the effect is negligible for embedded machines, this is not necessarily the case for clusters.

@Stebalien
Copy link
Member Author

It's bounded to 1024 empty directories. Whenever we store an object, we store it at /xx/YYYY...xx. However, when we delete the object, we don't delete the /xx/ directory. Given that xx is base32, that gives us at most 32^2 = 1024 directories (steady state). An alternative is to just create them all up-front.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ipfs repo gc does not fully reverse ipfs add
3 participants