Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3 missing 1 file #6047

Closed
Adblu opened this issue Mar 29, 2021 · 4 comments
Closed

s3 missing 1 file #6047

Adblu opened this issue Mar 29, 2021 · 4 comments
Assignees
Labels
closed-for-staleness guidance Question that needs advice or information. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. s3

Comments

@Adblu
Copy link

Adblu commented Mar 29, 2021

I have used aws cli tool to move couple of folders named: 2014, 2015, 2016 etc from root directory into

/2015/

into:

/images/2015/

command used:

aws s3 sync s3://path/to/bucket/2014 s3://path/to/bucket/2014/images --profile john@smith.com

When I moved them it seems that there is one file less in each bucket:

Before copying:

Screenshot from 2021-03-26 10-33-36

After coping:

Screenshot from 2021-03-26 10-33-47

Could you help me to understand this phenomena ?

@Adblu Adblu added the needs-triage This issue or PR still needs to be triaged. label Mar 29, 2021
@stobrien89 stobrien89 self-assigned this Mar 30, 2021
@stobrien89 stobrien89 added s3 and removed needs-triage This issue or PR still needs to be triaged. labels Mar 30, 2021
@stobrien89
Copy link
Member

Hi @Adblu,

Thanks for writing in! Just wanted to confirm before we move forward— did you use s3 cp or s3 sync? It sounded like cp from what you wrote, but just wanted to be sure.

@Adblu
Copy link
Author

Adblu commented Mar 30, 2021

@stobrien89 I have used sync. Here is a sample command:

aws s3 sync s3://path/to/bucket/2014 s3://path/to/bucket/2014/images --profile john@smith.com

@stobrien89
Copy link
Member

Hi @Adblu,

Thanks for confirming! There are a few different reasons you may be missing files after a sync, some of which are mentioned in this issue. You can find a good overview of sync behavior here.

Anecdotally, s3 cp --recursive may be a better option for some users, depending on what they're trying to accomplish, and I think that may actually be a better option for you if you're just looking to copy all files from one directory to another. Let me know what you think!

@stobrien89 stobrien89 added guidance Question that needs advice or information. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels Mar 31, 2021
@github-actions
Copy link

github-actions bot commented Apr 7, 2021

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

@github-actions github-actions bot added closing-soon This issue will automatically close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Apr 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-for-staleness guidance Question that needs advice or information. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. s3
Projects
None yet
Development

No branches or pull requests

2 participants