Is this library still supported?

[christophetd]

Hi!

I see there are quite a lot of (unresolved) open issues and forgotten pull requests, which makes me wonder if this library is still supported?

If not, it would be a good idea to mention it in the read me.

Thanks :)

Christophe

[RoryH]

😞

[andrewrk]

I made a fundamental mistake when making this library, which was depending on the official AWS SDK package, which is probably the worst piece of shit JavaScript package I've ever depended on. I put so much careful thought into preventing race conditions and handling every situation correctly and efficiently, only to have my effort wasted by incompetent Amazon engineers. I should have known better, I worked there for a few months before NOPEing the hell out of there.

Look at how much bullshit I've had to wade through:
aws/aws-sdk-js#312

The code example I gave you earlier is simple and effective. Delete all the weird hacks, just use pipe, and everything is fixed.

There's no bug in Node.js. The mystery has been solved. The bug is in aws-sdk-js. Please fix your code.

aws/aws-sdk-js#291
aws/aws-sdk-js#290
aws/aws-sdk-js#313
aws/aws-sdk-js#459

So if this module is going to survive, it must be changed to remove the dependency on the aws sdk js package and instead make direct HTTP requests, like knox already does.

I'm over here working on Genesis DAW and Zig language, not really motivated to put work into this Node.js library unless someone wants to pay me by the hour.

If someone proves to be competent and wants to take over maintenance of this module I will happily hand it over.

[christophetd]

Thank you for your answer!

[thenitai]

@andrewrk I'm sorry to hear of all your issues with the Amazon library. For what it matters, I've updated the embedded AWS-SDK to latest AWS-SDK (2.3.x) and your library still works. Your library is very useful and makes it easy to work with S3, especially as knox (without another version) doesn't support multipart-upload.

[rmharrison]

@andrewrk, @thenitai knox also doesn't appear to support 'sync-like' functionality, which you do a great-job with via uploadDir() and clientUploadDirParams().

Yours is the best of the related libraries we've looked at:

• s3-sync
  • Unmaintained
• A different s3-sync, [node-s3-cli]
  • Built-on AWS SDK
  • no users
• knox
  • No sync

Additionally, sync is not included in the SDK. CLI has many known issues:

• Does not support per-file metadata: S3 sync recursively with per-object metadata aws/aws-cli#2045
• Does not copy metadata for multi-part uploads: Multipart uploads can lead to loss of metadata aws/aws-cli#1145
• Issues with E-tag comparisons (possible reason why files aren't being uploaded): AWS S3 Sync Issues aws/aws-cli#599
• Does not support folder structures (implication: empty directories aren't synced). No plan to support folders: No easy way to copy a single directory to s3 aws/aws-cli#2069

Does anyone have a recommendation for a sync library that I've missed? One that doesn't drop files without warning (thanks aws-sdk 😿)

Recommendations on syncing with per-file metadata?

[kensaggy]

I made a fundamental mistake when making this library, which was depending on the official AWS SDK package, which is probably the worst piece of shit JavaScript package I've ever depended on. I put so much careful thought into preventing race conditions and handling every situation correctly and efficiently, only to have my effort wasted by incompetent Amazon engineers. I should have known better,....

+1

[freethejazz]

@andrewrk Would you be amenable to adding a "Needs Maintainer" message on the readme and repo description, potentially linking to this issue from the readme? You can figure it out based on the last commit date, open issues, etc., but the explicit call out would be nice.