Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add delete tracking and symlink/unlink to OverlayFS #8857

Merged
merged 13 commits into from
Oct 5, 2023
Merged

Add delete tracking and symlink/unlink to OverlayFS #8857

merged 13 commits into from
Oct 5, 2023

Conversation

lettertwo
Copy link
Contributor

@lettertwo lettertwo commented Feb 25, 2023 •
edited
Loading

NOTE: This PR was originally titled `Add CopyOnWriteToMemoryFS` (expand to see orignal description)

CopyOnWriteToMemoryFS is effectively a subclass of OverlayFS with a few key differences:

  • it tracks deletes (with OverlayFS, a deleted file won't report as deleted as long as it exists on the underlying FS)
  • it tracks symbolic links and unlinks
  • it expects only a 'readable' FS, whereas OverlayFS also expects a 'writable' FS.
  • It layers an internally managed 'writable' MemoryFS over the 'readable' FS, (like OverlayFS does), but it only falls back to the 'readable' FS for read operations if the read path has not been modified or deleted.

The motivation for this addition is to allow testing of deletions of files that are otherwise found on the underlying FS (in lieu of copying them to the memory FS first).

Alternatives

This originally was going to be added to OverlayFS, but ultimately, this represents a use case distinct from what OverlayFS provides. OverlayFS incidentally covers many of the 'copy-on-write' use cases, but that is not its primary goal.

This PR makes the following additions to OverlayFS:

  • it tracks deletes (previously, a deleted file wouldn't report as deleted as long as it exists on the underlying FS)
  • it only falls back to the 'readable' FS for read operations if the read path has not been modified or deleted.
  • it tracks symbolic links and unlinks

The motivation for these additions is to allow testing of deletions of files that are otherwise found on the underlying FS (in lieu of copying them to the memory FS first).

@parcel-benchmark
Copy link

parcel-benchmark commented Feb 25, 2023 •
edited
Loading

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.53s +18.00ms
Cached 254.00ms +4.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

React HackerNews ✅

Timings

Description Time Difference
Cold 4.10s -9.00ms
Cached 435.00ms +28.00ms ⚠️

Cold Bundles

Bundle Size Difference Time Difference
dist/PermalinkedComment.e9dc4a75.js 3.92kb +0.00b 379.00ms +62.00ms ⚠️
dist/UserProfile.8945a243.js 1.38kb +0.00b 379.00ms +62.00ms ⚠️
dist/NotFound.8b44a81d.js 269.00b +0.00b 379.00ms +63.00ms ⚠️
dist/logo.8dd07848.png 244.00b +0.00b 275.00ms +49.00ms ⚠️

Cached Bundles

Bundle Size Difference Time Difference
dist/PermalinkedComment.e9dc4a75.js 3.92kb +0.00b 339.00ms +19.00ms ⚠️
dist/UserProfile.8945a243.js 1.38kb +0.00b 339.00ms +20.00ms ⚠️
dist/NotFound.8b44a81d.js 269.00b +0.00b 339.00ms +20.00ms ⚠️

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 35.17s -441.00ms
Cached 2.31s -5.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/media-viewer.38e3999a.js 536.13kb +0.00b 11.91s +1.01s ⚠️
dist/ConfigPanelFieldsLoader.182d39bc.js 303.43kb +0.00b 8.27s -614.00ms 🚀
dist/card.d06de810.js 138.91kb +0.00b 8.26s -610.00ms 🚀
dist/ElementBrowser.e8f01080.js 61.94kb +0.00b 8.26s -611.00ms 🚀
dist/esm.bfca2115.js 59.30kb +0.00b 8.27s -614.00ms 🚀
dist/ConfigPanelFieldsLoader.2b7c03be.js 15.74kb +0.00b 8.26s -611.00ms 🚀
dist/ui.8c117104.js 14.48kb +0.00b 8.26s -611.00ms 🚀
dist/ConfigPanelFieldsLoader.5dfde67d.js 13.63kb +0.00b 8.26s -611.00ms 🚀
dist/pdfRenderer.01deafa1.js 12.04kb +0.00b 8.27s -614.00ms 🚀
dist/mobile-upload.3baad8e4.js 7.79kb +0.00b 8.27s -613.00ms 🚀
dist/mobile-upload.7a892a37.js 7.79kb +0.00b 8.27s -613.00ms 🚀
dist/ru.0cf3f40e.js 2.81kb +0.00b 8.26s -612.00ms 🚀
dist/uk.282f23b1.js 2.76kb +0.00b 8.26s -612.00ms 🚀
dist/codeViewerRenderer.51140ec8.js 2.61kb +0.00b 11.70s +2.81s ⚠️
dist/th.137e1013.js 2.60kb +0.00b 8.26s -612.00ms 🚀
dist/vi.b46097db.js 2.09kb +0.00b 8.26s -611.00ms 🚀
dist/pt_BR.b9e37d37.js 2.06kb +0.00b 8.13s +1.90s ⚠️
dist/tr.c85d90a9.js 2.03kb +0.00b 8.26s -612.00ms 🚀
dist/sv.1c06c95c.js 1.98kb +0.00b 8.26s -611.00ms 🚀
dist/zh_TW.b7c55aa6.js 1.86kb +0.00b 8.26s -611.00ms 🚀
dist/zh.b01fe721.js 1.84kb +0.00b 8.26s -611.00ms 🚀
dist/workerHasher.540c9790.js 1.56kb +0.00b 8.26s -610.00ms 🚀
dist/workerHasher.c840c607.js 1.56kb +0.00b 8.27s -613.00ms 🚀
dist/sk.4be9c93f.js 656.00b +0.00b 8.26s -611.00ms 🚀
dist/pt_PT.e211e609.js 635.00b +0.00b 6.17s -2.24s 🚀
dist/simpleHasher.c14e20b4.js 589.00b +0.00b 8.26s -622.00ms 🚀
dist/simpleHasher.23db7a52.js 589.00b +0.00b 8.27s -613.00ms 🚀
dist/index.html 248.00b +0.00b 11.96s +5.72s ⚠️

Cached Bundles

Bundle Size Difference Time Difference
dist/media-viewer.38e3999a.js 536.13kb +0.00b 11.91s +3.66s ⚠️
dist/EmojiPickerComponent.4a196252.js 188.61kb +0.00b 11.91s +3.65s ⚠️
dist/card.d06de810.js 138.91kb +0.00b 11.20s +2.95s ⚠️
dist/ConfigPanelFieldsLoader.28b428a5.js 82.73kb +0.00b 11.91s +3.65s ⚠️
dist/esm.bfca2115.js 59.30kb +0.00b 11.19s +2.94s ⚠️
dist/pdfRenderer.01deafa1.js 12.04kb +0.00b 11.20s +2.95s ⚠️
dist/mobile-upload.7a892a37.js 7.79kb +0.00b 11.19s +2.94s ⚠️
dist/pl.bce591be.js 2.25kb +0.00b 7.20s +1.16s ⚠️
dist/pt_BR.b9e37d37.js 2.06kb +0.00b 7.20s +1.16s ⚠️
dist/heading3.73972e33.js 1.35kb +0.00b 6.01s +623.00ms ⚠️
dist/heading2.c27d912d.js 1.17kb +0.00b 6.01s +623.00ms ⚠️
dist/heading4.be08fc9e.js 1.12kb +0.00b 6.01s +623.00ms ⚠️
dist/pt_PT.e211e609.js 635.00b +0.00b 7.20s +1.15s ⚠️

Three.js ✅

Timings

Description Time Difference
Cold 2.93s -76.00ms
Cached 350.00ms +22.00ms ⚠️

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

Click here to view a detailed benchmark overview.

@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from 446d6a4 to 3395e57 Compare February 25, 2023 03:22
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from f397949 to ea3ee72 Compare February 26, 2023 20:09
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from 3395e57 to 25b1ea1 Compare February 26, 2023 20:10
@mischnic mischnic mentioned this pull request Feb 28, 2023
Closed
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from a588195 to bf638c2 Compare June 27, 2023 17:05
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from 594ac02 to 90495b5 Compare June 30, 2023 22:14
@lettertwo lettertwo changed the title Add CopyOnWriteToMemoryFS Add delete tracking and symlink/unlink to OverlayFS Jun 30, 2023
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from fb39c4f to 62c473b Compare July 10, 2023 20:36
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from 90495b5 to 1b7d54f Compare July 10, 2023 20:58
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from 62c473b to 58af4a1 Compare July 10, 2023 22:08
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch 2 times, most recently from 926da5b to ffcd4ca Compare July 12, 2023 18:14
@lettertwo lettertwo mentioned this pull request Jul 17, 2023
Merged
2 tasks
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from cc6c1d8 to 90c9e90 Compare July 25, 2023 20:43
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from ffcd4ca to d3f08a4 Compare July 25, 2023 20:53
@lettertwo lettertwo force-pushed the lettertwo/fs-fixture branch from 90c9e90 to 4a248b8 Compare August 15, 2023 19:55
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch 2 times, most recently from 6716803 to 4926544 Compare August 16, 2023 18:30
Base automatically changed from lettertwo/fs-fixture to v2 August 16, 2023 23:23
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from 4926544 to a575496 Compare August 16, 2023 23:35
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch 2 times, most recently from 13f4f4f to c6c3fa0 Compare October 4, 2023 17:32
@lettertwo lettertwo force-pushed the lettertwo/copy-on-write-to-memory-fs branch from c6c3fa0 to 0463c5b Compare October 4, 2023 22:29
mattcompiles
mattcompiles approved these changes Oct 4, 2023
View reviewed changes
Copy link
Contributor

@mattcompiles mattcompiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📁

@lettertwo lettertwo merged commit ce58970 into v2 Oct 5, 2023
@lettertwo lettertwo deleted the lettertwo/copy-on-write-to-memory-fs branch October 5, 2023 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattcompiles mattcompiles mattcompiles approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lettertwo @parcel-benchmark @mattcompiles