feat(archive): UntarStream and TarStream

[BlackAsLight]

Following on from this pull request, #4538, I didn't know how to revert the changes on that branch so just made a new one since the idea is that I'm creating a new API and not replacing one.

Closes #1658

Usage Examples

import { TarStream, type TarStreamInput } from "@std/archive/tar-stream"

await ReadableStream.from<TarStreamInput>([
  { path: "potato/" },
  {
    path: "deno.json",
    size: (await Deno.stat("deno.json")).size,
    readable: (await Deno.open("deno.json")).readable,
  },
])
  .pipeThrough(new TarStream())
  .pipeThrough(new CompressionStream("gzip"))
  .pipeTo((await Deno.create("archive.tar.gz")).writable)

import { UntarStream } from "@std/archive/untar-stream"
import { dirname, normalize } from "@std/path"

const scope = "./archive/"
for await (
  const entry of (await Deno.open("archive.tar.gz"))
  .readable
  .pipeThrough(new DecompressionStream("gzip"))
  .pipeThrough(new UntarStream())
) {
  const path = normalize(scope + entry.path)
  await Deno.mkdir(dirname(path), { recursive: true })
  await entry.readable?.pipeTo((await Deno.create(path)).writable)
}

[iuioiua]

From a quick skim, this PR shows promise, but there are opportunities to simplify and make the implementation more straightforward to understand. @crowlKats, can you see this as a viable alternative to #1985? If so, I'll go ahead and help polish this up.

[crowlKats]

The reason why this PR is wroking and mine is not is because this one does not use byte streams. Not sure if this should be a blocking action or not, but its definitively a downgrade when comparing to the non-stream API and doesnt line up with other streams we do in CLI that do file or io handling.

That aside, as I discussed with @kt3k, we are a bit concerned that since this PR implements tar from scratch, this will require a lot more exhaustive & thorough tests

[BlackAsLight]

... when comparing to the non-stream API and doesnt line up with other streams we do in CLI that do file or io handling.

With regards to this comment, I'd have assumed there was an intention to remove all Deno references, making it compatible with other runtimes and therefore not offer file or io handling.

Although that was 2y ago now so things may have changed.

[crowlKats]

that has nothing to do with what I have said. I am talking about streams behaviour being consistent across similar purpouses, in this case being byte streams instead of normal streams

[iuioiua]

@BlackAsLight, we're still determining whether to pursue this PR or #1985. We'll come back to you once a decision has been made.

[codecov]

Codecov Report

Attention: Patch coverage is 63.54916% with 152 lines in your changes are missing coverage. Please review.

Project coverage is 91.13%. Comparing base (8c87b7f) to head (dacaea5).

Files	Patch %	Lines
archive/tar_stream.ts	54.59%	83 Missing and 1 partial ⚠️
archive/untar_stream.ts	70.68%	68 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4548      +/-   ##
==========================================
- Coverage   91.44%   91.13%   -0.31%     
==========================================
  Files         480      482       +2     
  Lines       37324    37741     +417     
  Branches     5320     5391      +71     
==========================================
+ Hits        34132    34397     +265     
- Misses       3136     3287     +151     
- Partials       56       57       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[BlackAsLight]

Looking into what bytes streams are and how they work, I do think I could implement them into both streams. At least for when they're serving content out. I don't think I'd run into the same problem you're facing in implementing byte streams with it closing early or something. Although I am yet to actually test it.

On a side note the byte streams examples on MDN make use of both the byteRequest and enqueue for pushing, but my implementation only works on a pulling method so it would only ever be doing one or the other based off what the user provided in the options.

[BlackAsLight]

I don't really understand why, but for some reason when I added byte support for TarStream it consumed the text variable from the tests instead of copying them. Even when { mode: 'byob' } wasn't provided.

Adding byte support for the UnTarStream does seem doable from my understanding of it, but will be more complex.

Even though you haven't decided if you want to pursue my implementation or not. I'm still going to try and make the improvements in the mean time as I have the time.

[iuioiua]

Leo and I looked at this and we're mostly happy with it. Few things we'd like to see:

1. Reverting UntarStream behavior to prioritize ergonomics.
2. Throwing with appropriate error class instances. E.g. RangeError or TypeError, etc.
3. Turning validation functions into assertion functions.

[iuioiua]

Just a few small bits remaining. Once they're addressed, this will LGTM! @kt3k @crowlKats, PTAL.

[iuioiua]

@crowlKats, did you still want to add a type field to these interfaces?

[crowlKats]

yes, to easily distinguish in code what entry is what kind

[iuioiua]

I've made a few tweaks, including adding a type: "file" | "directory" field for TarStream. Now, LGTM! Thank you for your work, @BlackAsLight. And thank you to all that reviewed and contributed to this PR! Let's test and continue to iterate on this experimental API.