Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Handle file uploads #3086

Closed
wants to merge 7 commits into from

Conversation

pixelmund
Copy link
Contributor

@pixelmund pixelmund commented Dec 21, 2021

Hello,

i've gone ahead and tackled the file upload issue, this fixes #70

This implementation uses @fastify/busboy which is the forked version of busboy by fastify.
Busboy seems to support any kind of environment and is used by many people, so it should be battle tested enough to include it in Kit. I've researched a bit about netlify, vercel etc.. Since serverless environments normally don't have access to the filesystem this can still be used to upload the files to AWS or any other provider.

I've added two methods to ReadOnlyFormData: file(key:string) and files(key: string) those basically have the same functionality as get(key: string) and getAll(key: string) but it get's the files instead. If you have some better ideas for the api/names, feel free to let me know.

I have added a new test project, because i couldn't find one for body parsing. The test project contains tests for urlencoded and multipart forms.

To test the file uploads i've installed formdata-node if you guys know a better way to test this without installing the dev-dependency just let me know.

Thanks for reading this wall of text, i hope we can get file uploads out soon because they would be handy for some upcoming projects at work and i'm sure many people are waiting for this.

Documentation is pretty much TODO, i'm not really good in writing those but i'm sure someone can tackle those for me.

EDIT:
We might want to make the busboy options configurable inside svelte.config.js?

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
  • This message body should clearly illustrate what problems it solves.
  • Ideally, include a test that fails without this PR but passes with it.

Tests

  • Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

  • If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpx changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

@changeset-bot
Copy link

changeset-bot bot commented Dec 21, 2021

🦋 Changeset detected

Latest commit: 3917890

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@sveltejs/kit Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pboling
Copy link

pboling commented Dec 22, 2021

Thanks for tackling this!

@Rich-Harris
Copy link
Member

Thank you. Unfortunately it's more complicated than this!

The problem is only partly about parsing multipart forms. (@fastify/busboy seems like a mostly-decent solution to this problem, but its use of the [text-decoding] polyfill renders it unsuitable — it's much too large a dependency. TextDecoder is supported natively in all the environments we care about, so at the very least we would need to create our own fork of busboy that omits the polyfill.)

The real problem here is that files can be large. Lambda-based environments have a fairly restrictive maximum payload (in these cases it's probably better to use the endpoint to generate a signed upload URL for S3 or whatever, rather than handling uploads directly), but for environments where it is practical for endpoints to handle large files, it's essential that they can do so in a streaming fashion so that they don't have to keep the entire body in memory. (In this PR, this line is incorrect — in the case where data is uploaded in multiple chunks, the file object will only contain the last of those chunks.)

Of course, it's not just files. You can post arbitrarily large data with an arbitrary content-type — for example, you might want to upload some image data from a <canvas>, for which you'd likely post the binary data with an image/png content type:

function save_canvas() {
  saving = true;

  canvas.toBlob(async blob => {
    await fetch('/save-image', {
      method: 'POST',
      body: blob
    });

    saving = false;
  });
}

So we probably need to rethink how request bodies are handled across the board. I've opened a mini-RFC — #70 (comment) — so I'll close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle file uploads
3 participants