media upload with read stream buffers the entire file into RAM

[agoode]

I am doing this:

drive.files.insert({
    resource: {
    title: 'test.mp4'
    },
    media: {
    mimeType: 'application/octet-stream',
    body: stream
    }
}, view);

test.mp4 is 3.7GB. top shows that node is using 3.862GB of RAM. I noticed this when I used progress-stream and saw it go immediately to 100%.

[rakyll]

It seems like sandwich-stream is ignoring the max number of bytes it should read at this line https://github.com/connrs/node-sandwich-stream/blob/master/lib/sandwich-stream.js#L28, ends up buffering the stream until the end.

@ryanseys, we should be able to pipe fs.ReadableStream to req with minimal effort (https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/lib/storage/index.js#L259). We can drop the dependencies.

[ryanseys]

I do pipe the readable stream if there's no multipart required (no resource). If we need to form the multipart body, I use multipart-stream to build the streamable multipart and pipe that to req.

[rakyll]

I use multipart-stream to build the streamable multipart and pipe that to req.

Yes, but multipart-stream buffers the entire file into memory because SandwichStream is a broken readable stream implementation.

Basic media uploads should be working fine when #235 is merged, and I'll try to fix multipart uploads when I have more bandwidth.