Adding Resumable Uploads

This walkthrough shows how to add support for resumable uploads using tus to a Rails app. The flow will go like this:

1. User selects file(s)
2. Files are uploaded asynchronously to a resumable upload endpoint
3. Uploaded file JSON data is written to a hidden field
4. Form is submitted instantaneously as it only has to submit the JSON data
5. JSON data is assigned to the Shrine attachment attribute

NOTE: If you would like to have resumable uploads directly to S3, see the uppy-s3_multipart gem.

1. Installation

Add Shrine, aws-sdk-s3, tus-ruby-server and shrine-tus to the Gemfile:

# Gemfile

gem "shrine", "~> 3.0"
gem "aws-sdk-s3", "~> 1.14"
gem "tus-server", "~> 2.3"
gem "shrine-tus", "~> 2.1"

and run bundle install.

2. Initializer

Add your S3 credentials to your application:

$ rails credentials:edit

s3:
  bucket:            "<YOUR_BUCKET>"
  region:            "<YOUR_REGION>"
  access_key_id:     "<YOUR_ACCESS_KEY_ID>"
  secret_access_key: "<YOUR_SECRET_ACCESS_KEY>"
# ...

Then create an initializer which configures your S3 storage with those credentials and loads default plugins:

# config/shrine.rb

require "shrine"
require "shrine/storage/s3"
require "shrine/storage/tus"

s3_options = Rails.application.credentials.s3

Shrine.storages = {
  cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
  store: Shrine::Storage::S3.new(**s3_options),
  tus:   Shrine::Storage::Tus.new,
}

Shrine.plugin :activerecord           # load Active Record integration
Shrine.plugin :cached_attachment_data # for retaining cached file on form redisplays
Shrine.plugin :restore_cached_data    # refresh metadata for cached files

Notice the additional :tus storage, which will be used for downloading files from the tus server.

3. Migration

Add the <attachment>_data text or JSON column to the table to which you want to add the attachment:

$ rails generate migration add_video_data_to_movies video_data:text # or :jsonb

This should generate the following migration:

class AddVideoDataToMovies < ActiveRecord::Migration
  def change
    add_column :movies, :video_data, :text # or :jsonb
  end
end

Run rails db:migrate to apply the migration.

4. Attachment

Create an uploader for the types of files you'll be uploading, and configure it to use tus storage for cache:

# app/uploaders/video_uploader.rb

class VideoUploader < Shrine
  # use Shrine::Storage::Tus for temporary storage
  storages[:cache] = storages[:tus]
end

and add an attachment attribute to your model:

# app/models/movie.rb

class Movie < ApplicationRecord
  include VideoUploader::Attachment(:video)

  validates_presence_of :video
end

5. Form

In your form you can now add form fields for the attachment attribute:

<!-- app/views/movies/_form.html.erb -->

<%= form_for @movie do |f| %>
  <!-- ... -->
  <div>
    <%= f.label :video %>
    <%= f.hidden_field :video, value: @movie.cached_video_data, class: "upload-data" %>
    <%= f.file_field :video, class: "upload-file" %>
  </div>
<% end %>

<p class="upload-preview"></p>

In your controller make sure to allow the attachment param:

# app/controllers/movies_controller.rb

class MoviesController < ApplicationController
  # ...
  def create
    @movie = Movie.new(movie_params)

    if @movie.save
      redirect_to @movie
    else
      render :new
    end
  end
  # ...

  private

  def movie_params
    params.require(:movie).permit(..., :video) # permit attachment param
  end
end

6. Direct upload

We can now add asynchronous direct uploads to the mix. We'll be using Uppy and its Tus plugin, which will upload selected files to our tus-ruby-server.

6a. Tus Server

We'll first create an initializer that configures our tus server to use AWS S3 storage:

# config/initializers/tus.rb
# ...
require "tus/server"
require "tus/storage/s3"

s3_options = Rails.application.credentials.s3

Tus::Server.opts[:storage] = Tus::Storage::S3.new(**s3_options)
Tus::Server.opts[:redirect_download] = true # redirect download requests to S3

Then we'll mount it in our routes:

# config/routes.rb

Rails.application.routes.draw do
  # ...
  mount Tus::Server => "/files"
end

6b. Uppy

Now we can setup Uppy to do the direct uploads. First we'll add the package to our bundle (we're assuming you're using [webpacker]):

$ yarn add uppy

Now we can setup direct uploads, where selected files will go to Shrine's tus server, and upload result will be written to the hidden attachment field:

// app/javascript/fileUpload.js

import 'uppy/dist/uppy.min.css'

import {
  Core,
  FileInput,
  Informer,
  ProgressBar,
  Tus,
} from 'uppy'

function fileUpload(fileInput) {
  const hiddenInput = document.querySelector('.upload-data'),
        uploadPreview = document.querySelector('.upload-preview'),
        formGroup = fileInput.parentNode

  // remove our file input in favour of Uppy's
  formGroup.removeChild(fileInput)

  const uppy = Core({
      autoProceed: true,
    })
    .use(FileInput, {
      target: formGroup,
    })
    .use(Informer, {
      target: formGroup,
    })
    .use(ProgressBar, {
      target: uploadPreview,
    })
    .use(Tus, {
      endpoint: '/files',     // path to our tus server
      chunkSize: 5*1024*1024, // required unless tus-ruby-server is running on Falcon
    })

  uppy.on('upload-success', (file, response) => {
    // show information about the uploaded file
    uploadPreview.innerHTML = `name: ${file.name}, type: ${file.type}, size: ${file.size}`

    // construct uploaded file data from the tus URL
    var uploadedFileData = {
      id: response.uploadURL,
      storage: "cache",
      metadata: {
        filename: file.name,
        size: file.size,
        mime_type: file.type,
      }
    }

    // set hidden field value to the uploaded file data so that it's submitted
    // with the form as the attachment
    hiddenInput.value = JSON.stringify(uploadedFileData)
  })
}

export default fileUpload

// app/javascript/packs/application.js
// ...
import fileUpload from 'fileUpload'

// listen on 'turbolinks:load' instead of 'DOMContentLoaded' if using Turbolinks
document.addEventListener('DOMContentLoaded', () => {
  document.querySelectorAll('.upload-file').forEach(fileInput => {
    fileUpload(fileInput)
  })
})

And that's it, now when a video is selected it will be asynchronously uploaded to your tus server, showing a progress bar. The upload will be automatically resumed in case of any interruptions.

See Also

• Adding Direct App Uploads
• Adding Direct S3 Uploads
• uppy-s3_multipart
• Demo