File Upload API

Each "Upload" step has an optional feature, for the tech-savvy: "File-upload API"

When you enable the feature, Workbench will supply a CreateFileUrl and ApiToken.

Here are the steps -- in any programming language -- for uploading a file:

1. POST to CreateFileUrl

Send a POST request to CreateFileUrl, passing ApiToken as a bearer token. The request body must be a JSON object with "filename" and "size" keys.

The response will be JSON. Its `"tusUploadUrl" will be an absolute URL.

In Python, for example, using dummy CreateFileUrl and ApiToken:

import requests
from pathlib import Path

CreateFileUrl = 'https://app.workbenchdata.com/api/v1/workflows/123/steps/step-234/files'
ApiToken = 'some-api-token'
FileToUpload = Path('/path/to/test.csv')

creation_response = requests.post(
    CreateFileUrl,
    headers={"Authorization": f"Bearer {ApiToken}"},
    json={"filename": FileToUpload.basename, "size": FileToUpload.stat().st_size},
)
upload_url = creation_response.json()["tusUploadUrl"]

Or with cURL:

FILENAME="file.csv"
CREATE_FILE_URL="https://app.workbenchdata.com/api/v1/workflows/123/steps/step-234/files"
API_TOKEN="some-api-token"
curl \
    -XPOST \
    -d '{"filename": "'$FILENAME'", "size": "'$(stat --format=%s $FILENAME)'"}' \
    -H 'Content-Type: application/json' \
    -H 'Accept: application/json' \
    -H 'Authorization: Bearer '$API_TOKEN \
    $CREATE_FILE_URL
# ... and read `"tusUploadUrl"`

Keep upload_url secret! It holds powers similar to ApiToken.

2. PATCH to tusUploadUrl

The URL Workbench provides adheres to the TUS Core Protocol, for resumable uploads. To support resume, use a TUS client. (Be sure to set its url to tusUploadUrl.)

If you don't need to support resume, you can simulate the TUS client with a single HTTP request. Send your raw file contents in a PATCH request, with HTTP headers Content-Type, Tus-Resumable and Upload-Offset hard-coded as follows. In Python:

requests.patch(
    upload_url,
    data=FileToUpload.open(),
    headers={
        "Content-Type": "application/offset+octet-stream",
        "Tus-Resumable": "1.0.0",
        "Upload-Offset": "0",
    },
)

Or with cURL:

curl -XPATCH \
    --data-binary @$FILENAME \
    -H 'Content-Type: application/offset+octet-stream' \
    -H 'Tus-Resumable: 1.0.0' \
    -H 'Upload-Offset: 0' \
    $TUS_UPLOAD_URL

3. Watch the Step in Workbench

When your upload finishes, Workbench will update your Step to use the new File.

At this point, you won't be able to upload any more.

4. Beware of special cases

• If you don't complete an upload within a few hours, your partial data will be deleted.