Report completion event from self hosted Companion

[Mosharush]

• We want our users to be able to close the browser while the cloud transfer in the background is in progress.
• We want that Companion will report upon completion without any user interaction / activity.
• Uppy dashboard sends /get/xyz-some-gdrive-fileid request, the client receives the file token for him to communicate and receive updates from the WebSocket.
• Since we want to know when the upload was completed without user interaction, we added another route called /watch/{websocket-token}. Companion opens a client socket to itself on the Companion. Then we wait for a success message from that websocket using the token.
• Sometimes the websocket doesn’t return anything and this solution doesn’t serve us well and often fails.
• We tried using the emitter, but it didn’t work as well, we don’t receive any events. (BTW, Emitter without any parameter returns an TS error… so we can’t use the already initialized emitter)
• We tried to send the same parameters to the emitter, (redisUrl, redisPubScope) but still we didn’t receive any event.

Our dream case suggested solutions:

1. We could use the emitter “on” to receive any new file. Then get the file ID and add listener to upload done event, and then do whatever further processing needed to run on this file.
2. On the /get request to provide successCallback, failCallback so that the flow will be easier to implement and use.

Currently we are stuck as our websocket solution isn’t holding water due to our auto-scale solution we created for Companion on our servers.

What are we doing wrong? Is there something we missed?

[lonormaly]

Same here 😔

[lonormaly]

Hey @mifi @Murderlon , Is someone on this? We really need this sorted somehow.

Is there a way to use a paying support service with the on-premise solution?

Thanks,
Shai

[mifi]

Hi. This was recently implemented in #3544

Did you try the code from the documentation: https://uppy.io/docs/companion/ under "Events"?

I just tested with google drive and it works for me:

  const companionApp = companion.app(companionOptions)

  const { companionEmitter: emitter } = companionApp
  emitter.on('upload-start', ({ token }) => {
    console.log('Upload started', token)

    function onUploadEvent ({ action, payload }) {
      if (action === 'success') {
        emitter.off(token, onUploadEvent) // avoid listener leak
        console.log('Upload finished', token, payload.url)
      } else if (action === 'error') {
        emitter.off(token, onUploadEvent) // avoid listener leak
        console.error('Upload failed', payload)
      }
    }
    emitter.on(token, onUploadEvent)
  })

::1 - - [16/Jul/2022:03:55:13 +0000] "OPTIONS /drive/get/... HTTP/1.1" 204 0 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
::1 - - [16/Jul/2022:03:55:13 +0000] "OPTIONS /drive/get/... HTTP/1.1" 204 0 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
companion: 2022-07-16T03:55:13.260Z [info] companion.client.version uppy client version @uppy/companion-client=2.2.1
companion: 2022-07-16T03:55:13.530Z [debug] 4feea5f4-3689-4bc1-af02-9255916745ffnull Instantiating uploader.
companion: 2022-07-16T03:55:13.534Z [debug] 4feea5f4-3689-4bc1-af02-9255916745ffnull Starting download stream.
companion: 2022-07-16T03:55:14.420Z [debug] 4feea5f4-3689-4bc1-af02-9255916745ffnull Waiting for socket connection before beginning remote download/upload.
companion: 2022-07-16T03:55:14.421Z [debug] 3e307118 uploader.socket.wait waiting for socket connection
::1 - - [16/Jul/2022:03:55:14 +0000] "POST /drive/get/... HTTP/1.1" 200 48 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
companion: 2022-07-16T03:55:14.432Z [info] socket.connect connection received from 3e307118-ca9a-4051-a5e8-07b2993d9266
companion: 2022-07-16T03:55:14.432Z [debug] 3e307118 uploader.socket.wait socket connection received
companion: 2022-07-16T03:55:14.432Z [debug] 4feea5f4-3689-4bc1-af02-9255916745ffnull Socket connection received. Starting remote download/upload.
Upload started 3e307118-ca9a-4051-a5e8-07b2993d9266
companion: 2022-07-16T03:55:14.817Z [debug] 3e307118 uploader.total.progress 0 114142 0.00%
companion: 2022-07-16T03:55:14.821Z [debug] 3e307118 uploader.total.progress 767 114142 0.67%
companion: 2022-07-16T03:55:14.977Z [debug] 3e307118 uploader.total.progress 34461 114142 30.19%
companion: 2022-07-16T03:55:15.458Z [debug] 3e307118 uploader.total.progress 114142 114142 100.00%
companion: 2022-07-16T03:55:15.459Z [debug] 3e307118 cleanup
Upload finished 3e307118-ca9a-4051-a5e8-07b2993d9266 https://tusd.tusdemo.net/files/...

notice the logged "Upload started" and "Upload finished"

if not, which version of companion are you running?

[Mosharush]

The companion.app method returns an Express instance and will not provide any Event Emitter.
Maybe we doing something wrong?

Full relevant code:

import express from 'express';
import companion from '@uppy/companion';
import options from './options';

const companionApp = companion.app(options)
const { companionEmitter: emitter } = companionApp

const app = express();
app.use(options.server.path, companionApp);

[Murderlon]

@Mosharush are you on the latest version?

[Mosharush]

@Mosharush are you on the latest version?

Yes, I saw it's added to the app method but the type file does not align with it:

export function app(optionsArg?: object): import('express').Express;

[Murderlon]

Right! This changed for the upcoming 3.0: #3827. @mifi seems we still need to change the types on the current version and on the 3.x branch as well.

[mifi]

Oh, so it's just a typescript issue? Does it work with ts-ignore?