OS design questions

[darkdarkdragon]

Some problems in no particular order:

1. Should broadcasters be configured at start with concrete OS options - say, to require transcoder to supply input OS, or broadcaster should be configured with range of allowed OS options and should negotiate concrete options with transcoder for ever job? Or it should be allowed to configure OS for every job somehow?

2. Save mode - just an idea - it should be allowed to configure broadcaster, say, like now (broadcaster node serves segments for live streaming), plus saving all the segments into OS (and storing manifest with all segments in same OS) - for later replays

3. Now broadcaster keeps only three segments in memory - should OS also delete old segments? Or it should be configurable?

4. If we are serving source segments from broadcaster and asking transcoder to put segments into our S3 - should we also put source segments in our S3? Or manifest will contain urls to broadcaster and urls to S3?

5. For some OS we just need to put urls to manifest, but for some we should download segments and put somewhere else, possibly in other OS (say, we are asking transcoder to put segments into own S3 bucket, but then we should download it from there, and possibly put into own S3 to be served for CDN) - not sure how to configure all this

[j0sh]

Great questions. Maybe it'll help to break down the types of OS:

"Internal" OS: Local HTTP, IPFS
"External" OS: S3

Should broadcasters be configured at start with concrete OS options

I think the only thing that might need explicit configuration is external OS such as S3, and that's optional. So if S3 credentials are set, then S3 is made available. Internal OS should be on by default.

Someday, it might be useful to have flags that disable a particular internal OS backend such as local HTTP or IPFS (and that should be reflected through the TranscoderInfo signaling), but I don't think we need to launch with those flags. We can discuss the details of signaling / negotiation separately, but I think we can build the initial OS support without explicit signaling, aside from credentials for external OS.

to require transcoder to supply input OS

I think external OS should be optional for all roles, and the OS owner chooses whether or not to use it. B shouldn't impose a requirement for T to supply a particular external OS, and vice versa. However, we do need a minimum level of service. Making internal OS mandatory (or at least local HTTP) should be enough, in the absence of S3.

Once we have capability negotiation, then we can think about how to handle incompatibilities in OS preferences. Do you think this is enough to start with?

Or it should be allowed to configure OS for every job somehow?

Probably simplest to configure it on a per-node basis for now. So if you start a node with S3 configured, then all streams will use S3. If you restart without S3, then S3 won't be used.

Save mode

Yes! We absolutely need a better story for post-transcode content management. We'll definitely need to think about where/how OS fits into that picture. Also related, #559

Now broadcaster keeps only three segments in memory - should OS also delete old segments? Or it should be configurable?

For now, OS should not delete segments. Let's keep it simple and leave object management to the user. Along with our recent ETH Berlin experience, we might also want to start retaining segments on-disk locally, but that's a separate issue, #559

If we are serving source segments from broadcaster and asking transcoder to put segments into our S3 - should we also put source segments in our S3?

Yes, let's do that for consistency.

For some OS we just need to put urls to manifest, but for some we should download segments and put somewhere else

If we check that the segment host is NOT our S3 bucket, then we have to download it. Does that work?

we are asking transcoder to put segments into own S3 bucket, but then we should download it from there, and possibly put into own S3 to be served for CDN

Maybe we should break this down a little.

1. Transcoder has the option to use its own S3 to post transcoded results. Broadcaster doesn't really care about this; the broadcaster has to download it anyway, whether the segments are coming from the transcoder itself, or transcoder's S3.
2. Broadcaster has the option to use its own S3 to upload source segments. The transcoder can receive either a URL containing a segment to download, or a direct upload from the broadcaster.
3. Transcoder also has the option of making its S3 available for broadcasters to upload the source.
4. Broadcaster also has the option of making its S3 available for transcoders to upload results.

To make it easier, we can start with just 1 and 2, then add 3 and 4 later. I don't think the case of "downloading from T's S3 to upload into B's S3" is important to handle right now, since we'll have support for uploading directly into the other's OS.

Clients should try to respect the other's preferences. So if B provides S3 credentials for T to use, then T should use those credentials, and vice versa.

We'd need to figure out what happens if B/T have incompatible OS providers, say a transcoder is up-to-date and has an Azure OS, but an older broadcaster doesn't understand how to use those credentials. But that can come later.

What do you think about making alternates available? If both B and T supply their own external OS, then T can post the results back in both OSes, and send back both URLs, along with a URL to its own local HTTP if that's configured. Could be useful for fallback / redundancy purposes, but I suspect we would need to build a bunch of support around it, so maybe later.

[j0sh]

OS merged in #562