Specify the transition between the local and remote playback when changing remote.state

[avayvod]

A spin-off from the issue #3:

• when the state is "disconnected" or "connecting", all interaction with HTMLMediaElement is forwarded to/from the local player
• when the state is "connected", all interaction with HTMLMediaElement is forwarded to/from the remote player
• when the state changes from "connecting" to "connected", the last known state of the local player is passed to the remote player
• when the state changes from "connected" to "disconnected", last known state of the remote player is passed to the local player

@foolip are there any methods we'd need to disable in "connecting" state? (for example, pause() is allowed but play() could be rejected)?

[jernoble]

Perhaps I'm misunderstanding, but ideally wouldn't the local- and remote-player states always be synchronized? Wouldn't then the last rule be unnecessary?

[avayvod]

Yes, as long as we're talking about the media element and its state exposed to the webpage. I believe I meant the backend for the local playback under "local player" which could just be suspended until the remote playback is stopped and then synchronized with the remote playback state to continue from there.

[foolip]

It would be nice if calls to play() or pause() while connecting still take effect remotely after connecting, or locally if the setup fails. Otherwise you'll likely have in-page UI with play/pause buttons that don't know about this odd in-between state and it just seems flaky to the user.

A tricky one is seeking, since that's defined in terms of a bunch of state that might be different on the remote side. I suppose seeking would have to just remember the requested time and base nothing on the local state until the connection is complete.

[mounirlamouri]

@foolip, I think that the idea here is that while connecting, all commands go to the local playback. That way, the developer can keep the playback running or can stop it while the connection is being made.

[foolip]

Hmm, so what seeking, which it can begin before the connection is complete, but doesn't finish locally until after?

[mounirlamouri]

I guess the local player state would change. Whether the network request related to the seek should be cancelled should probably be an implementation detail. When switching to connected, the remote playback should be in the same state as the local playback. In this case, it will also use the new seek position. Does that make sense?

[foolip]

It would be hard to get it into the same state, because it's locally showing a frame for time x but trying to seek to time y. While looking into seeking I filed whatwg/html#1362 but if that were fixed, I think that one could simply try to seek the remote to the official playback position after connection.

This is a basic question, but what happens to the local side when the connection is complete? If its internal media player is destroyed (perhaps to be recreated later) then all is well, but if it lives on then it might end up somewhere other than at the official playback position, but I guess restoring the remote state to local when disconnecting should take care of that?

[avayvod]

I guess in practice we'll bind the local state and remote states in this case to the external APIs/controls/UI, not the internal/remote media player - when one seeks, the UI/controls usually move to the target position so when the local state will be transferred to the remote playback device once the UA is connected to it, the seek for the target position will be sent to the remote playback device and the controls will reflect the remote player state immediately afterwards. Not sure if seek/fastSeek changes the page visible attributes of the media element to the desired position immediately or if there're some events that need to happen before that. In the latter case, the UA will have to remember to send these events and update the position once the remote player updates it.

[foolip]

Not sure if seek/fastSeek changes the page visible attributes of the media element to the desired position immediately or if there're some events that need to happen before that.

Setting currentTime also sets the "official playback position" which will cause currentTime to immediately return that same value. There's also the seeking attribute that becomes true immediately.

I don't thing any of this will be very hard to sort out, but it does mean that one may have to decide whether to seek the remote to the requested position or the actual position in case a seek finishes during the connecting phase.

[avayvod]

I think this is fixed by #49 although we could be more explicit about what's included in the state, perhaps. Let's file a new issue if we feel so.