Live SegmentTimeline handling broken when switching audio tracks

[TobbeEdgeware]

The setLiveEdgeSeekTarget function in src/streaming/controllers/ScheduleController.js is not working properly for live SegmentTimeline when switching audio language. This is very visible when the timeshiftBufferDepth is small, since the original start of the playback is used as live point.

I've made a test asset at http://vm2.dashif.org/livesim-dev/segtimeline_1/tsbd_24/testpic_6s/multiaudio.mpd

which has 24s timeshiftBufferDepth and 6s segments. If the asset is started, and one waits >20s and then switches audio language the player stops.

Looking into the log there is an error "Cannot read property 'startTime' of null" which happens in the setLiveEdgeSeekTarget function since request == null.

This can be traced back to liveEdge having the same value when called to switch language as it had when joining the live edge the first time.

I don't know if setLiveEdgeSeekTarget() should be modified, or maybe not called when switching audio language, but something is of course needed to initialize the new adaptationSet.

[TobbeEdgeware]

Using the players at http://dashif.org/reference/players/javascript/ it was seen that the test asset plays properly in Version 2.3, but not in Version 2.4.

[brndn4]

Any idea if audio-track switching will be fixed in the August 25 release?

[TobbeEdgeware]

I haven't have time to look at this for quite some time, and unfortunately, the latest dash.js still throws an exception in setLiveEdgeSeekTarget when switching language for the test asset http://vm2.dashif.org/livesim-dev/segtimeline_1/tsbd_24/testpic_6s/multiaudio.mpd.

The error is "TypeError: Cannot read property 'startTime' of null because request === null.

When I looked at this some time ago, I noticed that part of the strange behaviour was that the player switched back to the original language after just fetching the init segment and one segment of the new language. This was after getting around the exception mentioned above.

[brndn4]

@TobbeEdgeware Thanks a lot for the update. I am going to trying to look into this issue more. I am not very familiar with the codebase, so if you have any other info that might help, let me know. Thanks!

[brndn4]

@TobbeEdgeware For what it's worth, the first version of the code that I can reproduce this issue with is after this commit on 10/26/16: 2bd53f9

I was not able to reproduce the issue with the code before that commit.

[TobbeEdgeware]

@brndn4 It is true that the player doesn't crash before that commit, but the the actual language switch does not work, so there is an earlier severe error to the switching mechanism for segmenttimeline.

When adding logging of fetched segments, I see the following fetched audio urls when printing request.url at FragmentLoader.js line 104 (inside the success callback):

.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171420768256.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171421057024.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48_deu/init.mp4
.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171421345792.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48_deu/t72171420770304.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48_deu/t72171420192768.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48_deu/t72171421059072.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171421059072.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171421346816.m4s
.../segtimeline_1/tsbd_24/testpic_6s/A48/t72171421059072.m4s

This shows that the init segment and a few segments of the A48_deu representations are fetched, but than the player continues to fetch the old representation A48. Furthermore, it asks for the segment A48/t72171421059072.m4s two times. I have seen the player ask for the same segment many more times than that, something that doesn't happen for SegmentNumber template fetching.

My belief is that there is something fundamentally wrong in the event loop that schedules fetching of segments which makes it loose state and also trigger multiple parallel requests. @AkamaiDASH also noticed that this has a long history of not working properly.

[nicosang]

Hi @TobbeEdgeware,

I think I have found why sometimes, when a track audio switch occurs, the old representation is downloaded. In your sample the two audio adaptations have a 'main' role. It sould be ok according to the standard.

the problem is in the getAdaptationForType function which is called from updateData function of the stream when an update of the manifest has occured.

` function getAdaptationForType(manifest, periodIndex, type, streamInfo) {

    let adaptations = getAdaptationsForType(manifest, periodIndex, type);

    if (!adaptations || adaptations.length === 0) return null;

    if (adaptations.length > 1 && streamInfo) {
        let currentTrack = mediaController.getCurrentTrackFor(type, streamInfo);
        let allMediaInfoForType = adapter.getAllMediaInfoForType(streamInfo, type);
        for (let i = 0, ln = adaptations.length; i < ln; i++) {
            if (currentTrack && mediaController.isTracksEqual(currentTrack, allMediaInfoForType[i])) {
                return adaptations[i];
            }
            if (getIsMain(adaptations[i])) {
                return adaptations[i];
            }
        }
    }
    return adaptations[0];
}`

before the switch audio track A48_deu was selected. After the manifest updated, this previous function will return A48 instead of A48_deu because A48 has a 'main' role.

If you do agree with this point, I can provide a pull request. Unfortunately, it will not resolve the main issue... :-(

[brndn4]

@TobbeEdgeware Can you share how you got past the "TypeError: Cannot read property 'startTime' of null because request === null" exception? I want to see how that change along with @nicosang's suggestion might work.

[TobbeEdgeware]

@brndn4 I just made a simple check if request === null and returned early in that case:

        if (request === null) {
            return
        }
        if (isNaN(seekTarget) || request.startTime > seekTarget) {

With @nicosang's fix #2144 it then works to switch once to a new language. :-) The new language is fetched from where the old segment ended (+6s).

However, when going back, things are messy since earlier times are fetched.

So, it seems that me must transmit the fetching time from one language to the other.

[brndn4]

FWIW, I was able to get audio track switching working with these 2 changes above and by refreshing the manifest after a switch, which I found in this change: aeff4c9 (Manage audio track switch, by refreshing manifest).

I'm not sure if refreshing the manifest is more of a workaround than a fix, but I just wanted to mention it. Thanks!

[TobbeEdgeware]

PR #2151 seems to do the trick. Multiple switches back and forth work with that fix.

[nicosang]

Hi @TobbeEdgeware and @brndn4 ,

do you agree to close this issue?

Thanks,
Nico

[TobbeEdgeware]

Yes, I agree. Sorry for the slow response.