Flickering subtitles in DASH/H264 with B-frames present

[AGasztold]

Media3 Version

ExoPlayer 2.18.3

Devices that reproduce the issue

All Android devices.

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

1. Run attached media in Demo app
2. Turn on subtitles
3. Observe the flicker effect

Expected result

No flickering effect should be observed.

Actual result

Due to the presence of B frames in text chunks and the time offset of text segment relative to video segments:

SegmentTimeline for text: <S t="19" d="2000" r="283" />
SegmentTimeline for video: <S t="0" d="2000" r="89" />

there is an effect of flickering subtitles. When I applied the following workaround inside SubtitleOutputBuffer:

  public void setContent(long timeUs, Subtitle subtitle, long subsampleOffsetUs) {
    this.timeUs = (timeUs/1000000L) * 1000000L;
    this.subtitle = subtitle;
    this.subsampleOffsetUs =
        subsampleOffsetUs == Format.OFFSET_SAMPLE_RELATIVE ? this.timeUs : subsampleOffsetUs;
  }

which cancels this offset, flickering effect does not occur. Anyaway, this is not long term solution for me. I will appreciate your help.

Media

https://r.dcs.redlabs.pl/dash/o2/ATM-Lab/zos/vod-test/glitch/dash_b_frames/playlist.smil

Bug Report

• You will email the zip file produced by adb bugreport to dev.exoplayer@gmail.com after filing this issue.

[icbaker]

Thanks for reporting, and for including a repro stream. I'm able to see the same issue.

I've added some debug logging and it looks like this happens when we transition between Subtitle objects inside TextRenderer while a cue is meant to be shown on screen. The implementation details of this means we currently end up 'finishing' one Subtitle object with an empty list of cues at the same presentation timestamp of the first event of the next Subtitle (which contains the same cue text that should be on screen). This transition through an empty cue list is then propagated through TextOutput and ultimately to the SubtitleView and results in the flickering. We end up seeing
cues emitted roughly as follows (the timestamps are from your example media, the text I swapped out for this example):

positionUs=1000037010972  presentationTimeUs=37000000  cues=["foo bar"]  subtitle_1
positionUs=1000038002287  presentationTimeUs=38000000  cues=[]  subtitle_1
positionUs=1000038022400  presentationTimeUs=38000000	cues=["foo bar"]  subtitle_2

I'm still investigating exactly how the media results in this output.

[AGasztold]

Hello, thanks for your reply. Are the any news regarding this issue? I will appreciate your help.

[07steven01]

I'm facing the same issue. Is there a fix yet?

[icbaker]

I'm currently making some fairly major changes to TextRenderer that should resolve the problem I described above - I hope this will be part of 1.2.0.

[07steven01]

I can see 1.2.0-alpha01 is out. No mention of TextRenderer in the changelog. Are the changes still planned for 1.2.0 or rather some further release?

[icbaker]

I'm working on the TextRenderer changes (they are not submitted yet) - at the moment i'm not sure if they will be complete in time for 1.2.0.

[icbaker]

I've added experimental support for the new approach to subtitle parsing in DASH. This will be included in 1.2.0-rc01. You can try it out with DashMediaSource.Factory.experimentalParseSubtitlesDuringExtraction.