Audio/video sync issues

[replicant1]

Hi Pedro,

I have a very general question for you. We have a user that is experiencing problems with audio and video being way out of sync when they are streaming with our app. Our app uses RootEncoder 2.4.5 and RtmpCamera1. The user has a Google Pixel 9 running Android 14. We are at a bit of a loss as to where to even start in diagnosing the problem.

1. Do you think there would be any point in us switching over to the RtmpCamera2 API to try and address this?
2. Do you think the architecture of the Pixel 9 could have anything to do with this eg. could it be a multi-threading problem in RootEncoder that the Pixel 9 is exposing?
3. Is it possible the sync issue has nothing to do with the app at all and is something occurring on the network?
4. Do you have any suggestions on how we could go about diagnosing this issue?

Any advice you could provide would be much appreciated!

[pedroSG94]

Hello,

This is really difficult to say.

1. Do you think there would be any point in us switching over to the RtmpCamera2 API to try and address this?

This is recommended in most of cases but currently pixel 9 seems to have a problem to detect cameraId. This was reported today in other issue so I recommend you wait until this is solved.

2. Do you think the architecture of the Pixel 9 could have anything to do with this eg. could it be a multi-threading problem in RootEncoder that the Pixel 9 is exposing?

Normally the sync out is related with the timestamp of the video/audio frames. This device could have a problem where the timestamp is not correctly calculated for some reason.

Is it possible the sync issue has nothing to do with the app at all and is something occurring on the network?

This could happens but normally the problem should be in the encoder (this library), the server or the player.

Do you have any suggestions on how we could go about diagnosing this issue?

One important thing is know if the out of sync is from the start of the stream or it is increasing with the stream time. With this we can focus in one of this cases.

Also, record a local file could help. Since it is a timestamp issue, the problem should happens in a local file and we can discard other external cases.

[replicant1]

Thanks for the advice Pedro. Much appreciated!

[pedroSG94]

This is recommended in most of cases but currently pixel 9 seems to have a problem to detect cameraId. This was reported today in other issue so I recommend you wait until this is solved.

Forget this.
This issue is not related with the pixel 9. I didn't understand the device model.

[alexmck]

Hi Pedro,

I have managed to get a recording from the user of the Pixel 9. The videos were over the 10 MB video file size limit so I have zipped them.

Attached are 3 videos:

1. A recording that was saved with RootEncoder. recorded-on-pixel-9-with-rootencoder.mp4.zip
2. The FLV file that was recorded by the server. recorded-on-server.flv.zip
3. A transcode that was done by Bunny from the FLV file that was recorded by the server. transcoded-from-record-on-server.mp4.zip

As can be seen, the first video file that was saved on the users Pixel 9 with RootEncoder looks to be ok when playing back. With that said, MediaInfo shows the video has mdhd_Duration : 65595 while the audio has mdhd_Duration : 65498.

The FLV files appears to be messed up. VLC on Mac doesn't like it very much. The audio plays but the video is frozen.

Perhaps there is something else that might stand out to you here?

[pedroSG94]

Hello,

This is not good news.

The library use exactly the same timestamp in audio and video streaming and recording so the video recorded in local and the stream result should be the same.

I will try find more info about this looking the FLV file. I will try compare timestamps in both files (local mp4 and FLV/transcodec mp4)
Can you tell me which class are you using? RtmpCamera1, RtmpStream, etc.

[alexmck]

We are using RtmpCamera1 currently.