Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Audio is not rendered from one peer in PTT mode #412

Closed
dbkr opened this issue Jun 17, 2022 · 2 comments
Closed

Audio is not rendered from one peer in PTT mode #412

dbkr opened this issue Jun 17, 2022 · 2 comments
Labels
A-Walkie-Talkie Walkie-talkie / PTT (push-to-talk) mode O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Something isn't working: bugs, crashes, hangs, vulnerabilities, or other reported problems Z-Media-Failure Someone's audio or video isn't coming through Z-Platform-Specific

Comments

@dbkr
Copy link
Member

dbkr commented Jun 17, 2022 •
edited
Loading

Describe the bug
This is a bug to track the failures we've been seeing in PTT mode where audio packets are received by the browser but it doesn't play them and we don't see the PTT rings animating.

To Reproduce
Steps to reproduce the behavior:

  1. Enter PTT call on browser A
  2. Enter the same call on browser B
  3. Click the 'back' button to exit the call on browser B, then click on the same call & rejoin
  4. Press PTT button on browser A, check if audio is heard on browser B
  5. Repeat above two steps until you hear no audio and no rings animate

'browser A' was initially an Android device running element call in a webview. It does appear to be slightly easier to reproduce the bug with this setup. Some of us have been unable to repro with two desktop browsers.

We are assuming that there are other ways that this bug can be triggered, but this seems to repro the bug where closing browser B & re-visiting the URL for the call does not.

Expected behavior
Audio should be heard from browser B when pressing PTT button on browser A
@dbkr
Copy link
Member Author

dbkr commented Jun 17, 2022

What we know so far:

  • From webrtc-internals, we can see that browser B is receiving audio data, and this data appears to be non-silence judging by the way the amount of data received increases on audio activity.
  • However, there are no audio samples received in the inbound audio stream
  • RTPInboundAudioStream shows almost all packets are being discarded (packetsDiscarded ~= packetsReceived)

@dbkr
Copy link
Member Author

dbkr commented Jun 17, 2022

It looks like the reason the packets aren't making it anywhere is because the jitter buffer is somehow getting confused and throwing them away. The number of jitter buffer flushes keeps increasing, so the jitter buffer is filling up but never emitting anything.

Screenshot 2022-06-17 at 17 35 26

It looks like somehow the jitter buffer is getting into a mode where it's frequently flushing but still never actually getting into a mode where it can emit packets.

Comparison of the outbound audio stream on the sender & inbound on the receiver:

Screenshot 2022-06-17 at 17 39 50

So the timestamps correspond, although unsure exactly what [lastPacketReceivedTimestamp] is, which appears to be 18 seconds behind.

At this point, we probably need to start looking into WebRTC's jitter buffer code, and/or just workaround it in the app by, eg. restarting the call if we see packets arriving but no media. 😢

@fkwp fkwp added this to the Spike Embedded PTT in Element Android milestone Jun 28, 2022
@robintown robintown added A-Walkie-Talkie Walkie-talkie / PTT (push-to-talk) mode S-Major Severely degrades major functionality or product features, with no satisfactory workaround O-Occasional Affects or can be seen by some users regularly or most users rarely Z-Media-Failure Someone's audio or video isn't coming through T-Defect Something isn't working: bugs, crashes, hangs, vulnerabilities, or other reported problems Z-Platform-Specific labels Jun 30, 2022
@robintown robintown mentioned this issue Jul 1, 2022
Merged
@robintown robintown mentioned this issue Jul 1, 2022
Merged
su-ex added a commit to SchildiChat/matrix-js-sdk that referenced this issue Nov 25, 2022
@su-ex
Merge tag 'v21.2.0' into sc
6128de1 
* Make calls go back to 'connecting' state when media lost ([\matrix-org#2880](matrix-org#2880)).
* Add ability to send unthreaded receipt ([\matrix-org#2878](matrix-org#2878)).
* Add way to abort search requests ([\matrix-org#2877](matrix-org#2877)).
* sliding sync: add custom room subscriptions support ([\matrix-org#2834](matrix-org#2834)).
* webrtc: add advanced audio settings ([\matrix-org#2434](matrix-org#2434)). Contributed by @MrAnno.
* Add support for group calls using MSC3401 ([\matrix-org#2553](matrix-org#2553)).
* Make the js-sdk conform to tsc --strict ([\matrix-org#2835](matrix-org#2835)). Fixes matrix-org#2112 matrix-org#2116 and matrix-org#2124.
* Let leave requests outlive the window ([\matrix-org#2815](matrix-org#2815)). Fixes element-hq/element-call#639.
* Add event and message capabilities to RoomWidgetClient ([\matrix-org#2797](matrix-org#2797)).
* Misc fixes for group call widgets ([\matrix-org#2657](matrix-org#2657)).
* Support nested Matrix clients via the widget API ([\matrix-org#2473](matrix-org#2473)).
* Set max average bitrate on PTT calls ([\matrix-org#2499](matrix-org#2499)). Fixes element-hq/element-call#440.
* Add config option for e2e group call signalling ([\matrix-org#2492](matrix-org#2492)).
* Enable DTX on audio tracks in calls ([\matrix-org#2482](matrix-org#2482)).
* Don't ignore call member events with a distant future expiration date ([\matrix-org#2466](matrix-org#2466)).
* Expire call member state events after 1 hour ([\matrix-org#2446](matrix-org#2446)).
* Emit unknown device errors for group call participants without e2e ([\matrix-org#2447](matrix-org#2447)).
* Mute disconnected peers in PTT mode ([\matrix-org#2421](matrix-org#2421)).
* Add support for sending encrypted to-device events with OLM ([\matrix-org#2322](matrix-org#2322)). Contributed by @robertlong.
* Support for PTT group call mode ([\matrix-org#2338](matrix-org#2338)).
* Fix registration add phone number not working ([\matrix-org#2876](matrix-org#2876)). Contributed by @bagvand.
* Use an underride rule for Element Call notifications ([\matrix-org#2873](matrix-org#2873)). Fixes element-hq/element-web#23691.
* Fixes unwanted highlight notifications with encrypted threads ([\matrix-org#2862](matrix-org#2862)).
* Extra insurance that we don't mix events in the wrong timelines - v2 ([\matrix-org#2856](matrix-org#2856)). Contributed by @MadLittleMods.
* Hide pending events in thread timelines ([\matrix-org#2843](matrix-org#2843)). Fixes element-hq/element-web#23684.
* Fix pagination token tracking for mixed room timelines ([\matrix-org#2855](matrix-org#2855)). Fixes element-hq/element-web#23695.
* Extra insurance that we don't mix events in the wrong timelines ([\matrix-org#2848](matrix-org#2848)). Contributed by @MadLittleMods.
* Do not freeze state in `initialiseState()` ([\matrix-org#2846](matrix-org#2846)).
* Don't remove our own member for a split second when entering a call ([\matrix-org#2844](matrix-org#2844)).
* Resolve races between `initLocalCallFeed` and `leave` ([\matrix-org#2826](matrix-org#2826)).
* Add throwOnFail to groupCall.setScreensharingEnabled ([\matrix-org#2787](matrix-org#2787)).
* Fix connectivity regressions ([\matrix-org#2780](matrix-org#2780)).
* Fix screenshare failing after several attempts ([\matrix-org#2771](matrix-org#2771)). Fixes element-hq/element-call#625.
* Don't block muting/unmuting on network requests ([\matrix-org#2754](matrix-org#2754)). Fixes element-hq/element-call#592.
* Fix ICE restarts ([\matrix-org#2702](matrix-org#2702)).
* Target widget actions at a specific room ([\matrix-org#2670](matrix-org#2670)).
* Add tests for ice candidate sending ([\matrix-org#2674](matrix-org#2674)).
* Prevent exception when muting ([\matrix-org#2667](matrix-org#2667)). Fixes element-hq/element-call#578.
* Fix race in creating calls ([\matrix-org#2662](matrix-org#2662)).
* Add client.waitUntilRoomReadyForGroupCalls() ([\matrix-org#2641](matrix-org#2641)).
* Wait for client to start syncing before making group calls ([\matrix-org#2632](matrix-org#2632)). Fixes matrix-org#2589.
* Add GroupCallEventHandlerEvent.Room ([\matrix-org#2631](matrix-org#2631)).
* Add missing events from reemitter to GroupCall ([\matrix-org#2527](matrix-org#2527)). Contributed by @toger5.
* Prevent double mute status changed events ([\matrix-org#2502](matrix-org#2502)).
* Don't mute the remote side immediately in PTT calls ([\matrix-org#2487](matrix-org#2487)). Fixes element-hq/element-call#425.
* Fix some MatrixCall leaks and use a shared AudioContext ([\matrix-org#2484](matrix-org#2484)). Fixes element-hq/element-call#412.
* Don't block muting on determining whether the device exists ([\matrix-org#2461](matrix-org#2461)).
* Only clone streams on Safari ([\matrix-org#2450](matrix-org#2450)). Fixes element-hq/element-call#267.
* Set PTT mode on call correctly ([\matrix-org#2445](matrix-org#2445)). Fixes element-hq/element-call#382.
* Wait for mute event to send in PTT mode ([\matrix-org#2401](matrix-org#2401)).
* Handle other members having no e2e keys ([\matrix-org#2383](matrix-org#2383)). Fixes element-hq/element-call#338.
* Fix races when muting/unmuting ([\matrix-org#2370](matrix-org#2370)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-Walkie-Talkie Walkie-talkie / PTT (push-to-talk) mode O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Something isn't working: bugs, crashes, hangs, vulnerabilities, or other reported problems Z-Media-Failure Someone's audio or video isn't coming through Z-Platform-Specific
Projects
No open projects
Element Call issue triage
Status: Closed
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dbkr @fkwp @robintown