Speaker View mode not working

[muchachagrande]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Steps to reproduce

1. Start a group call
2. Set Speaker View mode
3. Let different speaker speak
4. The first participant joining the call is the one that remains focused

Expected behaviour

The focused participant is supposed to change automatically to the speaker

Actual behaviour

The focus remains on the first participant that joined the call

Talk app

Talk app version: (see apps admin page: /index.php/settings/apps)
16.0.2

Custom Signaling server configured: yes/no and version (see additional admin settings: /index.php/index.php/settings/admin/talk#signaling_server)
Yes, 1.1.2

Custom TURN server configured: yes/no (see additional admin settings: /index.php/settings/admin/talk#turn_server)
Yes

Custom STUN server configured: yes/no (see additional admin settings: /index.php/settings/admin/talk#stun_server)
Yes

Browser

Microphone available: yes/no
Yes

Camera available: yes/no
Yes

Operating system: Windows/Ubuntu/...
Ubuntu 20.04

Browser name: Firefox/Chrome/...
Firefox

Browser version: 85/96/...
111.0.1

Browser log

Server configuration

Operating system: Ubuntu/RedHat/...
Ubuntu 22.04

Web server: Apache/Nginx
Apache2

Database: MySQL/Maria/SQLite/PostgreSQL
Postgre

PHP version: 8.0/8.1/8.2
8.1

Nextcloud Version: (see admin page)
26.0.0

List of activated apps:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your server installation folder

Nextcloud configuration:

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your Nextcloud installation folder

Server log (data/nextcloud.log)

Insert your server log here

[nickvergessen]

Sometimes the mic of users is producing static noise and therefore their client always sends the "I'm speaking" signal.
Another thing is if you clicked on the video of that user they are "followed" and you will always stick with them. But in that case you should see a button at the bottom of the video:

[muchachagrande]

Hi @nickvergessen. For the tests I have made, I can confirm that I have not followed any user.
What I did to avoid the static noise of microphones is mute all users except the one that has to speak.
I tested both ways, with all micros muted except the speaker and with all micros unmuted.

Edit: I realize that the speaker was not being promoted when I was testing the recording server and all recorded videos were showing a single user.

[muchachagrande]

@nickvergessen. Is it possible to debug the reception of the "I'm speaking signal" to the desktop client.
I'm using one desktop client and up to four Android clients to conduit the tests.

[muchachagrande]

Some debugging I made today. I don't know if I'm doing right.
I observed the browser console during a call.
I couldn't see any "I'm speaking signal" message nor anything alike.
I saw many messages like this:

Unknown message from 1 datachannel
Object { payload: {…}, type: "nickChanged" }
​>
payload: Object { name: "Casa", userid: "casa" }
​>
type: "nickChanged"
​>
: Object { … }
nd5Pc5rZjZSnGL76WCgbwnGjM7mJ09jHPNT02mopp3N8PT1nSTh3YTc5Tm1tS0I5wTlaZW16RVpYLWR1bVRKTU1CVi1kZllQazhJQThpamc3NGtrUGdEaWpxWlJLY3pBeThwTHBRVERHRlRBdXwDeWxkaVF0S0ozWnRRekFXwm1GbVk4dzcySENCLV91VmZRRC1oOTQwZFhrSDV4wFlEYzAtUF94dTlTVGJLemJhdjBPck1qdzNreEYxbWNEUVh2w0hyRkVCVjRqV2NqS1RDYVB8NzA4MDA4MDg2MQ==
Object { id: Getter & Setter, parent: Getter & Setter, type: Getter & Setter, oneway: Getter & Setter, sharemyscreen: Getter & Setter, stream: Getter & Setter, receiverOnly: Getter & Setter, sendVideoIfAvailable: Getter & Setter, enableDataChannels: Getter & Setter, enableSimulcast: Getter & Setter, … }
webrtc.js:1611:3

This message repeated many times during the call.

The browser is Firefox 111.0.1 and the other client is Talk Client 15.0.1 for Android.
The same happened with another android client version 16.0.0. Also in a group call with the browser and both android clients.
Then I tested with a remote Firefox 111.0.1 under Windows 7 set with the built-in fake webrtc client.
With the browser client on the other side, the "nickChanged" message is not constantly arriving. It arrives only when I change the nick name on the browser. So, may be there is a bug in the Android client.
Another thing I noticed is that android client shows every participant as it were always muted. Another bug?

[SystemKeeper]

Not sure talk-android supports this: nextcloud/talk-android#88
Can you test desktop to desktop?

[muchachagrande]

Then I tested with a remote Firefox 111.0.1 under Windows 7 set with the built-in fake webrtc client.
With the browser client on the other side, the "nickChanged" message is not constantly arriving. It arrives only when I change the nick name on the browser. So, may be there is a bug in the Android client.

Yes, I tested from Windows 7 - Firefox 111 to Ubuntu 20.04 - Firefox 111.
Didn't see any "I'm speaking" message.

Tomorrow I'll make a call between 3 or 4 participants.

[muchachagrande]

Today I made a call with two more participants. We were 3 in the call.
In my test I could see the "speaking" and "stoppedSpeaking" messages coming from the other participants.
The first problem I could find is that the Speaker View mode didn't focus on the speaker but on the first that joined the call. The other one remained in a small frame below.
The second problem, may be related to the first, is that the recorded video always showed me and didn't switched to the speakers as they talked. The sound is fine, all the speakers are listened, but only showing my video frame.
This behavior of the recording (fourth) participant is coherent with the behavior of the rest of the browsers that didn't switch the main frame to the speaker.

Is it possible that the browser client is not responding to the "speaking" and "stoppedSpeaking" ?

EDIT:
Forgot to mention. All participants are Firefox 111 on Ubuntu 20.04

[muchachagrande]

I have made another call with 3 participants. This time I used Brave browser v1.50.125. The other two used Firefox 112.0

In my tests I could see that when Talk is used with signaling server the speaker view doesn't show the speaker but the first participant entering the call.
When I configured Talk without signaling server, the speaker view worked fine.
I'm using Signaling server v1.1.2

Both browsers, Firefox and Brave showed the same results.

[muchachagrande]

Yesterday I found out that using Janus multistream versions (1.x) makes Talk to not recognize the control messages coming through the data channel because it is expecting the fixed label "JanusDataChannel" and the multistream version can use various data channels so it labels the channels with the corresponding stream number.
For this reason Talk was not recognizing "speaking", "stoppedSpeaking" and the other messages.
Downgrading Janus to the 0.x versions (no multichannel) solves this issue so I'm closing it.

[nickvergessen]

^ @fancycode I guess you are aware of this?

[fancycode]

Janus 1.x is pretty much untested from my side. There is an open issue at strukturag/nextcloud-spreed-signaling#280 about testing it, but previous reports were that it "is working as with 0.x versions", so I didn't give this much priority.

Probably not much I can do anyway as the DataChannels are between the client and Janus with no way to change anything from the signaling server side.

[muchachagrande]

The label of the data channel in 1.x Janus versions is the number of the channel, given that there can be more than one.
The name I saw in the browser debugger was just the number "1". The client is expecting the "status" or "JanusDataChannel" label, as can be seen in line 1591 of webrtc.js. At first try, one could accept the number "1" as the label to catch the first data channel of Janus 1.x versions. This could be repeated in every other place where the data channel label is evaluated, including the desktop and mobile clients.