Change AVAudioSession defaults to iOS defaults

[hiroshihorie]

Currently, the default category is AVAudioSessionCategoryPlayAndRecord which will automatically turn-on the microphone even for listen/view-only cases.

Instead of forcing developer to use playAndRecord, developer should be responsible to set the relevant category, which will result mic to turn on if playAndRecord or record is used. (PR #5)

This PR sets the defaults to iOS defaults.

https://developer.apple.com/documentation/avfaudio/avaudiosession/capturing_stereo_audio_from_built-in_microphones

[davidzhao]

this looks good to me. question is how does this work for existing flutter-webrtc users? LiveKit iOS SDK already handles management of the audio session.

feels like flutter-webrtc should add this management layer before upgrading to this build. here's the logic that we use to init audio session when the first audio track is received. and here is when it's published

@cloudwebrtc, do you want to add this to flutter-webrtc first?

[hiroshihorie]

@davidzhao Thanks for pointing this out, this is my concern also. For flutter_webrtc, it would be ideal to have a new manageAudioSession option (which is default: true) to maintain compatibility, but new users could change this behavior (to false, then manage session on their own).

I'm still thinking but perhaps if manageAudioSession is true, and there is a local audio track added, category will be set to playAndRecord automatically, until all audio tracks are removed and category will be back to soloAmbient.

I might come up with a PR later, and would like to hear @cloudwebrtc's opinion about the design.

[cloudwebrtc]

hey, @davidzhao @hiroshihorie I think we need to find whether there is a localStream (including audio track) in the flutter-webrtc plugin to switch category, when there is no loalstream created, there is no need to use playAndRecord

These operations can be done internally by flutter-webrtc native, and users have no perception.

What do you think？

[davidzhao]

These operations can be done internally by flutter-webrtc native, and users have no perception.

Yes! I agree this could be done transparently by flutter-webrtc.

I think only two actions are needed:

• when user creates a local audio track, set audio session to playAndRecord
• when a onTrack event is triggered, and it's an audio track, set audio session to play (if it's currently ambient)

[hiroshihorie]

@cloudwebrtc @davidzhao Yes I agree it should be handled internally, but there should be an option for this to be off. So if the developer is managing the audio session themself it will not cause conflict.

[davidzhao]

if we only set audiosession if it's set to ambient, then it should be ok right?

if dev set it to something else, we would not override.

similarly with playAndRecord, we would only set it if it wasn't set

[hiroshihorie]

Should be ok most of the time but I'm still thinking if it's ok for the uncommon cases.

Some categoryOptions and modes don't work with different categories, and switching category transparently might reset these settings.

I think it still might be safe to have an option to turn off this auto-management of audio session, since it's hard to understand 100% of developer's intention.

soloAmbient does not support background playback so it is possible that developer would like to change this to playback https://developer.apple.com/documentation/avfaudio/avaudiosession/category/1616509-playback

Could be almost fully managed :
No LOCAL AUDIO: Set one of playback (dev may want soloAmbient, ambient for some reason)
Has LOCAL AUDIO track: Set record
Has LOCAL AUDIO track and REMOTE AUDIO track: SetplayAndRecord

Problem 1: How to know user's preferred playback mode (playback, soloAmbient, ambient)
Possible solution: Store the initial playback category and set it back to that.
Problem 2: How should we handle categoryOptions and modes ?

I'm not sure about multiRoute category.

Actually this is over-complicating the issue haha 😅

[davidzhao]

Some categoryOptions and modes don't work with different categories, and switching category transparently might reset these settings.

You are right that it's possible they might want ambient. I think it's ok for users to actually tell us those use cases.. and we can add a toggle then to disable this? Just trying to avoid over-engineering this for the initial version. So proposal is:

1. set to playback (with appropriate mode) if current category is ambient AND a remote audio track is received
2. set to playAndRecord (with appropriate mode) if current category is not playAndRecord AND a local track is created.

wdyt?

[hiroshihorie]

@davidzhao Sounds good! won't hurt to have an option to turn this off tho!

[davidzhao]

👍