Support visionOS Front/Persona Camera

[gching]

Ref: https://livekit-users.slack.com/archives/C07FVFGAUKX/p1723112131622469

Changes

• Still not working as expected
• visionOS does not support DiscoverySession or .default to find the front / persona camera device, but supports .systemPreferredCamera which returns only one device which is the front / persona camera
• Added some logic to return the device in DeviceManager
• Using https://github.com/livekit-examples/swift-example, querying for the device works as expected, but starting the video stream timeouts on waiting for the dimensions to resolve

  client-sdk-swift/Sources/LiveKit/Participant/LocalParticipant.swift

  Line 481 in eaf1e24

  log("[Publish] Waiting for dimensions to resolve...")

Miscellaneous

• Xcode 16 beta 4
• visionOS 2 beta 5

[CLAassistant]

All committers have signed the CLA.

[hiroshihorie]

Thanks for looking into this, do you see :
"starting camera capturer device" in the logs ?

[gching]

Yep! Seeing the following:

2024-08-09T03:12:34-0700 info LiveKitSDK : [LiveKit] CameraCapturer.startCapture() starting camera capturer device: <AVCaptureFigVideoDevice: 0x109034e00 [Front Camera][com.apple.avfoundation.avcapturedevice.built-in_video:1]>, format: (format: 'vide'/'420v' 1920x1080, { 1- 30 fps}, photo dims:{1920x1080,2112x1188}, supports vis (max strength:Low), max zoom:67.50 (upscales @1.00), supports multicam, dimensions: Dimensions(1920x1080)), fps: 30(0...30)
2024-08-09T03:12:34-0700 debug LiveKitSDK : [LiveKit] LocalParticipant._publish(track:options:) [Publish] Waiting for dimensions to resolve...
2024-08-09T03:12:34-0700 debug LiveKitSDK : [LiveKit] AsyncCompleter<Dimensions>.wait(timeout:) Dimensions waiting milliseconds(20000) with id: 95F1FA24-20EE-47F6-9FF7-5F622B5F4844
cannot add handler to 0 from 1 - dropping
2024-08-09T03:12:54-0700 debug LiveKitSDK : [LiveKit] AsyncCompleter<Dimensions>.wait(timeout:) Wait 95F1FA24-20EE-47F6-9FF7-5F622B5F4844 timedOut
2024-08-09T03:12:54-0700 error LiveKitSDK : [LiveKit] LocalParticipant._publish(track:options:) [publish] failed LocalVideoTrack(sid: nil, name: camera, source: Source(rawValue: 1)), error: Error Domain=io.livekit.swift-sdk Code=101 "Timed out" UserInfo={NSLocalizedDescription=Timed out}
2024-08-09T03:12:54-0700 debug LiveKitSDK : [LiveKit] CameraCapturer.init(delegate:) startStopCounter 1 -> 0
2024-08-09T03:12:54-0700 debug LiveKitSDK : [LiveKit] AsyncTimer.deinit 

Note - I changed the timeout to be 20000 milliseconds just to see if it changed anything (it didn't), so don't worry about that in the above logs!

[hiroshihorie]

Thanks for your hints, I have another branch working on this. Will push in a few hours.

[bcherry]

@hiroshihorie I actually have the same exact problem in main on my iPhone 14 Pro so it's not visionOS-specific

[hiroshihorie]

I have it working on this branch
There was a mistake in for visionOS in WebRTC level so I patched it.

[gching]

Amazing Hiroshi! Tested #460 on my device and persona is working! Thank you for getting it working and patching WebRTC as well 🚀

Closing this PR given #460