On devices with many media devices, some which are in use, loading media device fails.

[Tenkir]

Describe the bug

Some of our production machines have many media devices (10+ video sources), in other WebRTC based systems, we are able to connect these devices to WebRtc. On LiveKit, a DeviceInUse error is thrown.

In other WebRTC systems, if a media device fails, it will iterate through the list until it finds an available device. LiveKit however just fails and forces the user to manually select a working device.

Reproduction

1. On a device with many media sources
2. Have an application control the input of the first media device
3. Connect to LiveKit room
4. Observe connection to the camera fails

Reproduction:
https://codesandbox.io/s/livekit-example-8itvr1?file=/src/index.ts

Logs

ℹ️ [{"type":"Button:on-click","meta":{"about":"#i515ffa12f9ca"}}]
converter-funcs.ts:267 🕸🚰
converter-funcs.ts:267 💥RtcRoomContainer {about: 'a7c06d12-845c-47ad-87ea-57696ce1d662'}
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:connect","meta":{"about":"a7c06d12-845c-47ad-87ea-57696ce1d662"}}]
Room.ts:1224 room event connectionStateChanged {event: 'connectionStateChanged', args: Array(1)}
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:on-state-changed","meta":{"about":"#i533715ee1543"}}]
SignalClient.ts:196 connecting to ws://localhost:7880/rtc?access_token={{token_removed_for_security}}&auto_subscribe=1&sdk=js&version=1.5.0&protocol=8
Room.ts:280 connected to Livekit Server version: 0.15.5, region: 
Room.ts:1224 room event signalConnected {event: 'signalConnected', args: Array(0)}
RTCEngine.ts:393 received server offer {RTCSdpType: 'offer', signalingState: 'stable'}
PCTransport.ts:187 setting munged local description
SignalClient.ts:327 sending answer
RTCEngine.ts:329 primary PC state changed connecting
RTCEngine.ts:329 primary PC state changed connected
LocalParticipant.ts:877 updating track subscription permissions {allParticipantsAllowed: true, participantTrackPermissions: Array(0)}
Room.ts:1224 room event connectionStateChanged {event: 'connectionStateChanged', args: Array(1)}
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:on-state-changed","meta":{"about":"#i533715ee1543"}}]
Room.ts:1224 room event connected {event: 'connected', args: Array(0)}
RTCEngine.ts:482 on data channel 1, _reliable
RTCEngine.ts:482 on data channel 3, _lossy
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:on-room-connection","meta":{"remoteParticipants":[],"about":"#i533715ee1543"}}]
converter-funcs.ts:267 🕸RtcRoomContainer
converter-funcs.ts:267 💥RtcRoomContainer {about: '*'}
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:request-device-permission","meta":{"about":"*"}}]
RtcRoomContainerComponent.tsx:161 Requesting devices
ShowInstructionsContainer.tsx:202 ℹ️ []
RtcRoomContainerComponent.tsx:214 DOMException: Could not start video source
RtcRoomContainerComponent.tsx:215 DeviceInUse
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:on-device-permission-denied","meta":{"deviceError":"DeviceInUse","about":"#i533715ee1543"}}]
Room.ts:1224 room event connectionQualityChanged {event: 'connectionQualityChanged', args: Array(2)}
ShowInstructionsContainer.tsx:202 ℹ️ [{"type":"RtcRoomContainer:on-connection-quality-changed","meta":{"quality":"excellent","participantSid":"PA_LjUS2WWmS2g7","about":"#i533715ee1543"}}]

System Info

Browser:
  Firefox 107.0
  Chrome 107.0.5304.108

System:
  OS: Linux 5.4 Debian GNU/Linux 10 (buster) 10 (buster)
  CPU: (48) x64 AMD Ryzen Threadripper 3960X 24-Core Processor
  Memory: 37.20 GB / 50.10 GB
  Container: Yes
  Shell: 5.7.1 - /bin/zsh
Binaries:
  Node: 16.17.1 - ~/.volta/tools/image/node/16.17.1/bin/node
  Yarn: 3.2.4 - ~/.volta/tools/image/yarn/1.22.11/bin/yarn
  npm: 8.19.2 - ~/.volta/tools/image/npm/8.19.2/bin/npm

Severity

annoyance

Additional Information

Not sure if this qualifies as a "bug" per-say, as it is technically working, but not in a way that would create an ideal user experience. Potentially adding an option to allow the developer to iterate through available devices, like how Google Meet works would be an enhancement?

[lukasIO]

Thanks for the report!

I guess you are not specifying any particular device, but rather the problem is around the default device when you call setCameraEnabled?

[Tenkir]

Thanks for the report!

I guess you are not specifying any particular device, but rather the problem is around the default device when you call setCameraEnabled?

Correct, this is when using setCameraEnabled or enableCameraAndMicrophone.

[lukasIO]

When requesting a device via getUserMedia (which is what our APIs do) browsers should - theoretically - return a device that is available, as long as no specific device is requested.
looking at the logs you provided, I'm not certain where the DOM exception is coming from. Is there an exception within the livekit-client-sdk ? And is it one of these exceptions? https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia#exceptions

[Tenkir]

The error we get is

DOMException: Could not start video source
code: 0
message: "Could not start video source"
name: "NotReadableError"

Passing that error object into MediaDeviceFailure returns DeviceInUse.