Bug 211483

Summary: WebRTC replaceTrack adding latency
Product: WebKit Reporter: Marc <marc>
Component: WebRTCAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: daginge, eric.carlson, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Safari 13   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
iOS replaceTrack Delay Example none

Marc
Reported 2020-05-05 19:46:17 PDT
I'm using the following code to switch cameras on iOS. It works, however it seems to add around 1s of latency to the video. Each time I switch cameras it appears to grow. I have managed to replicate it in Safari Technology Preview. navigator.mediaDevices .getUserMedia({ video: { deviceId: { exact: selectedCamera } } }) .then(function(stream) { let videoTrack = stream.getVideoTracks()[0]; var sender = pc.getSenders().find(function(s) { return s.track.kind == videoTrack.kind; }); sender.replaceTrack(videoTrack); }) .catch(function(err) { console.error('Error happens:', err); });
Attachments
iOS replaceTrack Delay Example (12.66 MB, video/quicktime)
2020-05-07 17:14 PDT, Marc 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2020-05-06 03:47:19 PDT
<rdar://problem/62924912>
youenn fablet
Comment 2 2020-05-07 02:54:58 PDT
Hi Marc, just tried your code in Web Inspector on https://webrtc.github.io/samples/src/content/peerconnection/pc1/ and I do not see the added latency. Would you be able to provide a full example?
Marc
Comment 3 2020-05-07 17:13:20 PDT
Sure. I'm testing this on my iPhone XS. 1. Load https://webrtc.github.io/samples/src/content/peerconnection/pc1/ 2. Click Start (Allow) and then Call 3. Wave at the camera to show that its almost in sync 4. paste this code into the console: let cameraIds = []; navigator.mediaDevices.enumerateDevices().then(function(deviceInfos) { for (let i = 0; i !== deviceInfos.length; ++i) { const deviceInfo = deviceInfos[i]; if (deviceInfo.kind === 'videoinput') { cameraIds.push(deviceInfo.deviceId); } } }); let idCount = 1; function swapCamera() { navigator.mediaDevices .getUserMedia({ video: { deviceId: { exact: cameraIds[idCount] } } }) .then(function(stream) { console.log('gotstream'); localVideo.srcObject = stream; let videoTrack = stream.getVideoTracks()[0]; var sender = pc1.getSenders().find(function(s) { return s.track.kind == videoTrack.kind; }); sender.replaceTrack(videoTrack); }) .catch(function(err) { console.error('Error happens:', err); }); idCount++; if(idCount >= cameraIds.length) { idCount = 0; } } 5. call swapCamera() function 6. call swapCamera() function again to return to first camera 7. wave at camera to see the delay.
Marc
Comment 4 2020-05-07 17:14:44 PDT
Created attachment 398816 [details] iOS replaceTrack Delay Example Example video showing the latency. Notice the latency increases the more I switch cameras.
Dag-Inge Aas
Comment 5 2020-07-29 05:10:36 PDT
I'm unable to replicate this now on iPhone X running iOS 13.6. Could you retest Marc on iOS 13.6 and see if you still see the same thing?
Marc
Comment 6 2020-08-08 15:54:17 PDT
(In reply to daginge from comment #5) > I'm unable to replicate this now on iPhone X running iOS 13.6. Could you > retest Marc on iOS 13.6 and see if you still see the same thing? Thanks for looking into this. I think you are right. I can't seem to replicate it now.
Note You need to log in before you can comment on or make changes to this bug.