Bug 239849

Summary: [SafariViewServices] WebRTC can’t access camera feed when returning from a native application
Product: WebKit Reporter: Dimitri Friedli <dimitri>
Component: WebRTCAssignee: Nobody <webkit-unassigned>
Status: RESOLVED MOVED    
Severity: Critical CC: bfulgham, milen.yordanov, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Safari 15   
Hardware: iPhone / iPad   
OS: iOS 16   
Attachments:
Description Flags
Screen recording of the steps needed to reproduce the issue. none

Description Dimitri Friedli 2022-04-28 06:30:08 PDT 
Created attachment 458524 [details]
Screen recording of the steps needed to reproduce the issue.

This issue might be related to https://bugs.webkit.org/show_bug.cgi?id=234509

# Environment 
URL: 			https://safari-15-webrtc-issues.netlify.app 
iOS Version: 		15.4.1
Device: 		iPhone 12 Pro
Safari Version: 	Safari 15.4
Safari Settings:	default
Reproduction Rate:      100%

The sample is based on the official WebRTC Demo (https://webrtc.github.io/samples/src/content/devices/input-output/). 
The only difference is that we added a manifest.json to make the demo “installable” without the Safari search bar.


# Steps to Reproduce the Issue
1. Go to https://safari-15-webrtc-issues.netlify.app 
2. Use Share > Add to Homescreen to install the app on the iPhone
3. Open the PWA via the added app icon
4. Stay a few seconds in the open web application
5. >>>> Scanner is working <<<<
6. Switch into the iOS Weather Application
7. Stay in the iOS Weather Application for ~10 seconds
8. Lock the Screen and wait ~10 seconds
9. Unlock the Screen and stay in the iOS Weather Application for ~10 seconds
10. Switch back to the installed WebRTC Test application
11. >>>> The camera feed of the scanner does not work <<<<

# Actual Behavior
The PWA does not get access to the camera feed. 
The user needs to perform a refresh, close, and re-open the application *OR* change the camera source (e.g., change from the front-facing camera to the camera on the back) to access the camera feed again. 

# Expected Behavior
The camera feed is immediately available when coming back to the installed PWA, OR the PWA does ask for camera permissions again. In either case, the user can continue using the application.
Comment 1 youenn fablet 2022-04-28 06:39:07 PDT 
It might be that the video is interrupted (see https://bugs.webkit.org/show_bug.cgi?id=239842 for instance).
Would calling video.play() fix the issue?

Would you be able to reproduce the issue and send me the corresponding sysdiagnose (youenn@apple.com), ideally with the timestamp of the issue?
Comment 2 Radar WebKit Bug Importer 2022-05-05 06:31:49 PDT 
<rdar://problem/92791775>
Comment 3 Dimitri Friedli 2022-05-05 07:55:28 PDT 
(In reply to youenn fablet from comment #1)
> It might be that the video is interrupted (see
> https://bugs.webkit.org/show_bug.cgi?id=239842 for instance).
> Would calling video.play() fix the issue?
> 
> Would you be able to reproduce the issue and send me the corresponding
> sysdiagnose (youenn@apple.com), ideally with the timestamp of the issue?

Hey, unfortunately, video.play() does not seem to resolve the issue. 
Re sysdiagnose: The browser console does not output any error – the installed PWA just can't access the camera feed anymore. It should be reproducible using the steps mentioned.
Comment 4 Dimitri Friedli 2022-09-16 05:47:00 PDT 
We did some testing with the released iOS 16 version, and the issue is still persistent.
Comment 5 milen 2023-03-29 06:39:08 PDT 
I tested it on iOS 16.4, the issue is still present.

This is what happens:

### Safari Mode
 
1) Open Safari.
1.1) Navigate to https://safari-15-webrtc-issues.netlify.app
1.2) Switch to another app
1.3) Switch back to Safari
1.4) The video track receives events 'muted' and 'unmuted'. The audio track receives no events.
1.5) The app continues to work as expected.


### Homescreen standalone app
 
2) Open Safari, navigate to https://safari-15-webrtc-issues.netlify.app, Add to Homescreen to install the app on the iPhone
2.1) Open the app from the homescreen
2.2) Switch to another app
2.3) Switch back to the app
2.4) Both video and audio tracks receive event 'ended'. track.readyState is also 'ended'. 
2.5) The app does not work anymore. The stream is not usable and GUM have to be called again. The user have to allow usage of the webcam and the mic, AGAIN.


Is this the expected behaviour of the apps installed to the homescreen?

To me it seems like a BUG. Just switching to another app should not KILL the audio and video streams.
Comment 6 youenn fablet 2023-04-06 15:15:16 PDT 
> To me it seems like a BUG. Just switching to another app should not KILL the
> audio and video streams.

Right, a fix is indeed needed outside of WebKit.
Comment 7 Brent Fulgham 2023-04-11 17:19:53 PDT 
The cause of this issue is outside of WebKit, so resolving as MOVED.

The relevant team is working on the issue under rdar://98308077.