Bug 241152

Summary: A black screen appears in a muted video element outside the viewport
Product: WebKit Reporter: Kyu Simm <simmkyu>
Component: MediaAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: devalevenkatesh, eric.carlson, jer.noble, simmkyu, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Other   
OS: Other   
Attachments:
Description Flags
WebKit-black-screen-repro-video none

Description Kyu Simm 2022-05-31 13:48:12 PDT 
Created attachment 459903 [details]
WebKit-black-screen-repro-video

Problem:
In macOS and iOS Safari, a video element shows a black screen if they are not visible on the page.

A video element should have the following properties set to cause the issue.
- muted = true
- srcObject = null

Suppose you set the "srcObject" to a new video stream when the video element is outside the browser's viewport.
The video element shows a black screen.


Steps to reproduce:
1. Use macOS or iOS Safari.
2. Open https://codepen.io/simmkyu/pen/abqqoNd in Safari. This demo is based on the official WebRTC PeerConnection sample that requires Wi-Fi connection. See Appendix A for more information.
3. Choose "Start" to enable the left video.
4. Choose "Call" to enable the right video using WebRTC peer connection.
5. Zoom into the "Call" and "Hang Up" sections. Ensure that the right video is not visible.
6. Choose "Hang Up" to turn off the right video.
7. Wait for 5 seconds. Do not scroll or touch the web page.
8. Choose "Call" again. Scroll down to confirm that the right video element shows a black screen.


Actual results:
- The right video element shows a black screen.


Expected results:
- The right video element should show the left video's content. Refresh the browser tab and run the "Steps to reproduce" section again while the portion of the right video element is visible. The right video element will show the left video's stream.


Workaround:
- When the video element shows a black screen, call the "HTMLMediaElement.play" API.


Tested environments:
- MacBook Pro 2019 (macOS 11.6.5, Safari 15.5)
- iPhone 13 Pro (iOS 15.5)
- iPhone X (iOS 15.4)
- iPad Pro (iOS 15.4.1)
- iPad Pro (iOS 14.7.1)


Additional information:
- The bug does not occur if the portion of a video element is visible.
- The bug does not occur in iOS Chrome.
- The bug does not occur if a video element succeeds in rendering the content more than once. (i.e., if you fail to reproduce the issue, refresh the browser tab and run the "Steps to reproduce" section again.)
- The black screen video element does not recover even after setting srcObject to another video stream or rotating iOS devices. You have to refresh the browser tab or call the "HTMLMediaElement.play" API.
- "HTMLMediaElement.paused" is "true". Safari automatically updates "paused" to "true" when a video element is hidden.


Appendix A: About the CodePen demo
- The demo is based on the WebRTC PeerConnection sample: https://webrtc.github.io/samples/src/content/peerconnection/pc1/
- The demo makes two changes to the WebRTC sample: 1) muted = true and 2) srcObject = null.
- The demo does not work on cellular network. Use Wi-Fi to reproduce the issue.
Comment 1 Radar WebKit Bug Importer 2022-06-07 13:49:36 PDT 
<rdar://problem/94562636>
Comment 2 youenn fablet 2022-06-16 03:15:15 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/1575
Comment 3 youenn fablet 2022-06-16 03:15:31 PDT 
Uploaded PR should fix this.
One workaround is to not set srcObject within a user gesture but use a setTimeout so that we do not remove the invisible autoplay restriction.
Or to actually set srcObject as part of user gesture if the video element is visible.
Comment 4 EWS 2022-06-16 07:38:31 PDT 
Committed r295591 (251596@main): <https://commits.webkit.org/251596@main>

Reviewed commits have been landed. Closing PR #1575 and removing active labels.
Comment 5 Venkatesh Devale 2022-08-03 16:06:26 PDT 
This is marked as fixed but the issue is still not resolved.

iOS 15.6 Safari
macOS 12.4 Safari 15.5

We had added a workaround for this which is now running into a AbortError. The workaround was:

```
// In Safari, a hidden video element can show a black screen.
// See https://bugs.webkit.org/show_bug.cgi?id=241152 for more information.
if (new DefaultBrowserBehavior().requiresVideoPlayWorkaround() && videoElement.paused) {
  videoElement.play();
}
```

The codepath runs on a user gesture which is a button click from user. The videoElement.play() is running into the AbortError.

Detailed Error:
[Error] Fatal error: this was going to be caught, but should not have been thrown. – AbortError: The operation was aborted.
AbortError: The operation was aborted.
    fatal (Prod:2:986193)
    fatal
    (anonymous function) (Prod:2:985999)
[Log] [DEMO] AbortError: The operation was aborted. (Prod, line 2)
[Error] Unhandled Promise Rejection: AbortError: The operation was aborted.
    (anonymous function)
    rejectPromise

If I comment out the `videoElement.play()` then the AbortError goes away but then the existing black screen issue still exists.

Per suggestion, I tried wrapping the srcObject setting inside a IntersectionObserver, but still get the AbortError:
const callback: IntersectionObserverCallback = (entries: IntersectionObserverEntry[]) => { 
  if(entries[0].isIntersecting) {
    if (videoElement.srcObject !== videoStream) {
      videoElement.srcObject = videoStream;
      // In Safari, a hidden video element can show a black screen.
      // See https://bugs.webkit.org/show_bug.cgi?id=241152 for more information.
      if (new DefaultBrowserBehavior().requiresVideoPlayWorkaround() && videoElement.paused) {
        videoElement.play();
      }
    }
  }
};

if (!!window.IntersectionObserver) {
  const observer = new IntersectionObserver(callback);
  observer.observe(videoElement);
}

Should I be re-opening this bug?