Bug 244167

Summary:	[macOS] Screen recording in background tab only works for about 1 minute
Product:	WebKit	Reporter:	modler.daniel
Component:	WebRTC	Assignee:	youenn fablet <youennf>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	eric.carlson, jer.noble, webkit-bug-importer, youennf
Priority:	P2	Keywords:	InRadar
Version:	Safari 15
Hardware:	Mac (Intel)
OS:	macOS 11

Description modler.daniel 2022-08-21 04:17:46 PDT

I'm currently trying and failing to workaround the following:

1. Call getDisplayMedia() to get a screen stream (Tried different fps, etc. constraints already)
2. Attach a MediaRecorder instance to it, with a timeslice of about 5000ms (but it doesn't matter a lot)
3. Put Safari in the background (minimize it)
4. Watch some video in another application or do normal work
5. After about one minute and 30 seconds, switch back to the Safari tab
6. Save the video to disk

I expect the video to contain all activity done on the system.
What happens however is that after about one minute, the framerate of the video is reduced to about one frame every 10 seconds, which is unusable for a screen recording.

I also tried to render the video to a canvas, and use HTMLCanvasElement.captureStream() instead as input to MediaRecorder.
I can get JavaScript to execute every few ms and render the video (by using a ScriptProcessorNode). (I need to use WebGL for that, due to captureStream() being black for 2d Contexts).
However, the video shows the same behaviour - there are only changes every 10 seconds after a minute.

I see this is related to some kind of background throttling. However, background throttling shouldn't be applied if MediaRecorder instances are running. 

Currently, we have to recommend our users to use a Chromium based browser for our web application, due to that.

Comment 1 modler.daniel 2022-08-21 04:22:49 PDT

I also found out that this seems to be related to the display stream and not to MediaRecorder. It's easy to reproduce that by using WebRTC.

* Create a room at jitsi meet in Safari
* Join the room on another pc
* Share only screen on Safari
* Put Safari into background
* Wait one minute

After that, the screen stream will be throttled down to about 1 frame every 1 seconds.

Comment 2 modler.daniel 2022-08-22 01:03:28 PDT

Found out that this happens on Intel Macs, but not on Silicon Macs (with the same latest Safari 15.6).

Unfortunately, I cannot detect Apple Silicon from my web application.

Comment 3 Radar WebKit Bug Importer 2022-08-22 01:51:26 PDT

<rdar://problem/98966467>

Comment 4 youenn fablet 2022-08-22 03:18:43 PDT

Pull request: https://github.com/WebKit/WebKit/pull/3528

Comment 5 modler.daniel 2022-08-22 03:51:19 PDT

Just wanted to add a few more observations in case they are helpful:

* This happens also when the tab is additionally actively sharing webcam video. Webcam video is *not* affected by the problem, but display media framerate drops to 0.1fps after ~1 minute. Also reproducible using Jitsi.

* When rendering display media to an intermediate canvas at a fast interval (e.g. every 200ms) and recording the resulting captureStream(), and rendering frame information to every frame (frame number and milliseconds), the following observation can be made: 
  (a) The output captureStream has a good frame rate, but the input (the HTMLVideoElement containing the original display stream) only gets a new frame every 10 seconds (after the first minute)
  (b) When testing using a non-display stream for the input HTMLVideoElement (e.g. by using a sample mp4 file), the problem does *not* occur.

So this problem seems to directly affect only the stream returned by getDisplayMedia().

Comment 6 EWS 2022-08-25 03:29:16 PDT

Committed 253769@main (c888c485b787): <https://commits.webkit.org/253769@main>

Reviewed commits have been landed. Closing PR #3528 and removing active labels.