NEW206636
getUserMedia video stream to texture sometimes gives old frames.
https://bugs.webkit.org/show_bug.cgi?id=206636
Summary getUserMedia video stream to texture sometimes gives old frames.
Nicholas Butko
Reported 2020-01-22 17:40:47 PST
Created attachment 388497 [details] source code for repro case Sometimes calling gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, video) populates the texture with frame data that is old, usually 2-3 frames old but sometimes 15 or more. When drawing these textures to a canvas, this gives an impression that the video feed is jumping, because a scene from a while ago will flash momentarily. The effect is rare in normal conditions but can be reliably triggered by extending the length of an animation frame by a long random interval, e.g. 20ms-200ms. The effect can be mitigated by skipping every other animation frame, i.e. const nativeRaf = window.requestAnimationFrame window.requestAnimationFrame = (fn) => nativeRaf(() => nativeRaf((timestamp) => fn(timestamp))) A repro case can be viewed on an iphone here: 8th.io/ft4h9 The repro app does the following: * getUserMedia -> stream -> video * video -> texture (there is a rotating pool of 10 textures) * texture -> luminance shader -> framebuffer -> readpixels -> uint8array * texture -> canvas -> window * uint8array -> frame hash code based on pixel content -> frame hash history * search frame hash history to find duplicate hashes * if there is a duplicate hash, print the number of frames ago for the duplicate frame pixels (do this for the 5 most recent duplicates), and increment a counter for the number of duplicate frames. We tested this with an iPhone11 Pro with iOS 13.3. Source code for the repro is attached.
Attachments
source code for repro case (7.34 KB, text/javascript)
2020-01-22 17:40 PST, Nicholas Butko
no flags
Radar WebKit Bug Importer
Comment 1 2020-01-22 22:52:34 PST
youenn fablet
Comment 2 2020-01-23 06:00:55 PST
I am wondering whether this still repro with WebKit ToT. We got rid off re-enqueuing samples when the display layer could not go fast enough, which could explain why old samples would come back.
Simon Taylor
Comment 3 2022-01-07 23:41:03 PST
As you mention using a texture pool, this sounds like a duplicate of Bug 203148. Effectively there's an optimization in WebKit that prevents re-uploading frames if the source video hasn't changed since the last texImage2d call, but it fails to check whether a different texture is now bound. In that case the previous content of the newly-bound texture is left unchanged, which with a texture pool will probably appear to be an older frame.
Note You need to log in before you can comment on or make changes to this bug.