WebKit Bugzilla
New
Browse
Search+
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
NEW
206636
getUserMedia video stream to texture sometimes gives old frames.
https://bugs.webkit.org/show_bug.cgi?id=206636
Summary
getUserMedia video stream to texture sometimes gives old frames.
Nicholas Butko
Reported
2020-01-22 17:40:47 PST
Created
attachment 388497
[details]
source code for repro case Sometimes calling gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, video) populates the texture with frame data that is old, usually 2-3 frames old but sometimes 15 or more. When drawing these textures to a canvas, this gives an impression that the video feed is jumping, because a scene from a while ago will flash momentarily. The effect is rare in normal conditions but can be reliably triggered by extending the length of an animation frame by a long random interval, e.g. 20ms-200ms. The effect can be mitigated by skipping every other animation frame, i.e. const nativeRaf = window.requestAnimationFrame window.requestAnimationFrame = (fn) => nativeRaf(() => nativeRaf((timestamp) => fn(timestamp))) A repro case can be viewed on an iphone here: 8th.io/ft4h9 The repro app does the following: * getUserMedia -> stream -> video * video -> texture (there is a rotating pool of 10 textures) * texture -> luminance shader -> framebuffer -> readpixels -> uint8array * texture -> canvas -> window * uint8array -> frame hash code based on pixel content -> frame hash history * search frame hash history to find duplicate hashes * if there is a duplicate hash, print the number of frames ago for the duplicate frame pixels (do this for the 5 most recent duplicates), and increment a counter for the number of duplicate frames. We tested this with an iPhone11 Pro with iOS 13.3. Source code for the repro is attached.
Attachments
source code for repro case
(7.34 KB, text/javascript)
2020-01-22 17:40 PST
,
Nicholas Butko
no flags
Details
View All
Add attachment
proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1
2020-01-22 22:52:34 PST
<
rdar://problem/58825426
>
youenn fablet
Comment 2
2020-01-23 06:00:55 PST
I am wondering whether this still repro with WebKit ToT. We got rid off re-enqueuing samples when the display layer could not go fast enough, which could explain why old samples would come back.
Simon Taylor
Comment 3
2022-01-07 23:41:03 PST
As you mention using a texture pool, this sounds like a duplicate of
Bug 203148
. Effectively there's an optimization in WebKit that prevents re-uploading frames if the source video hasn't changed since the last texImage2d call, but it fails to check whether a different texture is now bound. In that case the previous content of the newly-bound texture is left unchanged, which with a texture pool will probably appear to be an older frame.
Note
You need to
log in
before you can comment on or make changes to this bug.
Top of Page
Format For Printing
XML
Clone This Bug