Bug 237392

Summary:	REGRESSION (r290658): 2X fast/mediastream/getUserMedia-to-canvas (layout-tests) are constant text failures
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	WebRTC	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=237042

Description Robert Jenner 2022-03-02 14:06:10 PST

The following two fast/mediastream/getUserMedia-to-canvas layout-tests are constant text failures on iOS15:

fast/mediastream/getUserMedia-to-canvas-1.html
fast/mediastream/getUserMedia-to-canvas-2.html

HISTORY:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=fast%2Fmediastream%2FgetUserMedia-to-canvas-1.html&test=fast%2Fmediastream%2FgetUserMedia-to-canvas-2.html

TEXT DIFF for "fast/mediastream/getUserMedia-to-canvas-1.html":
 PASS func: with2DContext, angle: 0, imageBitmap: false
 PASS func: with2DContext, angle: 0, imageBitmap: true
-PASS func: with2DContext, angle: 90, imageBitmap: false
-PASS func: with2DContext, angle: 90, imageBitmap: true
-PASS func: with2DContext, angle: 180, imageBitmap: false
-PASS func: with2DContext, angle: 180, imageBitmap: true
+FAIL func: with2DContext, angle: 90, imageBitmap: false assert_array_approx_equals: white rect not found func: with2DContext, angle: 90, imageBitmap: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: with2DContext, angle: 90, imageBitmap: true assert_array_approx_equals: white rect not found func: with2DContext, angle: 90, imageBitmap: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: with2DContext, angle: 180, imageBitmap: false assert_array_approx_equals: white rect not found func: with2DContext, angle: 180, imageBitmap: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: with2DContext, angle: 180, imageBitmap: true assert_array_approx_equals: white rect not found func: with2DContext, angle: 180, imageBitmap: true property 0, expected 255 +/- 11, expected 255 but got 0
 PASS func: withWebGL, angle: 0, imageBitmap: false, subImage: false, unpackFlipY: false
 PASS func: withWebGL, angle: 0, imageBitmap: false, subImage: false, unpackFlipY: true
 PASS func: withWebGL, angle: 0, imageBitmap: false, subImage: true, unpackFlipY: false
@@ -13,5 +13,5 @@
 PASS func: withWebGL, angle: 0, imageBitmap: true, subImage: false, unpackFlipY: true
 PASS func: withWebGL, angle: 0, imageBitmap: true, subImage: true, unpackFlipY: false
 PASS func: withWebGL, angle: 0, imageBitmap: true, subImage: true, unpackFlipY: true
-PASS func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: false
+FAIL func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0

DIFF URL:
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/r290667%20(1795)/fast/mediastream/getUserMedia-to-canvas-1-pretty-diff.html


TEXT DIFF for "fast/mediastream/getUserMedia-to-canvas-2.html":
-PASS func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: true
-PASS func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: false
-PASS func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: true
-PASS func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: false
-PASS func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: true
-PASS func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: false
-PASS func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: true
-PASS func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: false
-PASS func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: true
-PASS func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: false
-PASS func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: true
-PASS func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: false
-PASS func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: true
-PASS func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: false
-PASS func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: true
+FAIL func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: false, subImage: false, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: false, subImage: true, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: true, subImage: false, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 90, imageBitmap: true, subImage: true, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: false, subImage: false, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: false, subImage: true, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: true, subImage: false, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: false assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: false property 0, expected 255 +/- 11, expected 255 but got 0
+FAIL func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: true assert_array_approx_equals: white rect not found func: withWebGL, angle: 180, imageBitmap: true, subImage: true, unpackFlipY: true property 0, expected 255 +/- 11, expected 255 but got 0
 

DIFF URL:
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/r290667%20(1795)/fast/mediastream/getUserMedia-to-canvas-2-pretty-diff.html

Comment 1 Radar WebKit Bug Importer 2022-03-02 14:06:39 PST

<rdar://problem/89711530>

Comment 2 Robert Jenner 2022-03-02 14:11:02 PST

Was able to reproduce on iOS15 Production ToT using the following test:

run-webkit-tests --iOS-simulator <insert test name here>

I was also able to bisect a regression point. Testing at r290658, did reproduce the failure. It also reproduced at r290657. However, When I checked out r290657 locally and ran testing on the same ToT revision, both tests passed. When I checked out r290658 locally, and ran again I got the failure. 

So it does look like r290658 caused these failures. 

https://github.com/WebKit/WebKit/commit/d08ae850cf27b3ef2f0c82a1bea1788a050c11f7

Comment 3 Robert Jenner 2022-03-02 14:13:49 PST

I'd also like to point out that these constant failures are slowing down EWS, seen here:

https://ews-build.webkit.org/#/builders/68/builds/9518

Comment 4 Robert Jenner 2022-03-02 18:46:18 PST

I have reverted the change that broke these tests here:

https://github.com/WebKit/WebKit/commit/fd21a61ddd42d7b93f4701b5b9523280839148d2