Bug 217938

Summary:	REGRESSION (268605?): [ Mojave+ wk2 ] 4 wpt/service-workers tests are flaky failures
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	cdumez, jiewen_tan, webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Karl Rackler 2020-10-19 18:56:39 PDT

Description:
http/wpt/service-workers/fetchEvent.https.html
http/wpt/service-workers/header-filtering.https.html
http/wpt/service-workers/mac/throttleable.https.html
http/wpt/service-workers/online.https.html

The most recent failure that I see this on the dashboard, after a series of consistent passing, was 10/16/20 at 268605, and introduces a new daemon to handle all WebAuthn requests, which could have raised the flakiness. The change was introduced here https://trac.webkit.org/changeset/268605/webkit.

I have made multiple reproductions attempts, but so far have been unable to reproduce the issue.

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=http%2Fwpt%2Fservice-workers%2FfetchEvent.https.html&test=http%2Fwpt%2Fservice-workers%2Fheader-filtering.https.html&test=http%2Fwpt%2Fservice-workers%2Fmac%2Fthrottleable.https.html&test=http%2Fwpt%2Fservice-workers%2Fonline.https.html&platform=mac&limit=50000

Diff Log:
http/wpt/service-workers/header-filtering.https.html

--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/header-filtering.https-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/header-filtering.https-actual.txt
@@ -1,12 +1,13 @@
+
 
 PASS Prepare tests: setup worker and register the client
 PASS Prepare tests: Add a frame controlled by service worker
-PASS Test same-origin fetch
-PASS Test cors cross-origin fetch
-PASS Test no-cors cross-origin fetch
-PASS Test same-origin script load
-PASS Test no-cors script load
-PASS Test cors script load
-PASS Test HTML load
+FAIL Test same-origin fetch assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test cors cross-origin fetch assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test no-cors cross-origin fetch assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test same-origin script load assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test no-cors script load assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test cors script load assert_array_equals: value is "Test requires internals API to get all response headers", expected array
+FAIL Test HTML load assert_array_equals: value is "Test requires internals API to get all response headers", expected array
 PASS After tests clean-up
 
http/wpt/service-workers/online.https.html

--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/online.https-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/online.https-actual.txt
@@ -1,4 +1,4 @@
 
 PASS Setup worker
-PASS Test service worker online/offline
+FAIL Test service worker online/offline assert_equals: expected "PASS" but got "FAIL: test require internals"
 

http/wpt/service-workers/service-worker-spinning-install.https.html

--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-install.https-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-install.https-actual.txt
@@ -1,5 +1,7 @@
 
-PASS Spin at running
-PASS Spin in install
-PASS Spin after install
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Spin at running  Test timed out
+NOTRUN Spin in install
+NOTRUN Spin after install
+

http/wpt/service-workers/service-worker-spinning-message.https.html

--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-message.https-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-message.https-actual.txt
@@ -1,4 +1,6 @@
 
-PASS Spin in message
-PASS Spin after message
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Spin in message Test timed out
+NOTRUN Spin after message
+

Comment 1 Radar WebKit Bug Importer 2020-10-19 18:57:15 PDT

<rdar://problem/70466134>

Comment 2 Karl Rackler 2020-10-19 19:09:32 PDT

I have marked these tests as failing while this issue is investigated.
https://trac.webkit.org/changeset/268707/webkit

Comment 3 Karl Rackler 2020-10-19 19:22:18 PDT

Diff Log

http/wpt/service-workers/fetchEvent.https.html

--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/fetchEvent.https-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/fetchEvent.https-actual.txt
@@ -1,10 +1,10 @@
 
 PASS Setup worker
-PASS FetchEvent respondWith should throw if called twice
+FAIL FetchEvent respondWith should throw if called twice assert_equals: expected "PASS" but got "Exception: undefined is not an object (evaluating 'self.internals.createBeingDispatchedFetchEvent')"
 PASS FetchEvent request is SameObject
-PASS FetchEvent should be in error if responding with undefined
-PASS FetchEvent should be in error if not responding with a Response
-PASS FetchEvent should be in error if responding with a Promise that does not resolve to a Response
-PASS FetchEvent should be in error if responding with a Promise that rejects
-PASS FetchEvent should resolve when responding with a Response
+FAIL FetchEvent should be in error if responding with undefined assert_equals: expected "PASS" but got "Exception: Can't find variable: internals"
+FAIL FetchEvent should be in error if not responding with a Response assert_equals: expected "PASS" but got "Exception: Can't find variable: internals"
+FAIL FetchEvent should be in error if responding with a Promise that does not resolve to a Response assert_equals: expected "PASS" but got "Exception: Can't find variable: internals"
+FAIL FetchEvent should be in error if responding with a Promise that rejects assert_equals: expected "PASS" but got "Exception: Can't find variable: internals"
+FAIL FetchEvent should resolve when responding with a Response assert_equals: expected "PASS" but got "Exception: Can't find variable: internals"

http/wpt/service-workers/Mac/throttleable.https.html

--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/mac/throttleable.https-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/mac/throttleable.https-actual.txt
@@ -1,5 +1,5 @@
 
 PASS Setup worker
-PASS Service worker should not be throttleable
-PASS Service worker should be throttleable
+FAIL Service worker should not be throttleable assert_false: expected false got "needs internals"
+FAIL Service worker should be throttleable assert_true: expected true got "needs internals"