Bug 217669

Summary:	REGRESSION (iOS 14): [ iOS wk2 ] service-workers-spinning tests are flaky failures
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	hector_i_lopez, koivisto, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Karl Rackler 2020-10-13 14:17:51 PDT

Description:
http/wpt/service-workers/service-worker-spinning-activate.https.html
http/wpt/service-workers/service-worker-spinning-fetch.https.html
http/wpt/service-workers/service-worker-spinning-install.https.html
http/wpt/service-workers/service-worker-spinning-message.https.html

The first failure that I see this on the dashboard was 09/29/20 at r267741, but I was not able to find a commit that seemed relevant that would have caused this.  
All of the failures test times are approximately 27-29 seconds.  The tests also alternate between flaky failing and time out.  The test behavior has been like this since the tests began running on iOS 14.

I have attempted to reproduce the behavior, but so far have been unsuccessful.

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=http%2Fwpt%2Fservice-workers%2Fservice-worker-spinning-activate.https.html&test=http%2Fwpt%2Fservice-workers%2Fservice-worker-spinning-fetch.https.html&test=http%2Fwpt%2Fservice-workers%2Fservice-worker-spinning-install.https.html&test=http%2Fwpt%2Fservice-workers%2Fservice-worker-spinning-message.https.html&platform=ios&limit=50000

Diff Log:
http/wpt/service-workers/service-worker-spinning-activate.https.html

--- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-activate.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-activate.https-actual.txt
@@ -1,5 +1,7 @@
 
 
-PASS Spin in activate
-PASS Spin after activate
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Spin in activate Test timed out
+NOTRUN Spin after activate
+

Comment 1 Radar WebKit Bug Importer 2020-10-13 14:18:13 PDT

<rdar://problem/70266069>

Comment 2 Karl Rackler 2020-10-13 14:42:11 PDT

I have marked this test as failing while this issue is investigated.
https://trac.webkit.org/changeset/268422/webkit

Comment 3 Hector Lopez 2020-10-19 19:04:27 PDT

http/wpt/service-workers/service-worker-spinning-install.https.html

Test is also a flaky failure on macOS Debug. First occurrence of failure is at r267615 with a crash but has recently only been a failure

History:

https://results.webkit.org/?suite=layout-tests&test=http%2Fwpt%2Fservice-workers%2Fservice-worker-spinning-message.https.html&platform=mac&limit=50000

Diff:
--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-install.https-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/http/wpt/service-workers/service-worker-spinning-install.https-actual.txt
@@ -1,5 +1,7 @@
 
-PASS Spin at running
-PASS Spin in install
-PASS Spin after install
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Spin at running  Test timed out
+NOTRUN Spin in install
+NOTRUN Spin after install
+

Test expectation while investigated:

https://trac.webkit.org/changeset/268706/webkit

Comment 4 Hector Lopez 2020-10-20 18:16:32 PDT

Test expectation was set for;
http/wpt/service-workers/service-worker-spinning-install.https.html
But should have been for:

[tag] [reply] [−] Description Karl Rackler 2020-10-13 14:17:51 PDT
Description:
http/wpt/service-workers/service-worker-spinning-activate.https.html
http/wpt/service-workers/service-worker-spinning-fetch.https.html
http/wpt/service-workers/service-worker-spinning-install.https.html
http/wpt/service-workers/service-worker-spinning-message.https.html

Adjustment:

https://trac.webkit.org/changeset/268778/webkit