Bug 209281

Summary:

[ iOS wk2 and Mac wk2 ] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html is flaky failing.

Product:

WebKit

Reporter:

Jason Lawrence <Lawrence.j>

Component:

Tools / Tests

Assignee:

Rob Buis <rbuis>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

darin, rbuis, webkit-bot-watchers-bugzilla, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Mac

OS:

macOS 10.14

See Also:

https://bugs.webkit.org/show_bug.cgi?id=204147

Attachments:

Description	Flags
Patch	none

Description Jason Lawrence 2020-03-19 08:15:38 PDT

imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

Description:
This test is flaky failing on Mac wk2 and iOS wk2. The flaky failures are visible since the introduction of the test.

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Ffetch%2Fstale-while-revalidate%2Fframe-removal.html&limit=50000

Diff:
--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal-actual.txt
@@ -1,4 +1,4 @@
   
 
-PASS Frame removal should cancel its revalidations 
+FAIL Frame removal should cancel its revalidations assert_equals: expected "2" but got "1"

Comment 1 Radar WebKit Bug Importer 2020-03-19 08:16:22 PDT

<rdar://problem/60634122>

Comment 2 Jason Lawrence 2020-03-19 08:29:58 PDT

I can reproduce this issue with r258694 using the command below.
run-webkit-tests --iterations 1400 --exit-after-n-failures 3 --force -f --no-retry imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html

[12/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[32/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
[63/1400] imported/w3c/web-platform-tests/fetch/stale-while-revalidate/frame-removal.html failed unexpectedly (text diff)
Exiting early after 3 failures. 60 tests run.


57 tests ran as expected, 3 didn't (1340 didn't run):

Comment 3 Jason Lawrence 2020-03-19 09:51:16 PDT

I have marked this test as failing while this issue is investigated.
https://trac.webkit.org/changeset/258703/webkit

Comment 4 Rob Buis 2020-04-24 02:34:40 PDT

This should be fixed by https://bugs.webkit.org/show_bug.cgi?id=210905.

Comment 5 Rob Buis 2020-04-24 02:35:02 PDT

Oops, closing wrong bug.

Comment 6 Rob Buis 2020-05-05 05:43:40 PDT

Created attachment 398499 [details]
Patch

Comment 7 Darin Adler 2020-05-09 17:13:12 PDT

Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

> LayoutTests/imported/w3c/ChangeLog:15
> +        * web-platform-tests/fetch/stale-while-revalidate/frame-removal.html:
> +        * web-platform-tests/fetch/stale-while-revalidate/resources/stale-frame.py:
> +        (main):

Will this change get upstreamed?

Comment 8 EWS 2020-05-10 00:26:20 PDT

Committed r261455: <https://trac.webkit.org/changeset/261455>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 398499 [details].

Comment 9 Rob Buis 2020-05-11 00:40:48 PDT

Comment on attachment 398499 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=398499&action=review

>> LayoutTests/imported/w3c/ChangeLog:15
>> +        (main):
> 
> Will this change get upstreamed?

I would like to, but Google has some problems with the test related to the chrome implementation, see https://github.com/web-platform-tests/wpt/pull/22292. I'll try to follow the hints in that issue to see if we can get to a test that works for multiple implementations.