Bug 248824

Summary: Many cache-storage tests failing on EWS in sequence
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: Service WorkersAssignee: Sihui Liu <sihui_liu>
Status: RESOLVED FIXED    
Severity: Normal CC: ryanhaddad, sihui_liu, simon.fraser, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=248826

Description Jonathan Bedard 2022-12-06 09:46:13 PST
EWS results indicate it's possible for cache-storage tests to kick a worker into a bad state, causing all cache-storage tests in that worker to fail. Some examples of test runs where we're seeing this:
https://ews-build.webkit.org/#/builders/70/builds/23009
https://ews-build.webkit.org/#/builders/70/builds/23008
https://ews-build.webkit.org/#/builders/70/builds/22996
https://ews-build.webkit.org/#/builders/70/builds/22992
Comment 1 Radar WebKit Bug Importer 2022-12-06 09:49:00 PST
<rdar://problem/103030498>
Comment 2 Jonathan Bedard 2022-12-06 09:50:30 PST
Pull request: https://github.com/WebKit/WebKit/pull/7209
Comment 3 EWS 2022-12-06 10:01:23 PST
Test gardening commit 257415@main (9d139ae059b9): <https://commits.webkit.org/257415@main>

Reviewed commits have been landed. Closing PR #7209 and removing active labels.
Comment 4 Ryan Haddad 2022-12-06 10:27:47 PST
Pull request: https://github.com/WebKit/WebKit/pull/7212
Comment 5 EWS 2022-12-06 10:34:55 PST
Test gardening commit 257416@main (d53145462ffa): <https://commits.webkit.org/257416@main>

Reviewed commits have been landed. Closing PR #7212 and removing active labels.
Comment 6 Jonathan Bedard 2022-12-06 11:23:40 PST
The first test we see this in is https://ews-build.webkit.org/#/builders/70/builds/22896.

With that in mind, I think the "regression" range is:
http://commits.webkit.org/compare/257364@main...257373@main

That said, I put regression in quotes, because I suspect https://commits.webkit.org/257367@main changed test ordering to "cause" this, which doesn't really help us narrow down the issue.
Comment 7 Jonathan Bedard 2022-12-06 12:52:43 PST
Pull request: https://github.com/WebKit/WebKit/pull/7220
Comment 8 Jonathan Bedard 2022-12-07 21:42:37 PST
https://ews-build.webkit.org/#/builders/70/builds/23167 is when we skip the css-nesting tests
https://ews-build.webkit.org/#/builders/70/builds/23212 is when we run the css-nesting and our cache-storage tests.

https://commits.webkit.org/257367@main is definitely the point where EWS regressed, but doesn't really help us determine when the behavior itself regressed, and perhaps it isn't even a regression, but rather, a problem our cache-storage tests have always had and was just revealed by changing test order slightly.
Comment 9 youenn fablet 2022-12-08 00:07:44 PST
(In reply to Jonathan Bedard from comment #8)
> https://ews-build.webkit.org/#/builders/70/builds/23167 is when we skip the
> css-nesting tests
> https://ews-build.webkit.org/#/builders/70/builds/23212 is when we run the
> css-nesting and our cache-storage tests.
> 
> https://commits.webkit.org/257367@main is definitely the point where EWS
> regressed, but doesn't really help us determine when the behavior itself
> regressed, and perhaps it isn't even a regression, but rather, a problem our
> cache-storage tests have always had and was just revealed by changing test
> order slightly.

The cache-storage tests seem to pass ok in https://ews-build.webkit.org/#/builders/70/builds/23212, I only see a gamepad/gamepad-polling-access.html failure.
I do not see a proof that would tell us whether test order is the reason, nor do we have a proof to rule it out.

https://ews-build.webkit.org/#/builders/70/builds/2321 seems to indicate we could try reenable cache-storage tests.
Comment 10 Sihui Liu 2023-03-30 15:37:24 PDT
Pull request: https://github.com/WebKit/WebKit/pull/12207
Comment 11 EWS 2023-03-30 17:15:37 PDT
Committed 262380@main (9579456a8bc1): <https://commits.webkit.org/262380@main>

Reviewed commits have been landed. Closing PR #12207 and removing active labels.