Bug 212836

Summary: REGRESSION: macOS WK1 tests intermittently exiting early after 500 failures
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, ggaren, jbedard, Lawrence.j, pgyanchandani, simon.fraser, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Comment 1 Radar WebKit Bug Importer 2020-06-05 10:10:10 PDT
<rdar://problem/64033778>
Comment 2 Ryan Haddad 2020-06-05 12:56:45 PDT
In at least a few cases that I checked, it looks like the expected output from one test is showing up in the test that follows it, causing them all to fail:

--- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-actual.txt
@@ -1,11 +1,9 @@
-Tests invalidation on align-self style change (just overflow). Passes if there is no red.
+Tests invalidation on align-self style change. Passes if there is no red.
 
 (repaint rects
-  (rect 0 2 200 200)
-  (rect 0 2 200 200)
-  (rect 0 52 200 200)
-  (rect 0 252 200 100)
-  (rect 0 2 200 50)
-  (rect 0 2 800 14)
+  (rect 0 302 100 300)
+  (rect 0 302 100 50)
+  (rect 0 52 100 300)
+  (rect 100 52 100 300)
 )
 

--- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-actual.txt
@@ -1,9 +1,8 @@
-Tests invalidation on align-self style change. Passes if there is no red.
+Tests invalidation on align-items style change (just overflow). Passes if there is no red.
 
 (repaint rects
-  (rect 0 302 100 300)
-  (rect 0 302 100 50)
-  (rect 0 52 100 300)
-  (rect 100 52 100 300)
+  (rect 0 2 200 350)
+  (rect 0 52 200 300)
+  (rect 0 2 800 14)
 )
Comment 3 Jonathan Bedard 2020-06-05 13:11:47 PDT
Looking through history, this is definitely a recent regression:
https://results.webkit.org/suites?flavor=wk1&platform=mac

The regression is before r262527, and https://trac.webkit.org/changeset/262527/webkit is actually my best guess. If that is the commit to blame, it means that commit has simply exposed an underlying issue, not caused it, since that commit adds a bunch of new tests.
Comment 4 Jonathan Bedard 2020-06-05 13:28:26 PDT
(In reply to Jonathan Bedard from comment #3)
> Looking through history, this is definitely a recent regression:
> https://results.webkit.org/suites?flavor=wk1&platform=mac
> 
> The regression is before r262527, and
> https://trac.webkit.org/changeset/262527/webkit is actually my best guess.
> If that is the commit to blame, it means that commit has simply exposed an
> underlying issue, not caused it, since that commit adds a bunch of new tests.

Although in a discussion with bot watchers, it was pointed out to me that the test added in https://trac.webkit.org/changeset/262524/webkit seems to be the last test ran in any given worker before all tests in that worker start failing. So it seems that reverting https://trac.webkit.org/changeset/262524/webkit is the best course of action.
Comment 5 Jason Lawrence 2020-06-05 13:45:38 PDT
In an effort to resolve this issue, I have reverted r262524.
https://trac.webkit.org/changeset/262651/webkit
Comment 6 Alexey Proskuryakov 2020-06-07 13:41:46 PDT
Hasn't happened since then, marking resolved.