macOS WK1 tests are intermittently exiting early after 500 failures https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5060 https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5053 https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5038 https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5035 https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15054 https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15037 https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15035 https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15029
<rdar://problem/64033778>
In at least a few cases that I checked, it looks like the expected output from one test is showing up in the test that follows it, causing them all to fail: --- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-expected.txt +++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-actual.txt @@ -1,11 +1,9 @@ -Tests invalidation on align-self style change (just overflow). Passes if there is no red. +Tests invalidation on align-self style change. Passes if there is no red. (repaint rects - (rect 0 2 200 200) - (rect 0 2 200 200) - (rect 0 52 200 200) - (rect 0 252 200 100) - (rect 0 2 200 50) - (rect 0 2 800 14) + (rect 0 302 100 300) + (rect 0 302 100 50) + (rect 0 52 100 300) + (rect 100 52 100 300) ) --- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-expected.txt +++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-actual.txt @@ -1,9 +1,8 @@ -Tests invalidation on align-self style change. Passes if there is no red. +Tests invalidation on align-items style change (just overflow). Passes if there is no red. (repaint rects - (rect 0 302 100 300) - (rect 0 302 100 50) - (rect 0 52 100 300) - (rect 100 52 100 300) + (rect 0 2 200 350) + (rect 0 52 200 300) + (rect 0 2 800 14) )
Looking through history, this is definitely a recent regression: https://results.webkit.org/suites?flavor=wk1&platform=mac The regression is before r262527, and https://trac.webkit.org/changeset/262527/webkit is actually my best guess. If that is the commit to blame, it means that commit has simply exposed an underlying issue, not caused it, since that commit adds a bunch of new tests.
(In reply to Jonathan Bedard from comment #3) > Looking through history, this is definitely a recent regression: > https://results.webkit.org/suites?flavor=wk1&platform=mac > > The regression is before r262527, and > https://trac.webkit.org/changeset/262527/webkit is actually my best guess. > If that is the commit to blame, it means that commit has simply exposed an > underlying issue, not caused it, since that commit adds a bunch of new tests. Although in a discussion with bot watchers, it was pointed out to me that the test added in https://trac.webkit.org/changeset/262524/webkit seems to be the last test ran in any given worker before all tests in that worker start failing. So it seems that reverting https://trac.webkit.org/changeset/262524/webkit is the best course of action.
In an effort to resolve this issue, I have reverted r262524. https://trac.webkit.org/changeset/262651/webkit
Hasn't happened since then, marking resolved.