Bug 213896 - [ macOS wk1 ] imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html is flaky failing
Summary: [ macOS wk1 ] imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-0...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-07-02 12:34 PDT by Karl Rackler
Modified: 2020-07-02 15:47 PDT (History)
5 users (show)

See Also:

Attachments
Patch (1.28 KB, patch)
2020-07-02 13:07 PDT, Karl Rackler 		no flags Details | Formatted Diff | Diff
Patch (1.61 KB, patch)
2020-07-02 14:58 PDT, Karl Rackler 		no flags Details | Formatted Diff | Diff
Patch (1.41 KB, patch)
2020-07-02 15:10 PDT, Karl Rackler 		no flags Details | Formatted Diff | Diff
Show Obsolete (2) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2020-07-02 12:34:51 PDT 
imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html 

Description:
The test is flaky failing on [ macOS wk1 ] release.  It was passing consistently until 5/29/2020.

History: 
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-flexbox%2Foverflow-auto-008.html&flavor=wk1&platform=mac

Diff:  
--- /Volumes/Data/slave/catalina-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008-actual.txt
@@ -1,4 +1,4 @@
  
 
-PASS hbox dimensions 
+FAIL hbox dimensions assert_equals: expected 104 but got 89
Comment 1 Radar WebKit Bug Importer 2020-07-02 12:36:17 PDT 
<rdar://problem/65046136>
Comment 2 Karl Rackler 2020-07-02 13:07:04 PDT 
Created attachment 403391 [details]
Patch
Comment 3 EWS 2020-07-02 13:35:34 PDT 
Committed r263859: <https://trac.webkit.org/changeset/263859>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 403391 [details].
Comment 4 Karl Rackler 2020-07-02 14:58:04 PDT 
Reopening to attach new patch.
Comment 5 Karl Rackler 2020-07-02 14:58:05 PDT 
Created attachment 403400 [details]
Patch
Comment 6 EWS 2020-07-02 15:00:01 PDT 
Tools/Scripts/svn-apply failed to apply attachment 403400 [details] to trunk.
Please resolve the conflicts and upload a new patch.
Comment 7 Karl Rackler 2020-07-02 15:04:42 PDT 
I am able to reproduce this issue on 263855 using the commands below.

run-webkit-tests --iterations 500 --exit-after-n-failures 3 --force -f imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html

r263855:
[22/500] imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html failed unexpectedly (text diff)
[22/500] imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html failed unexpectedly (text diff)
[22/500] imported/w3c/web-platform-tests/css/css-flexbox/overflow-auto-008.html failed unexpectedly (text diff)

6 tests ran as expected, 3 didn't (491 didn't run):
Comment 8 Karl Rackler 2020-07-02 15:10:48 PDT 
Created attachment 403403 [details]
Patch
Comment 9 EWS 2020-07-02 15:31:59 PDT 
Committed r263864: <https://trac.webkit.org/changeset/263864>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 403403 [details].
Comment 10 Alexey Proskuryakov 2020-07-02 15:47:02 PDT 
Re-opening, because this only updated expectations.

Looking at the complete history, I see failures all the way back to 5/19, which is when the test was imported. So it was imported flaky.