Bug 241205

Summary: REGRESSION (250836@main): [ iOS ] fast/forms/textfield-outline.html is a flaky failure
Product: WebKit Reporter: Karl Rackler <rackler>
Component: New BugsAssignee: Ryan Reno <rreno>
Status: RESOLVED FIXED    
Severity: Normal CC: brandonstewart, hhjalmarsson, rreno, simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Karl Rackler 2022-06-01 16:57:27 PDT 
Description:
fast/forms/textfield-outline.html

The first failure that I saw on the dashboard was on 5/20/2022 at 250788@main.

REPRODUCTION STEPS
I can reproduce this on 250778@main, but unable to reproduce it on 250777@main or earlier.

Command: 
run-webkit-tests --ios-simulator --debug --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Pass ]

History:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fforms%2Ftextfield-outline.html&platform=ios&limit=50000

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"
Comment 1 Radar WebKit Bug Importer 2022-06-01 16:57:46 PDT 
<rdar://problem/94255807>
Comment 2 Karl Rackler 2022-06-01 17:14:25 PDT 
250778@main caused the flaky failure.
Reverted at 250801@main and test passing again.
250836@main caused the flaky failure again.
Comment 3 Karl Rackler 2022-06-01 17:28:31 PDT 
REPRODUCTION STEPS
I can reproduce this on 250836@main but cannot reproduce it on 250835@main or back earlier to 250801@main.

Command: 
run-webkit-tests --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Regressions: Unexpected text-only failures (1)
  fast/forms/textfield-outline.html [ Failure ]
Comment 4 Karl Rackler 2022-06-02 06:49:39 PDT 
I have marked this test as a flaky failure while this issue is investigated.
Comment 5 Karl Rackler 2022-06-02 06:53:29 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/1240
Comment 6 EWS 2022-06-02 06:57:19 PDT 
Test gardening commit r295126 (251217@main): <https://commits.webkit.org/251217@main>

Reviewed commits have been landed. Closing PR #1240 and removing active labels.
Comment 7 Ryan Reno 2022-06-28 12:49:55 PDT 
I've been able to reproduce this locally. It's consistently the same diff - the scrollX is missing from the layer. Even though it's flaky it at least fails in the same way every time.
Comment 8 Simon Fraser (smfr) 2022-07-06 11:52:34 PDT 
Looks like the timer-based scroll on focus() is racing with test completion.
Comment 9 Simon Fraser (smfr) 2022-07-06 11:53:03 PDT 
See scheduleScrollToFocusedElement().
Comment 10 Ryan Reno 2022-07-12 14:36:27 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/2339
Comment 11 EWS 2022-07-13 09:44:40 PDT 
Committed 252417@main (2e8bcf908a7d): <https://commits.webkit.org/252417@main>

Reviewed commits have been landed. Closing PR #2339 and removing active labels.
Comment 12 Hercules Hjalmarsson 2022-07-15 14:14:08 PDT 
Re-opening bug. It looks like after landing 252417@main all the tests are passing consistently except on iOS15 wk2 Release arm64 that is a flaky text failure.

DIFF:
--- /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"

COMMAND:
(Using arm64 machine)

run-webkit-tests --ios-simulator --iterations=50 fast/forms/textfield-outline.html

RESULT:

~18% failure rate.

Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Missing Pass ]
Comment 13 Hercules Hjalmarsson 2022-07-15 14:16:21 PDT 
Test ran on iOS ToT.
Comment 14 Hercules Hjalmarsson 2022-07-18 13:01:38 PDT 
Set expectation. Flaky text failure is now only happening on iOS15 wk2 Release arm64.
Comment 15 EWS 2022-07-18 13:02:06 PDT 
Test gardening commit 252575@main (945df646018d): <https://commits.webkit.org/252575@main>

Reviewed commits have been landed. Closing PR #2516 and removing active labels.
Comment 16 Ryan Reno 2022-11-17 20:31:52 PST 
Pull request: https://github.com/WebKit/WebKit/pull/6620
Comment 17 EWS 2022-11-18 14:33:54 PST 
Committed 256856@main (93ece5e17f0a): <https://commits.webkit.org/256856@main>

Reviewed commits have been landed. Closing PR #6620 and removing active labels.