Bug 240616

Summary:	REGRESSION (r294317): [ iOS ] Thirteen layout tests are flaky or consistent timeouts
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Karl Rackler <rackler>
Status:	REOPENED ---
Severity:	Normal	CC:	ap, kkinnunen, ryanhaddad, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=240613

Description Karl Rackler 2022-05-18 18:20:06 PDT

Description:
editing/input/cocoa/autocorrect-off.html
editing/input/cocoa/autocorrect-on.html
editing/undo-manager/undo-manager-item-labels.html
fast/forms/datalist/datalist.html
fast/events/ios/keyup.html
fast/shadow-dom/ios/accessory-bar-work-on-input-with-tabindex-in-shadow-tree.html
fast/forms/ios/autocapitalize-words.html
fast/forms/ios/form-control-refresh/select/select-multiple-picker.html
fast/forms/ios/inputmode-none.html
fast/forms/ios/inputmode-removing-none.html
fast/forms/ios/no-scrolling-when-moving-focus-between-adjacent-fields.html
fast/forms/validation-messages.html
mathml/fractions-invalid.html

The first failure I saw on the dashboard was on 5/17/2022 at r294317, which could have introduced the flaky or consistent timeouts.  All of the tests have increased the test times.  For an example are, an average test time at r294316 at 7 seconds and an average of 35 seconds at r294317 and forward.

All of these tests start the following behavior at r294317.

The following are consistent timeouts on [ iOS ]
editing/input/cocoa/autocorrect-off.html
editing/input/cocoa/autocorrect-on.html
fast/events/ios/keyup.html
fast/shadow-dom/ios/accessory-bar-work-on-input-with-tabindex-in-shadow-tree.html
fast/forms/ios/autocapitalize-words.html
fast/forms/ios/form-control-refresh/select/select-multiple-picker.html
fast/forms/ios/no-scrolling-when-moving-focus-between-adjacent-fields.html
mathml/fractions-invalid.html

The following are flaky timeouts on [ iOS ]
editing/undo-manager/undo-manager-item-labels.html
fast/forms/ios/inputmode-none.html flaky
fast/forms/validation-messages.html flaky
fast/forms/ios/inputmode-removing-none.html flaky

fast/forms/datalist/datalist.html is a flaky timeout or crash on [ iOS ]


REPRODUCTION STEPS
I can reproduce this on 294317, but unable to reproduce it on 294316 or earlier.

Command: 
run-webkit-tests --root 294317 --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/ios/autocapitalize-words.html

Result: 
Regressions: Unexpected timeouts (1)
  fast/forms/ios/autocapitalize-words.html [ Timeout ]

History:
https://results.webkit.org/?platform=ios&limit=50000&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=editing%2Finput%2Fcocoa%2Fautocorrect-off.html&test=editing%2Finput%2Fcocoa%2Fautocorrect-on.html&test=editing%2Fundo-manager%2Fundo-manager-item-labels.html&test=fast%2Fevents%2Fios%2Fkeyup.html&test=fast%2Fforms%2Fdatalist%2Fdatalist.html&test=fast%2Fforms%2Fios%2Fautocapitalize-words.html&test=fast%2Fforms%2Fios%2Fform-control-refresh%2Fselect%2Fselect-multiple-picker.html&test=fast%2Fforms%2Fios%2Finputmode-none.html&test=fast%2Fforms%2Fios%2Finputmode-removing-none.html&test=fast%2Fforms%2Fios%2Fno-scrolling-when-moving-focus-between-adjacent-fields.html&test=fast%2Fforms%2Fvalidation-messages.html&test=fast%2Fshadow-dom%2Fios%2Faccessory-bar-work-on-input-with-tabindex-in-shadow-tree.html&test=mathml%2Ffractions-invalid.html

Diff:
editing/input/cocoa/autocorrect-off.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250699@main%20(2597)/editing/input/cocoa/autocorrect-off-diff.txt

editing/input/cocoa/autocorrect-on.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250699@main%20(2597)/editing/input/cocoa/autocorrect-on-diff.txt

editing/undo-manager/undo-manager-item-labels.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250699@main%20(2597)/editing/undo-manager/undo-manager-item-labels-diff.txt

fast/forms/datalist/datalist.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250699@main%20(2597)/fast/forms/datalist/datalist-diff.txt

fast/events/ios/keyup.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/events/ios/keyup-diff.txt

fast/shadow-dom/ios/accessory-bar-work-on-input-with-tabindex-in-shadow-tree.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/shadow-dom/ios/accessory-bar-work-on-input-with-tabindex-in-shadow-tree-diff.txt

fast/forms/ios/autocapitalize-words.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/forms/ios/autocapitalize-words-diff.txt

fast/forms/ios/form-control-refresh/select/select-multiple-picker.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/forms/ios/form-control-refresh/select/select-multiple-picker-diff.txt

fast/forms/ios/inputmode-none.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250637@main%20(2581)/fast/forms/ios/inputmode-none-diff.txt

fast/forms/ios/inputmode-removing-none.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/250637@main%20(2581)/fast/forms/ios/inputmode-removing-none-diff.txt

fast/forms/ios/no-scrolling-when-moving-focus-between-adjacent-fields.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/forms/ios/no-scrolling-when-moving-focus-between-adjacent-fields-diff.txt
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/forms/ios/no-scrolling-when-moving-focus-between-adjacent-fields-stderr.txt

fast/forms/validation-messages.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/fast/forms/validation-messages-diff.txt

mathml/fractions-invalid.html
https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/250637@main%20(2624)/mathml/fractions-invalid-diff.txt

Comment 1 Radar WebKit Bug Importer 2022-05-18 18:20:44 PDT

<rdar://problem/93541706>

Comment 2 Karl Rackler 2022-05-18 18:39:43 PDT

I have marked this test as a flaky or consistent slow or one crash while this issue is investigated.

Comment 3 Karl Rackler 2022-05-18 18:44:33 PDT

Pull request: https://github.com/WebKit/WebKit/pull/760

Comment 4 EWS 2022-05-18 18:48:26 PDT

Test gardening commit r294455 (250721@main): <https://commits.webkit.org/250721@main>

Reviewed commits have been landed. Closing PR #760 and removing active labels.

Comment 5 Alexey Proskuryakov 2022-05-19 19:19:49 PDT

This is the same revision as in bug 240613, and both bugs track many tests. I don't see how it makes sense to have two bugs in this situation (and we probably just need to revert ASAP).

Comment 6 Karl Rackler 2022-05-20 07:53:25 PDT

Reverted r294317 at r294452 (250720@trunk) and this resolved the issue.

Comment 7 Karl Rackler 2022-05-20 09:19:16 PDT

Re-opening for pull request https://github.com/WebKit/WebKit/pull/835

Comment 8 Karl Rackler 2022-05-20 09:33:41 PDT

Pull request: https://github.com/WebKit/WebKit/pull/836

Comment 9 EWS 2022-05-20 09:36:28 PDT

Test gardening commit r294546 (250801@main): <https://commits.webkit.org/250801@main>

Reviewed commits have been landed. Closing PR #836 and removing active labels.