Bug 240538

Summary:	REGRESSION (r294316): [ iOS ] Eight imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation are a consistent failure
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Karl Rackler <rackler>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	ntim, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Karl Rackler 2022-05-17 13:41:27 PDT

Description:
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-checkValidity.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-badInput.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeOverflow.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeUnderflow.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-stepMismatch.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valid.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valueMissing.html
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-willValidate.html

The first failure I saw on the dashboard was on 5/17/2022 at r294317 and looks to have introduced the consistent failure. 

History:
https://results.webkit.org/?platform=ios&limit=50000&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-checkValidity.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-badInput.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-rangeOverflow.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-rangeUnderflow.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-stepMismatch.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-valid.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-validity-valueMissing.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fforms%2Fconstraints%2Fform-validation-willValidate.html

Link to Diffs:
imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-checkValidity.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-checkValidity-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-badInput.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-badInput-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeOverflow.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeOverflow-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeUnderflow.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-rangeUnderflow-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-stepMismatch.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-stepMismatch-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valid.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valid-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valueMissing.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-validity-valueMissing-diff.txt

imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-willValidate.html
https://build.webkit.org/results/Apple-iPadOS-15-Simulator-Release-WK2-Tests/250643%40main%20(1755)/imported/w3c/web-platform-tests/html/semantics/forms/constraints/form-validation-willValidate-diff.txt

Comment 1 Radar WebKit Bug Importer 2022-05-17 13:41:43 PDT

<rdar://problem/93449475>

Comment 2 Karl Rackler 2022-05-17 13:54:24 PDT

I have marked this test as a consistent failure while this issue is investigated.

Comment 3 Karl Rackler 2022-05-17 13:58:14 PDT

Pull request: https://github.com/WebKit/WebKit/pull/690

Comment 4 Ryan Haddad 2022-05-17 14:00:06 PDT

Are all the diffs showing PASS where we once showed FAIL? If so, sounds like a progression and we can rebaseline the tests.

Comment 5 Karl Rackler 2022-05-17 15:09:15 PDT

Rebaseline tests

Comment 6 Karl Rackler 2022-05-17 15:16:00 PDT

Pull request: https://github.com/WebKit/WebKit/pull/698

Comment 7 EWS 2022-05-17 15:32:10 PDT

Test gardening commit r294354 (250661@main): <https://commits.webkit.org/250661@main>

Reviewed commits have been landed. Closing PR #698 and removing active labels.

Comment 8 Tim Nguyen (:ntim) 2022-05-17 21:56:14 PDT

r294316 seems more likely

Comment 9 Karl Rackler 2022-05-18 08:15:40 PDT

(In reply to Tim Nguyen (:ntim) from comment #8)
> r294316 seems more likely

I agree.

Comment 10 Karl Rackler 2022-05-18 08:16:18 PDT

Rebaseline of tests complete.
Confirm all tests passing.