Bug 239871

Summary:	[ macOS wk1 ] Four imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower tests are flaky image failures
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Karl Rackler <rackler>
Status:	NEW ---
Severity:	Normal	CC:	koivisto, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Karl Rackler 2022-04-28 13:56:00 PDT

Description:
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html

The first failure that I saw on the dashboard for Big Sur was 2/15/2022 at r289847, and covers EWS rebase and is not relevant to causing this issue.  The first failure that I saw on the dashboard for Monterey was 3/17/2022 at r291422 but is modifying WebGPU/buffer and does not appear to be causing this issue.   This test seems to have been flaky since the start. 

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-003.html&platform=mac&flavor=wk1&limit=50000

Image Diff:
https://build.webkit.org/results/Apple-Monterey-Debug-WK1-Tests/250087@main%20(1970)/imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003-diffs.html

Comment 1 Radar WebKit Bug Importer 2022-04-28 13:56:20 PDT

<rdar://problem/92485701>

Comment 2 Karl Rackler 2022-04-28 14:11:49 PDT

REPRODUCTION STEPS
I can reproduce this on ToT (r293579).

Command: 
run-webkit-tests --root -1 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --pixel-tests --tolerance 0.0364 --iterations 100 -f --no-retry

Result:
Unexpected flakiness: image-only failures (1)
  imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html [ Pass ImageOnlyFailure ]

Comment 3 Karl Rackler 2022-04-28 14:26:14 PDT

I have marked this test as a flaky image failure while this issue is investigated.

Comment 4 Karl Rackler 2022-04-28 14:29:17 PDT

Pull request: https://github.com/WebKit/WebKit/pull/429

Comment 5 EWS 2022-04-28 15:48:22 PDT

Test gardening commit r293594 (250103@main): <https://commits.webkit.org/250103@main>

Reviewed commits have been landed. Closing PR #429 and removing active labels.

Comment 6 Karl Rackler 2022-08-25 16:18:40 PDT

Description:
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html
ADDING: 
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-007.html
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-015.html
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-102.html

After a series of consistent passes, the test imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html begins to fail at approximately 249091@main.

The other three tests after a series of consistent passes, begin to fail at approximately245577@main.  

History:
https://results.webkit.org/?limit=50000&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-007.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-015.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-102.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-003.html&platform=mac&flavor=wk1&recent=false

I have marked these tests as flaky image failures while this issue is investigated.

Comment 7 EWS 2022-08-25 16:25:50 PDT

Test gardening commit 253798@main (a3ccc9ae0d48): <https://commits.webkit.org/253798@main>

Reviewed commits have been landed. Closing PR #3681 and removing active labels.