Bug 248052

Summary:	WebKit TestRunner not running masonry layout tests correctly
Product:	WebKit	Reporter:	Sammy Gill <sgill26>
Component:	Tools / Tests	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED WORKSFORME
Severity:	Normal	CC:	ap, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Sammy Gill 2022-11-17 13:52:20 PST

When I try to run the masonry layout tests, the runner does not seem to be running the tests correctly. Most of these tests should fail since most of them are in the TestExpectations. When I run the tests I get 36 unexpected passes which should not happen. This happens on many other layout tests that I run.

For this particular issue, I use "run-webkit-tests LayoutTests/imported/w3c/web-platform-tests/css/css-grid/masonry/tentative/ --no-build"

Comment 1 Radar WebKit Bug Importer 2022-11-17 13:53:00 PST

<rdar://problem/102483669>

Comment 2 Alexey Proskuryakov 2022-11-17 19:11:57 PST

Could you please clarify what doesn't match your expectations?

There are 48 tests in this directory, and none of them are skipped. 44 of them have an ImageOnlyFailure expectation. So if any of them succeed instead of failing, "passed unexpectedly" is printed. 

However, these tests appear to be imported incorrectly. I checked masonry-intrinsic-sizing-004, and I see four files for it:

masonry-intrinsic-sizing-004-expected.html
masonry-intrinsic-sizing-004-ref.html
masonry-intrinsic-sizing-004.html

I don't think that it makes sense to have both -ref and -expected.

Comment 3 Sammy Gill 2023-12-12 12:33:12 PST

These tests have been running fine for me and others for some time now so I don't think we need to keep this bug open. It was probably just user error :)