Bug 250106

Summary:	Remove <!-- webkit-test-runner --> modeline from imported tests
Product:	WebKit	Reporter:	Elliott Williams <emw>
Component:	WebKit Misc.	Assignee:	Nobody <webkit-unassigned>
Status:	NEW ---
Severity:	Normal	CC:	bfulgham, ntim, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
Bug Depends on:	247926
Bug Blocks:

Description Elliott Williams 2023-01-04 14:38:15 PST

We shouldn't export the <!-- webkit-test-runner --> control statements to WPT. Instead, let's mark features WPT uses as `testable`, or mark the tests as expected to fail.

Moreover, I'd argue that we should avoid testing WPT using feature flags that are only turned on by our CI infra, so that our CI's ability to pass tests is reflective of WebKit's public WPT score.

Comment 1 Brent Fulgham 2023-01-04 15:00:55 PST

I think it's reasonable for us to have 'testable' features that run WPT locally to help us avoid regression bits of a feature that work while trying to complete other work.

If we think the feature is usable, we would make it 'preview', and it should show in STP scores for WPT. Once it's on-by-default it would show in stable scores for WPT.

I think this is a good idea to avoid artificial differences in WPT results, and to avoid modifying WPT tests as much as possible.

Comment 2 Tim Nguyen (:ntim) 2023-01-04 15:03:26 PST

I think the problem here is that if we don't export those changes, they get overriden next time someone imports.

Comment 3 Brent Fulgham 2023-01-04 15:12:28 PST

Right -- what I'm saying is that we should ditch the special comment, and just mark relevant features as 'testable' (just as Elliott suggested in this bug).

Comment 4 Tim Nguyen (:ntim) 2023-01-09 02:03:54 PST

Yeah that sounds reasonable.

Comment 5 Radar WebKit Bug Importer 2023-01-11 14:39:16 PST

<rdar://problem/104140583>