RESOLVED FIXED 276428
Tools/Scripts/run-webkit-tests does not recognize WPT crash tests in a crashtest directory 
https://bugs.webkit.org/show_bug.cgi?id=276428
Summary Tools/Scripts/run-webkit-tests does not recognize WPT crash tests in a crasht...
Frédéric Wang (:fredw)
Reported 2024-07-10 08:12:35 PDT
I understand we somehow support WPT crashtests now: https://web-platform-tests.org/writing-tests/crashtest.html https://searchfox.org/wubkat/search?q=crashtests&path=Tools%2FScripts&case=false&regexp=false However, when running $ Tools/Scripts/run-webkit-tests imported/w3c/web-platform-tests/mathml/crashtests I get "All tests skipped." If instead I do $ Tools/Scripts/run-webkit-tests imported/w3c/web-platform-tests/css/css-tables/crashtests I get "All 7 tests ran as expected." So only 7 of the 24 crash tests are considered, apparently only those with an explicit *-crash* suffix. $ ls LayoutTests/imported/w3c/web-platform-tests/css/css-tables/crashtests/ | wc --lines 24 $ ls LayoutTests/imported/w3c/web-platform-tests/css/css-tables/crashtests/*-crash* | wc --lines 7
Attachments
Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2024-07-17 08:13:14 PDT
<rdar://problem/131926145>
Ahmad Saleem
Comment 2 2024-07-17 08:20:28 PDT
Something related but might be different - it also case with SVG crash tests and editing related test case. It can import but it does not generate 'expected test files'.
Sam Sneddon [:gsnedders]
Comment 3 2024-07-17 09:14:38 PDT
https://github.com/WebKit/WebKit/blob/6eef85d192a8f02fa74fc620854c83dd0fff6fed/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py#L612-L620 should be handling this, but clearly isn't, unless we're somehow also ignoring them.
Harry Chen
Comment 4 2024-09-27 08:53:18 PDT
It seems that the tests aren't running since they're listed in `resource-files.json` wrongfully -- this commit [1] was the one that added them. I believe that the importer script used was on this revision [2], since the time lines up and it would also explain why `-crash.` tests aren't in `resource-files.json` but `crashtests` tests are (since the commit only checks for `-crash.` tests). Since it looks like the logic was fixed as part of the commit in [3], I think just removing the tests from `resource-files.json` should be good enough. [1]: https://github.com/WebKit/WebKit/commit/a31f8e4d6253f04c8d955e42985e6e018d3a1a86 [2]: https://github.com/WebKit/WebKit/commit/d8b1589e4915042608bb1a1a55bd291bcec76d51#diff-955991178e0e7971ad17ceface65532b503e98975aec31d0fc7093e0d80c3dc5R167-R169 [3]: https://github.com/WebKit/WebKit/commit/b70287103be9d7a826cb2697e7a951536dc2a04b
Harry Chen
Comment 5 2024-09-27 11:59:19 PDT
Pull request: https://github.com/WebKit/WebKit/pull/34374
EWS
Comment 6 2024-09-28 04:40:20 PDT
Committed 284409@main (5a279668079a): <https://commits.webkit.org/284409@main> Reviewed commits have been landed. Closing PR #34374 and removing active labels.
Note You need to log in before you can comment on or make changes to this bug.