| Summary: | run-webkit-tests should not fail if all tests found to run are skipped. | ||||||
|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Matt Lewis <jlewis3> | ||||
| Component: | Tools / Tests | Assignee: | Matt Lewis <jlewis3> | ||||
| Status: | RESOLVED FIXED | ||||||
| Severity: | Normal | CC: | ap, ews-watchlist, glenn, jbedard, webkit-bug-importer | ||||
| Priority: | P2 | Keywords: | InRadar | ||||
| Version: | WebKit Nightly Build | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Attachments: |
|
||||||
|
Description
Matt Lewis
2020-04-22 15:18:24 PDT
I don't know about this. Only selecting skipped tests seems like something we want to surface. The exit code really only matter in automation, and if we're skipping all tests, shouldn't we just avoid running that particular set of tests on that particular configuration? So I see your point here but after talking with Alexey, this was something we discussed as being a situation that we should't fail on. Perhaps he can provide a bit more behind his thoughts. Skipping and moving tests is a separate decision from what we run in automation. It is possible that we have some tests run one day, and they get moved to a different suite another day. I don't think that we'd want to change buildbot configuration in lockstep. Created attachment 399204 [details]
Patch
Comment on attachment 399204 [details]
Patch
r+ing because this behavior is changed correctly, although I disagree that we should be changing this behavior. Since Alexey, Matt and our other bot watchers are for this change, so I'm not going to block it.
Committed r261629: <https://trac.webkit.org/changeset/261629> |