run-webkit-tests should not fail if all tests found to run are skipped. Currently if tests are found to run but all of those tests are skipped, we return a failing value of -1. We should still fail if there are no test to run, but we shouldn't if all tests are skipped.
<rdar://problem/62213993>
I don't know about this. Only selecting skipped tests seems like something we want to surface. The exit code really only matter in automation, and if we're skipping all tests, shouldn't we just avoid running that particular set of tests on that particular configuration?
So I see your point here but after talking with Alexey, this was something we discussed as being a situation that we should't fail on. Perhaps he can provide a bit more behind his thoughts.
Skipping and moving tests is a separate decision from what we run in automation. It is possible that we have some tests run one day, and they get moved to a different suite another day. I don't think that we'd want to change buildbot configuration in lockstep.
Created attachment 399204 [details] Patch
Comment on attachment 399204 [details] Patch r+ing because this behavior is changed correctly, although I disagree that we should be changing this behavior. Since Alexey, Matt and our other bot watchers are for this change, so I'm not going to block it.
Committed r261629: <https://trac.webkit.org/changeset/261629>