Display list of test failures along-with the layout-test build step in the buildbot page. It would make it easy to notice the test failures without opening the complete 'layout test results' in a new tab. It would also make it easier to compare test-failures from the first run, re-run and re-run-without-patch.
Created attachment 395273 [details] Patch
e.g.: https://ews-build.webkit-uat.org/#/builders/34/builds/119 https://ews-build.webkit-uat.org/#/builders/34/builds/119/steps/10/logs/test-failures
Created attachment 395302 [details] Screenshot
Nice! Would it be possible to get the type of failure in that list too? Like text diff, timeout, crash, and such?
(In reply to John Wilander from comment #4) > Nice! Would it be possible to get the type of failure in that list too? Like > text diff, timeout, crash, and such? Just curious, are engineers not generally aware of the stdout output: https://ews-build.webkit-uat.org/#/builders/34/builds/119/steps/10/logs/stdio? That has what you're looking for.
(In reply to Jonathan Bedard from comment #5) > (In reply to John Wilander from comment #4) > > Nice! Would it be possible to get the type of failure in that list too? Like > > text diff, timeout, crash, and such? > > Just curious, are engineers not generally aware of the stdout output: > https://ews-build.webkit-uat.org/#/builders/34/builds/119/steps/10/logs/ > stdio? That has what you're looking for. That I don't know. I'm just generally in favor of minimal amounts of clicks, scrolling, and irrelevant information. If a patch failed, I want to see why immediately and I want to see as little other information as possible. The sooner I know something failed, the better. Personally, I would also love a "Stop processing" feature so that when I see some issue and I realize I need to fix my patch, I can immediately free up resources for others. Aakash and I have discussed that. It's for another change but in the same vein. Another thing we discussed was the ability to give a hint to EWS on which part of the test suite to test first, again striving for early failure. The quicker I know my patch is not ready for review or landing, the better for everyone.
(In reply to Jonathan Bedard from comment #5) > Just curious, are engineers not generally aware of the stdout output I personally find it confusing since it also list many tests which flaked (which I am not interested in).
(In reply to John Wilander from comment #4) > Nice! Would it be possible to get the type of failure in that list too? Like text diff, timeout, crash, and such? It's possible, but not straight-forward. I will attempt that in a separate patch.
Committed r259457: <https://trac.webkit.org/changeset/259457> All reviewed patches have been landed. Closing bug and clearing flags on attachment 395273 [details].
<rdar://problem/61258395>
Deployed on the server. Seems to be working fine. e.g.: https://ews-build.webkit.org/#/builders/32/builds/6728