Bug 214085

Summary:

[ews] Limit layout tests failures to display in the status-bubble tooltip and buildbot summary

Product:

WebKit

Reporter:

Aakash Jain <aakash_jain>

Component:

Tools / Tests

Assignee:

Aakash Jain <aakash_jain>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

aakash_jain, ap, jbedard, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=203678

Attachments:

Description	Flags
Screenshot - unreadable build page	none
Screenshot - unreadable status bubble	none
Patch	none
Patch	none

Description Aakash Jain 2020-07-08 08:49:20 PDT

Created attachment 403784 [details]
Screenshot - unreadable build page

Sometimes a patch can break lot of layout tests, especially after r262379 which removed the limit of 30 failures for webkit-wpt-import-bot@igalia.com

In those cases, the status-bubble tries to display all those failures, and becomes unreadable. Similarly Buildbot's build page (and builder page) display all the failing test names in the summary string, and becomes pretty unreadable. For e.g.: https://ews-build.webkit.org/#/builders/24/builds/21100 has 642 test failures and the build page is unreadable. Screenshots attached.

We should limit the number of failures to display in the status-bubble and summary strings. Complete list of failing tests can still be seen in the logs of the test-run, in the test-failures section, and in the results.html

Comment 1 Aakash Jain 2020-07-08 09:01:40 PDT

Created attachment 403785 [details]
Screenshot - unreadable status bubble

Comment 2 Aakash Jain 2020-07-08 09:13:36 PDT

Created attachment 403788 [details]
Patch

Comment 3 Alexey Proskuryakov 2020-07-08 11:30:39 PDT

Comment on attachment 403788 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=403788&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:1929
> +        new_failures_string = ', '.join(list(new_failures)[:self.NUM_FAILURES_TO_DISPLAY])

I think that it would be better to do sorted(), to avoid potential confusion caused by a random subset of failures.

Comment 4 Aakash Jain 2020-07-08 12:10:38 PDT

Created attachment 403801 [details]
Patch

Comment 5 EWS 2020-07-08 12:34:23 PDT

Committed r264127: <https://trac.webkit.org/changeset/264127>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 403801 [details].

Comment 6 Radar WebKit Bug Importer 2020-07-08 12:35:20 PDT

<rdar://problem/65236219>

Comment 7 Aakash Jain 2020-07-22 07:50:50 PDT

This change is working fine, e.g.: https://ews-build.webkit.org/#/builders/24/builds/22231