Bug 209226 - [ews] commit-queue should comment on bug in case of test failures
Summary: [ews] commit-queue should comment on bug in case of test failures
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks: 201934
  Show dependency tree / graph
 
Reported: 2020-03-18 06:45 PDT by Aakash Jain
Modified: 2020-03-19 15:04 PDT (History)
4 users (show)

See Also:

Attachments
Patch (4.26 KB, patch)
2020-03-18 07:01 PDT, Aakash Jain 		jbedard: review+
Details | Formatted Diff | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2020-03-18 06:45:02 PDT 
[ews] commit-queue should comment on bug and set cq- flag in case of test failures.
Comment 1 Aakash Jain 2020-03-18 07:01:03 PDT 
Created attachment 393845 [details]
Patch
Comment 2 Aakash Jain 2020-03-18 07:28:02 PDT 
Sample run: https://ews-build.webkit-uat.org/#/builders/26/builds/1593
Comment 3 Jonathan Bedard 2020-03-18 09:45:59 PDT 
Comment on attachment 393845 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=393845&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:1830
> +            self.build.addStepsAfterCurrentStep([CommentOnBug(), SetCommitQueueMinusFlagOnPatch()])

Is this the current behavior? If I recall, this logic doesn't apply to flaky tests, but I can see some circumstances where this behavior is undesirable.
Comment 4 Aakash Jain 2020-03-18 11:09:06 PDT 
(In reply to Jonathan Bedard from comment #3)
> Is this the current behavior? If I recall, this logic doesn't apply to flaky tests.
Yes, I think so. The comment and cq- would be when a test failure is confirmed (test consistently failed in first and second run, and passed in clean-tree-run). Test which were found to be flaky during the run would not affect the result.
Comment 5 Aakash Jain 2020-03-18 11:15:37 PDT 
Committed r258652: <https://trac.webkit.org/changeset/258652>
Comment 6 Radar WebKit Bug Importer 2020-03-18 11:16:19 PDT 
<rdar://problem/60592347>
Comment 7 Aakash Jain 2020-03-19 15:04:58 PDT 
This change seems to be working fine.

Patch with test failure: https://ews-build.webkit.org/#/builders/28/builds/63

Flaky test: https://ews-build.webkit.org/#/builders/28/builds/66