Bug 207118

Summary: run-javascriptcore-tests: Don't split binary results
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: Tools / TestsAssignee: Jonathan Bedard <jbedard>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, ryanhaddad, webkit-bug-importer, zhifei_fang
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch none

Description Jonathan Bedard 2020-02-03 08:28:51 PST
Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work. We should just report aggregated results for a single binary.
Comment 1 Jonathan Bedard 2020-02-03 08:30:41 PST
Created attachment 389515 [details]
Patch
Comment 2 Jonathan Bedard 2020-02-03 08:32:15 PST
https://results.webkit.org/suites?suite=javascriptcore-tests demonstrates how my original approach basically failed.
Comment 3 Zhifei Fang 2020-02-03 10:13:36 PST
Comment on attachment 389515 [details]
Patch

unofficially r+, in long term we should unify all the test output format, and let those binaries generate that standard output
Comment 4 Aakash Jain 2020-02-03 10:23:34 PST
(In reply to Jonathan Bedard from comment #0)
> Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work.
What's the issue?
Comment 5 Jonathan Bedard 2020-02-03 10:46:04 PST
(In reply to Aakash Jain from comment #4)
> (In reply to Jonathan Bedard from comment #0)
> > Experience over the last 2 months has shown that attempting to split up results from a single JSC test binary doesn't really work.
> What's the issue?

We can't really do it consistently, a sub-test which fails will sometimes have a different name than when it passes and often, exception messages will be tagged as actual test names.
Comment 6 Aakash Jain 2020-02-03 10:59:54 PST
rs=me
Comment 7 Jonathan Bedard 2020-02-03 11:37:43 PST
Committed r255578: <https://trac.webkit.org/changeset/255578>
Comment 8 Radar WebKit Bug Importer 2020-02-03 11:41:44 PST
<rdar://problem/59118504>