WebKit Bugzilla
New
Browse
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
RESOLVED FIXED
99800
Dromaeo should report individual test result
https://bugs.webkit.org/show_bug.cgi?id=99800
Summary
Dromaeo should report individual test result
Ryosuke Niwa
Reported
2012-10-18 20:13:39 PDT
Right now, Dromaeo repeats runs within subtests and report the aggregation of results from subtests. This alienates Dromaoe tests from other performance tests and prevents us from making the test more configurable for the
bug 97510
. We can mitigate the problem by reporting individual result for each subtest, and then summing each record to compute the aggregated result for each run. Say we have subtests t_1, t_2, ... t_n for some test t. Right now, Dromaeo measures 5 runs for t_1, t_2, ... t_n, and computes statistics (stdev, etc...) for each subtest t_i with i=1, ..., n. At the end of the day, we'll get n results, each of which contains mean, median, min, max, & stdev. I'd like to modify Dromaeo slightly so that it'll also report individual values it obtained in each run as "times". So for each t_i, we'll have times(t_i), which is a tuple with 5 values; e.g. (1, 2, 3, 4, 5). In total, we'll have 5n values for t. Then, I can compute the sum of times(t_i)[j] over i=1...n; this value represents a run of the aggregated test t. This is going to make Dromaeo tests behave like any other performance tests that report multiple runs.
Attachments
Work in progress
(5.48 KB, patch)
2012-10-18 20:22 PDT
,
Ryosuke Niwa
no flags
Details
Formatted Diff
Diff
Patch
(13.56 KB, patch)
2012-10-19 13:46 PDT
,
Ryosuke Niwa
eric
: review+
buildbot
: commit-queue-
Details
Formatted Diff
Diff
Show Obsolete
(1)
View All
Add attachment
proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1
2012-10-18 20:22:15 PDT
Created
attachment 169539
[details]
Work in progress
Ryosuke Niwa
Comment 2
2012-10-19 13:46:45 PDT
Created
attachment 169688
[details]
Patch
Build Bot
Comment 3
2012-10-19 16:20:17 PDT
Comment on
attachment 169688
[details]
Patch
Attachment 169688
[details]
did not pass mac-ews (mac): Output:
http://queues.webkit.org/results/14457610
New failing tests: fast/harness/perftests/runs-per-second-log.html
Ryosuke Niwa
Comment 4
2012-12-03 13:32:02 PST
Ping reviewers. This patch is crucial in making other improvements to the perf. test harness.
Eric Seidel (no email)
Comment 5
2012-12-03 13:39:55 PST
Comment on
attachment 169688
[details]
Patch Looks reasonable to me.
Ryosuke Niwa
Comment 6
2012-12-04 02:01:40 PST
Committed
r136492
: <
http://trac.webkit.org/changeset/136492
>
Emil A Eklund
Comment 7
2012-12-04 09:42:45 PST
These tests fail on chromium, please fix or rollout.
Note
You need to
log in
before you can comment on or make changes to this bug.
Top of Page
Format For Printing
XML
Clone This Bug