Bug 102500

Summary: DOM/DOMWalk.html is flaky in Chromium Win Release Perf bot
Product: WebKit Reporter: Kentaro Hara <haraken>
Component: Tools / TestsAssignee: Dominic Cooney <dominicc>
Status: RESOLVED FIXED    
Severity: Normal CC: dominicc, noel.gordon, rniwa, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch none

Kentaro Hara
Reported 2012-11-16 06:40:46 PST
It sometimes show the following error: Running DOM/DOMWalk.html (35 of 112) runFunction returned a non-positive value: 0 runFunction returned a non-positive value: 0 FAILED Finished: 9.609000 s See http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Perf%29/builds/4412/steps/perf-test/logs/stdio
Attachments
Patch (1.49 KB, patch)
2013-01-15 00:59 PST, Dominic Cooney 		no flags
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Dominic Cooney
Comment 1 2013-01-15 00:30:51 PST
*** Bug 104866 has been marked as a duplicate of this bug. ***
Dominic Cooney
Comment 2 2013-01-15 00:59:15 PST
Created attachment 182715 [details] Patch
Dominic Cooney
Comment 3 2013-01-15 01:03:11 PST
I am assuming that zero is a plausible result for some of the (micro) benchmarks in DOM/DOMWalk.html. One thing that makes me suspicious is that Dromaeo results on the Windows bots are typically numerically larger than those of the Mac bots, yet the Mac bot does not exhibit this failure. Perhaps the timer works differently on those platforms. If this warrants more investigation, the alternative is to add more instrumentation to Dromaeo so we can see which specific test functions are failing (it is a flaky set, usually >1 but from zero to a few tests fail this way.) For now proposing that we assume the result is correct and allow zero as a (very fast) result to green the Windows perf bot.
Kentaro Hara
Comment 4 2013-01-15 01:11:31 PST
Comment on attachment 182715 [details] Patch Looks reasonable to me as an immediate fix to suppress the win bot errors, though we want to improve the test so it won't return zero (zero means that we're not testing something meaningful).
Dominic Cooney
Comment 5 2013-01-15 01:17:28 PST
Comment on attachment 182715 [details] Patch Thank you for the quick review. I filed bug 106882 to improve the test.
WebKit Review Bot
Comment 6 2013-01-15 01:43:51 PST
Comment on attachment 182715 [details] Patch Clearing flags on attachment: 182715 Committed r139728: <http://trac.webkit.org/changeset/139728>
WebKit Review Bot
Comment 7 2013-01-15 01:43:54 PST
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.