Bug 34984

Summary: new-run-webkit-tests: MASTER BUG: Switch all webkit.org bots over
Product: WebKit Reporter: Eric Seidel (no email) <eric>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, abecsi, aroben, darin, dglazkov, dpranke, galpeter, hayato, joepeck, kbalazs, laszlo.gombos, leandro, mjs, mrowe, ossy, rakuco, rgabor, rniwa, tonyg, vestbo, webkit.review.bot, zoltan
Priority: P2 Keywords: InRadar, NRWT, ToolsHitList
Version: 528+ (Nightly build)   
Hardware: All   
OS: All   
Bug Depends on: 34826, 34983, 34985, 34986, 35001, 35006, 35049, 35053, 35055, 35265, 35266, 35358, 35359, 37007, 37736, 37738, 37739, 38063, 38756, 43565, 55907, 56047, 56729, 56730, 56731, 57640, 58293, 58625, 59894, 60162, 62178, 62194, 62425, 62497, 63199, 63494, 63523, 63549, 63786, 63791, 63832, 63838, 63839, 63845, 63956, 63973, 63983, 64071, 64086, 64092, 64346, 64347, 64445, 64514, 64564, 64812, 65719, 66227, 66228, 67216, 67268, 70416, 71087, 71173, 71364    
Bug Blocks: 57310    
Attachments:
Description Flags
Use a blacklist of unsupported platforms instead of a whitelist none

Eric Seidel (no email)
Reported 2010-02-16 13:26:56 PST
MASTER BUG: Switch webkit.org to using new python run-webkit-tests There are a lot of blocking issues. This will be a multi-week or possibly multi-month project. But I'm creating this tracking bug to help keep tabs on all the blocking issues.
Attachments
Use a blacklist of unsupported platforms instead of a whitelist (3.90 KB, patch)
2011-10-28 15:17 PDT, Eric Seidel (no email) 		no flags
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Mark Rowe (bdash)
Comment 1 2010-02-19 17:12:46 PST
Should there be bugs about making this work on each of the supported platforms? Mac OS X 10.4 in particular comes to mind as it has the oldest version of Python of the supported platforms.
Eric Seidel (no email)
Comment 2 2010-02-24 12:27:14 PST
Perhaps. I figure we'll cross that bridge when we come to it. I have no insight into Apple's deprecation plans, but its also possible that Tiger support will be dropped from WebKit before this work is completed. :)
Dirk Pranke
Comment 3 2011-03-10 16:29:00 PST
Taking ownership ... I plan to start a push to actually get this done, hopefully starting as early as next week (I just need to wrap up another project first, which should happen today or tomorrow).
Adam Barth
Comment 4 2011-03-10 16:30:04 PST
(In reply to comment #3) > Taking ownership ... I plan to start a push to actually get this done, hopefully starting as early as next week (I just need to wrap up another project first, which should happen today or tomorrow). That's exciting!
Adam Roben (:aroben)
Comment 5 2011-05-26 11:45:23 PDT
<rdar://problem/9509321>
Dirk Pranke
Comment 6 2011-06-21 21:15:14 PDT
Eric, I'm punting this to you now :)
Eric Seidel (no email)
Comment 7 2011-06-22 11:15:40 PDT
(In reply to comment #6) > Eric, I'm punting this to you now :) Thank you again Dirk for all your hard work on this. I'll pick this up next week.
Eric Seidel (no email)
Comment 8 2011-07-06 00:00:58 PDT
NRWT is enabled for all bots on build.webkit.org except Windows and WebKit2 as of: http://trac.webkit.org/changeset/90440 Hopefully it will stick!
Eric Seidel (no email)
Comment 9 2011-07-19 01:32:18 PDT
The change stuck. All bots except for Apple Win, WebKit2 and --leaks are running NRWT. Fixes for the remaining bots are underway. I'm taking vacation the next two weeks, so the remaining bots are unlikely to switch until August.
Eric Seidel (no email)
Comment 10 2011-08-08 10:04:23 PDT
I'm back. My goal is to switch WebKit2 and --leaks this week. I'll coordinate with Adam Roben and other windows folks to make sure we're moving forward on Win32 support, but it's not on my immediate goal list for this week. A stretch goal is to move some/all the existing bots over to parallel execution (which is the real benefit of this work).
Balazs Kelemen
Comment 11 2011-08-08 15:43:12 PDT
I have a good news on the topic: The Qt WebKit2 bot (hosted at http://build.webkit.sed.hu/) has already successfully switched.
Eric Seidel (no email)
Comment 12 2011-08-15 09:51:57 PDT
Update: Fixed a bunch of random bugs last week and switched the --leaks bot. WK2 works, but I've not switched the bots yet. Win and Lion theoretically work, but I've not tested. I will be working on bidi-isolate (bug 65617) this week. I will resume work on NRWT Aug. 22nd, with the gaol of switching the WK2 bots as soon as possible. WK2 (and a bunch of these other bugs) boil down to one issue that NRWT uses a ServerProcess abstraction which reads line-by-line only from stdout. stderr is collected in one long string and processed after the process is done. This means that we can't detect timeouts (since they're sent via #timeout over stderr) or WebProcess crashes (for wk2). I just need to redesign the ServerProcess abstraction to allow reading from both stdout and stderr in a line-by-line fashion. This will be the first thing I do when I resume NRWT work in a week.
Eric Seidel (no email)
Comment 13 2011-08-29 12:11:46 PDT
OK. My work on bug 65617 is basically done (just waiting for review). Back to working on NRWT full time.
Eric Seidel (no email)
Comment 14 2011-10-28 15:17:36 PDT
Created attachment 112919 [details] Use a blacklist of unsupported platforms instead of a whitelist
Eric Seidel (no email)
Comment 15 2011-10-28 15:24:44 PDT
Once this last patch lands, all ports will use NRWT by default, with two specific exceptions (win and qt-arm), and I think we should close this bug. I'm working my way through the polish bugs now.
WebKit Review Bot
Comment 16 2011-10-28 16:32:00 PDT
Comment on attachment 112919 [details] Use a blacklist of unsupported platforms instead of a whitelist Clearing flags on attachment: 112919 Committed r98780: <http://trac.webkit.org/changeset/98780>
WebKit Review Bot
Comment 17 2011-10-28 16:32:10 PDT
All reviewed patches have been landed. Closing bug.
Ryosuke Niwa
Comment 18 2011-10-29 00:55:25 PDT
It appears that this patch broke Qt bots: http://build.webkit.org/builders/Qt%20Linux%20Release/builds/39090/steps/layout-test/logs/stdio
Ryosuke Niwa
Comment 19 2011-10-29 01:18:56 PDT
I'm going to rollout the patch since Qt port's losing 100% of the test coverage due to this bug.
Ryosuke Niwa
Comment 20 2011-10-29 01:29:16 PDT
Reopen the bug since the patch was rolled out in r98801.
Csaba Osztrogonác
Comment 21 2011-10-29 02:07:08 PDT
Thanks for rolling out and unbreak Qt bots. :) I checked this fail, it was only a typo: isArm() vs isARM() .
Ryosuke Niwa
Comment 22 2011-10-29 02:09:20 PDT
(In reply to comment #21) > Thanks for rolling out and unbreak Qt bots. :) I checked this fail, it was only a typo: isArm() vs isARM() . Oh, could you reland the patch with that fix in that case?
Balazs Kelemen
Comment 23 2011-10-29 07:46:05 PDT
(In reply to comment #22) > (In reply to comment #21) > > Thanks for rolling out and unbreak Qt bots. :) I checked this fail, it was only a typo: isArm() vs isARM() . > > Oh, could you reland the patch with that fix in that case? Relanded in r98804.
Eric Seidel (no email)
Comment 24 2011-10-29 11:25:09 PDT
Thank you arm.
Raphael Kubo da Costa (:rakuco)
Comment 25 2012-05-13 15:09:38 PDT
Can we close this one now?
Ryosuke Niwa
Comment 26 2012-05-13 15:10:47 PDT
(In reply to comment #25) > Can we close this one now? Has Windows port moved to new-run-webkit-tests yet?
Dirk Pranke
Comment 27 2012-05-14 11:54:59 PDT
Nope, the Apple win port is still running ORWT. I would like to fix it, but the Apple win port is generally pretty unhappy these days so it's hard to isolate which issues are NRWT-related and which are just broken. Hopefully that will be fixed soon.
Csaba Osztrogonác
Comment 28 2014-02-06 07:52:49 PST
All bots run new-run-webkit-tests now, which is simple run-webkit-tests :)
Note You need to log in before you can comment on or make changes to this bug.