The test is very flaky since r267815. Inspecting the range r267800-r267815, r267807 introduced several changes in inspector code touching WebKitGTK code. I haven't bisected the range though (before r267815 the test sporadically timeout). To reproduce the failure: $ Tools/Scripts/run-webkit-tests --gtk --release --iterations 5 inspector/console/queryHolders.html Diff: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/r268725%20(16529)/inspector/console/queryHolders-diff.txt --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -75,7 +75,9 @@ [RepeatedReferenceHolder, Root] (2) -- Running test case: CommandLineAPI.queryHolders.OnlyHeldByDebugger -PASS: The result should have 0 items. +FAIL: The result should have 0 items. + Expected: 0 + Actual: 2 -- Running test case: CommandLineAPI.queryHolders.NonObject PASS: Should produce an exception.
I bisected the range [r267800-r267815] and I got r267814 as the first revision where I can reproduce the regression.
Carlos, I don't see anything obviously wrong which would cause this. Can you take a peek?
(In reply to Brian Burg from comment #2) > Carlos, I don't see anything obviously wrong which would cause this. Can you > take a peek? It seems this same issue was tracked in bug https://bugs.webkit.org/show_bug.cgi?id=217193 Recently the implementation of `item` introduced in r267814 was renamed to `at` in r268760, among other changes. Since then the test is no longer flaky, although it sporadically times out as it used to happen before. https://results.webkit.org/?suite=layout-tests&test=inspector%2Fconsole%2FqueryHolders.html&platform=GTK&platform=WPE
*** This bug has been marked as a duplicate of bug 217193 ***
This test is no longer flaky in WebKitGTK as the results in the WebKitGTK post-commit bot report this test has been consistently failing in the last 4000 revisions. The current diff is: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/r278645%20%282223%29/inspector/console/queryHolders-diff.txt --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -54,13 +54,13 @@ [Root] (1) -- Running test case: CommandLineAPI.queryHolders.PromiseThen -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseCatch -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseFinally -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.EventListenerCallback [<div id="event-listener-target">, Root] (2)
Now this test is failing with a different output: --- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-expected.txt +++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/inspector/console/queryHolders-actual.txt @@ -54,13 +54,13 @@ [Root] (1) -- Running test case: CommandLineAPI.queryHolders.PromiseThen -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseCatch -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.PromiseFinally -[Root, object] (2) +[Object, Root] (2) -- Running test case: CommandLineAPI.queryHolders.EventListenerCallback [<div id="event-listener-target">, Root] (2) @@ -75,7 +75,9 @@ [RepeatedReferenceHolder, Root] (2) -- Running test case: CommandLineAPI.queryHolders.OnlyHeldByDebugger -PASS: The result should have 0 items. +FAIL: The result should have 0 items. + Expected: 0 + Actual: 2 -- Running test case: CommandLineAPI.queryHolders.NonObject PASS: Should produce an exception. The reason for the [Root, object] and [Object, Root] difference is due to using a different label for Objects in GLib abd other WebKit ports. https://webkit-search.igalia.com/webkit/source/LayoutTests/inspector/console/queryHolders.html#83 It can be easily solved by capitalizing the propertiy names before sorting. As for the other part, the error is happening in this line: https://webkit-search.igalia.com/webkit/source/LayoutTests/inspector/console/queryHolders.html#162 I have no idea why size is 2 for GLib ports but 0 in other ports.