inspector/layers/layer-tree-manager.html is a text failure: --- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layer-tree-manager-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layer-tree-manager-actual.txt @@ -3,7 +3,9 @@ == Running test suite: LayerTreeManager -- Running test case: LayerTreeManager.layersForNode.root -PASS: Returned array should include all layers. +FAIL: Returned array should include all layers. + Expected: 5 + Actual: 0 PASS: Array elements should be Layer instances. -- Running test case: LayerTreeManager.layerTreeMutations.overlappingNodeIds --- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layers-anonymous-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layers-anonymous-actual.txt @@ -1,3 +1,5 @@ +CONSOLE MESSAGE: line 58: TypeError: undefined is not an object (evaluating 'anonymousLayers[0].nodeId') +FAIL: Timed out waiting for notifyDone to be called This is a test inspector/layers/layers-anonymous.html times out with output: @@ -15,14 +17,5 @@ === Check layers === -PASS: Expected number of anonymous layers. -PASS: The sole anonymous layer has a non-zero node id. -PASS: The sole anonymous layer has a :first-letter pseudo-element. +FAIL: Expected number of anonymous layers. Expected 1 but got 0 -=== Check node === - -PASS: Node was found. -PASS: Node has expected localName. -PASS: Node has id. -PASS: Node has expected id. - Also, this isn't limited to leaks, but it definitely happens far more often there.

Possibly related to https://trac.webkit.org/changeset/234330/webkit?

Created attachment 346793 [details] Test List I was able to reproduce the two failures by running the tests from the imported/ directory that ran directly before the inspector/ tests from the same test runner. I am going to work on bisecting down the blamed tests. Command Used: run-webkit-tests --test-list Test_list_directory --child-processes 1 -1 --leaks

Created attachment 346846 [details] Test List New list of tests. Noting that the imported/ directory is no longer ran. The results are flakey at around a 2/3 chance of having the two tests fail.

(In reply to Truitt Savell from comment #4) > Created attachment 346846 [details] > Test List > > New list of tests. Noting that the imported/ directory is no longer ran. The > results are flakey at around a 2/3 chance of having the two tests fail. For now I think this is the best list I can provide. removing any more tests from it either stops the failure condition or makes the test failure far less consistent.

Updating title to reflect that this isn't limited to the leaks bot, but it does happen far more frequently there.

Found that these three tests run in succession makes the failure occur around 90% of the time: imported/w3c/web-platform-tests/html/dom/interfaces.html inspector/layers/layer-tree-manager.html inspector/layers/layers-anonymous.html I used the command: run-webkit-tests --root testbuild-235621 imported/w3c/web-platform-tests/html/dom/interfaces.html inspector/layers/layer-tree-manager.html inspector/layers/layers-anonymous.html --child-processes 1 -1 --iterations 10 Using this I was able to begin bisecting the webkit revisions to find the start of the regression. I found the revision that caused these failures to occur to be https://trac.webkit.org/changeset/234330/webkit. If you run the above command on r234329 no failures will occur, if it is ran on r234330 than the failure reproduces.

<rdar://problem/44118147>