Bug 251107
| Summary: | [GTK][WPE] WebGL2 failures for GTK and WPE ports | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Alejandro G. Castro <alex> |
| Component: | WebKitGTK | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | bugs-noreply, clopez |
| Priority: | P2 | ||
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=257623 | ||
| Bug Depends on: | 251080, 251106, 252905, 252906, 252909 | ||
| Bug Blocks: | |||
Alejandro G. Castro
We use this bug as a reference to keep track of the failures and flakies we have marked when we enabled WebGL2 in bug 251106 and bug 251080.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Carlos Alberto Lopez Perez
*** Bug 252906 has been marked as a duplicate of this bug. ***
Carlos Alberto Lopez Perez
*** Bug 252905 has been marked as a duplicate of this bug. ***
Carlos Alberto Lopez Perez
On 260799@main I have updated the list of WebGL2 flaky tests based on what the post-commit bots reported since this tests were unskipped.
However, I tried to run locally the WebGL2 tests and I still get lots of unexpected failures and crashes that the bots don't report (using flatpak).
With GTK happens more than with WPE.
I'm not marking that now as I don't have more time to keep investigating this for today. Just leaving a note here about this issues.
On the GTK EWS this tests caused issues related to "persistent flakiness" where you could run the same test 11 times and the 11 times it would fail and then you rebuild and then it passes the 11 times.
So the GTK EWS WK2 bot was reporting false positives on random pull-request of unexpected failures on WebGL2 tests :(
Check the links to the EWS on bug 252905 for examples of that
Carlos Alberto Lopez Perez
We continue to have issues regarding this.
Some WebGL2 tests exhibit a flakiness behavior that make the EWS give false positives.
The EWS does this:
1. Run layout tests with patch and annotate the failures
2. Repeat each failure from 1. 10 times and annotate the ones that failed the 10 times, so not even once they passed-
3. Build WebKit without patch and run 10 times the ones that failed on 2. the 10 times.
4. Report as new failures the tests that passed 10 times without patch but failed 10 times with it.
So it is highly unlikely that a flaky tests can be a "false positive". Assuming a randomness factor of x% the odds of hitting 10 hits and 10 fails in a round are pretty low.
Yet this WebGL tests manage to do that.
I suspect it can be related to something on the graphic driver state that once activated makes it fail continuously.
Example of the issue, see the GTK bot runs on: https://github.com/WebKit/WebKit/pull/14386
Carlos Alberto Lopez Perez
More WebGL2 tests were gardened on 264809@main