At least in this example, it looks like the ACTUAL result is correct, and the EXPECTED result is incorrect for both tests: <https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK1%20(Tests)/r197819%20(4188)/results.html>

The tests are super flaky, failing around half of the time. Dan, could you please take a look? There was one failure on Yosemite too, so this may not be El Capitan specific.

Dan should be able to look at this shortly. Skipped tests to reduce noise in <http://trac.webkit.org/projects/webkit/changeset/197890>

Skipped test LayoutTests/media/video-with-file-url-allowed-by-csp-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html (added in bug #157100) in <https://trac.webkit.org/r200180>

(In reply to comment #4) > Skipped test > LayoutTests/media/video-with-file-url-allowed-by-csp-media-src-star-with- > AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html (added > in bug #157100) in <https://trac.webkit.org/r200180> It appears that this test is also flaky on WK2 <http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fvideo-with-file-url-allowed-by-csp-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html>

Moved skip(In reply to comment #5) > It appears that this test is also flaky on WK2 > <http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard. > html#showAllRuns=true&tests=media%2Fvideo-with-file-url-allowed-by-csp-media- > src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled. > html> Moved skip to platform/mac/TestExpectations in <http://trac.webkit.org/projects/webkit/changeset/200199>

<rdar://problem/26846436>

Also skipped security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star.html in https://trac.webkit.org/r202180

Updated test expectations in <http://trac.webkit.org/r202235>. Changed from skipping to flaky, because there is no reason to not run these tests, and it's good to have results on the dashboard. Also added one more test from this group, which was making bots red.

Skipped security/contentSecurityPolicy/video-with-blob-url-allowed-by-media-src-star.html on WebKit1 only again, because is times out every time.

security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html appears to be flaky on iOS Simulator as well, with the same diff.

The same issue happens on GTK and WPE. https://build.webkit.org/results/WPE-Linux-64-bit-Release-Tests/r275404%20(975)/security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-diffs.html Updated expectations on r275423

security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star.html is flaky on iOS-14-Simulator-WK2-Tests-EWS and the flaky failure is seen on the open source directory. security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html mentioned above seems to be still flaky on the open source. History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=security%2FcontentSecurityPolicy%2Fvideo-with-file-url-allowed-by-media-src-star.html&test=security%2FcontentSecurityPolicy%2Fvideo-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html Marked expectations for those two tests here: https://trac.webkit.org/changeset/281566/webkit

For security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html, the same images failure is seen on iOS 15 Debug and continue seeing on all other platforms. Updated test expectations : https://trac.webkit.org/changeset/283730/webkit