Bug 221218

Summary: [ BigSur ] imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html is flakey failing
Product: WebKit Reporter: Robert Jenner <jenner>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED WORKSFORME    
Severity: Normal CC: ben_schwartz, jer.noble, tsavell, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch
none
Patch
none
Update TestExpectations none

Robert Jenner
Reported 2021-02-01 11:12:49 PST
imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html Is flakey failing on MacOS Big Sur Wk2. HISTORY URL: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-config-change-webm-v-framesize.html DIFF URL: https://build.webkit.org/results/Apple-BigSur-Release-WK2-Tests/r272143%20(1311)/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-diff.txt DIFF TEXT: --- /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-expected.txt +++ /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-actual.txt @@ -1,3 +1,5 @@ -PASS Tests webm video-only frame size changes. +Harness Error (TIMEOUT), message = null +TIMEOUT Tests webm video-only frame size changes. Test timed out Bug Reproducible on r272141, same failure verified. Bug Produced using test: run-webkit-test imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html --iterations 1000 -f Regression point appears r272140, where the test did not crash. Bug appears to have been introduced in r272141
Attachments
Patch (1.52 KB, patch)
2021-02-11 12:18 PST, Robert Jenner 		no flags
DetailsFormatted DiffDiff
Patch (1.54 KB, patch)
2021-02-11 15:59 PST, Robert Jenner 		no flags
DetailsFormatted DiffDiff
Patch (1.51 KB, patch)
2021-02-11 16:26 PST, Robert Jenner 		no flags
DetailsFormatted DiffDiff
Update TestExpectations (1.53 KB, patch)
2021-02-17 16:20 PST, Robert Jenner 		no flags
DetailsFormatted DiffDiff
Show Obsolete (4) View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2021-02-01 11:13:15 PST
<rdar://problem/73837300>
Robert Jenner
Comment 2 2021-02-01 14:23:27 PST
We are unable to find a true regression point. This test appears to be very flakey. Tested back to r271039 just to see, and it did fail.
Robert Jenner
Comment 3 2021-02-11 12:18:54 PST
Created attachment 420026 [details] Patch
Ryan Haddad
Comment 4 2021-02-11 12:53:19 PST
(In reply to Robert Jenner from comment #2) > We are unable to find a true regression point. This test appears to be very > flakey. > > Tested back to r271039 just to see, and it did fail. I think this test has just been flaky since it was unskipped as part of https://trac.webkit.org/changeset/270720/webkit
Ryan Haddad
Comment 5 2021-02-11 12:56:07 PST
Comment on attachment 420026 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=420026&action=review > LayoutTests/platform/mac-wk2/TestExpectations:1329 > +webkit.org/b/221218 [ BigSur ] imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html [ Pass Failure ] Let's mark this BigSur+, since presumably it wold affect future releases too. > LayoutTests/platform/mac-wk2/TestExpectations:1331 > + Nit: extra newline here
Robert Jenner
Comment 6 2021-02-11 15:59:50 PST
Created attachment 420059 [details] Patch
Robert Jenner
Comment 7 2021-02-11 16:04:12 PST
(In reply to Ryan Haddad from comment #5) > Comment on attachment 420026 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=420026&action=review > > > LayoutTests/platform/mac-wk2/TestExpectations:1329 > > +webkit.org/b/221218 [ BigSur ] imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html [ Pass Failure ] > > Let's mark this BigSur+, since presumably it wold affect future releases too. > > > LayoutTests/platform/mac-wk2/TestExpectations:1331 > > + > > Nit: extra newline here Changes made, how does it look now?
Robert Jenner
Comment 8 2021-02-11 16:26:50 PST
Created attachment 420062 [details] Patch
Robert Jenner
Comment 9 2021-02-17 16:13:49 PST
It does appear to be flakey failing in wk1 as well. Updating title and patch.
Robert Jenner
Comment 10 2021-02-17 16:20:37 PST
Created attachment 420754 [details] Update TestExpectations
Truitt Savell
Comment 11 2021-02-17 16:22:34 PST
Comment on attachment 420754 [details] Update TestExpectations Clearing flags on attachment: 420754 Committed r273045 (234246@main): <https://commits.webkit.org/234246@main>
Ryan Haddad
Comment 12 2021-03-22 14:48:21 PDT
The test is consistently failing on Big Sur WK1 with the following diff: -PASS Tests webm video-only frame size changes. +FAIL Tests webm video-only frame size changes. assert_true: video/webm;codecs="vp8" is supported. expected true got false perhaps that is expected, though. I don't see any failures on Big Sur WK2 since 02/09/21: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-config-change-webm-v-framesize.html&flavor=wk2&platform=mac We may be able to remove the expectation for WK2 and close this one out.
Ben Schwartz
Comment 13 2024-06-03 16:58:56 PDT
This no longer appears to be occurring. Removing test expectations. Please file a new bug if issues persist. Tracking test expectations removal in bug #274990 (rdar://129080660).
Note You need to log in before you can comment on or make changes to this bug.