Bug 249417

Summary:	[ Ventura EWS ] Garden Ventura EWS only failures
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED CONFIGURATION CHANGED
Severity:	Normal	CC:	webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Robert Jenner 2022-12-15 11:22:07 PST

We are seeing quite a few failures on Ventura EWS that are only occurring on EWS and not on post-commit testers. The tests I have found so far are:


http/tests/media/fairplay/fps-hls-update-reject.html
http/tests/media/fairplay/fps-hls.html
http/tests/media/fairplay/fps-init-data-skd.html
http/tests/media/fairplay/fps-mse-unmuxed-key-renewal.html
http/tests/workers/service/getnotifications-stop.html
http/tests/workers/service/getnotifications.html
http/tests/workers/service/shownotification-allowed-document.html
http/tests/workers/service/shownotification-allowed.html
http/tests/workers/service/shownotification-invalid-data.html
http/wpt/push-api/pushEvent.any.serviceworker.html
imported/w3c/web-platform-tests/notifications/idlharness.https.any.html
imported/w3c/web-platform-tests/notifications/idlharness.https.any.serviceworker.html
imported/w3c/web-platform-tests/notifications/idlharness.https.any.sharedworker.html
imported/w3c/web-platform-tests/notifications/idlharness.https.any.worker.html

It's a combination of Failing and timing out as observed by the results page here:

https://ews-build.s3-us-west-2.amazonaws.com/macOS-AppleSilicon-Ventura-Debug-WK2-Tests-EWS/44c05b08-225-clean-tree/results.html

We can tell that they're not occurring on post-commit bots with the following history link:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-hls.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-init-data-skd.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-key-renewal.html&test=http%2Ftests%2Fworkers%2Fservice%2Fgetnotifications-stop.html&test=http%2Ftests%2Fworkers%2Fservice%2Fgetnotifications.html&test=http%2Ftests%2Fworkers%2Fservice%2Fshownotification-allowed-document.html&test=http%2Ftests%2Fworkers%2Fservice%2Fshownotification-allowed.html&test=http%2Ftests%2Fworkers%2Fservice%2Fshownotification-invalid-data.html&test=http%2Fwpt%2Fpush-api%2FpushEvent.any.serviceworker.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fnotifications%2Fidlharness.https.any.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fnotifications%2Fidlharness.https.any.serviceworker.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fnotifications%2Fidlharness.https.any.sharedworker.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fnotifications%2Fidlharness.https.any.worker.html&test=media%2FaudioSession%2FgetUserMedia.html

Comment 1 Radar WebKit Bug Importer 2022-12-15 11:22:26 PST

<rdar://problem/103411154>

Comment 2 Robert Jenner 2022-12-15 11:40:06 PST

It appears that this may be a test-list ordering issue. For the time being I am going to mark expectations for the above tests only for Ventura Apple Silicon WK2 arm64 below:

Comment 3 EWS 2022-12-15 11:56:28 PST

Test gardening commit 257949@main (c9019ce76e4b): <https://commits.webkit.org/257949@main>

Reviewed commits have been landed. Closing PR #7712 and removing active labels.

Comment 4 EWS 2023-01-10 16:01:36 PST

Test gardening commit 258751@main (7e9151364f67): <https://commits.webkit.org/258751@main>

Reviewed commits have been landed. Closing PR #8491 and removing active labels.

Comment 5 Robert Jenner 2023-01-10 16:02:40 PST

From what I could tell this was no longer happening. It may have been related to us having the wrong Xcode/Mac SDK on the EWS bots that has since been corrected. I have removed the expectations above. If it starts happening again we will need to re-file.