Bug 244162

Summary: REGRESSION (21E230-21E258): [ Monterey wk2 Release ] 76 accessibility tests are a consistent failure
Product: WebKit Reporter: Karl Rackler <rackler>
Component: AccessibilityAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: andresg_22, ap, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
test list none

Description Karl Rackler 2022-08-20 12:30:07 PDT 
Description:
76 accessibility tests ( test list.txt )  

The first failure I saw on the dashboard was approximately 6/10/2022 at 251461@main when the bot ( https://build.webkit.org/#/builders/562 ) ran 21E258, and the tests are consistently failing versus the consistently passing bot ( https://build.webkit.org/#/builders/365 ) running 21E230.  This is happening on Monterey wk2 Release x86_64 bots.

REPRODUCTION STEPS
Failing bot 21E258
https://build.webkit.org/#/builders/562/builds/719/steps/11/logs/stdio

Passing bot 21E230
https://build.webkit.org/#/builders/365/builds/4917/steps/12/logs/stdio

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=accessibility%2Faria-controls-with-tabs.html&test=accessibility%2Faria-current.html&test=accessibility%2Faria-labelledby-overrides-aria-label.html&test=accessibility%2Faria-liveregions-attributes.html&test=accessibility%2Faria-roledescription.html&test=accessibility%2Faria-roles.html&test=accessibility%2Faria-scrollbar-role.html&test=accessibility%2Faria-selected-menu-items.html&test=accessibility%2Fcontent-editable-as-textarea.html&test=accessibility%2Fdialog-showModal.html&test=accessibility%2Fdynamically-ignored-canvas.html&test=accessibility%2Fframe-disconnect-textmarker-cache-crash.html&test=accessibility%2Fhelp-text.html&test=accessibility%2Fkeyevents-for-actions-mimic-real-key-events.html&test=accessibility%2Fkeyevents-posted-for-increment-actions.html&test=accessibility%2Flabel-with-pseudo-elements.html&test=accessibility%2Flist-detection2.html&test=accessibility%2Fmac%2Farea-with-aria-label.html&test=accessibility%2Fmac%2Faria-hidden-changes-for-non-ignored-elements.html&test=accessibility%2Fmac%2Faria-menuitem-focus.html&test=accessibility%2Fmac%2Faria-multiselectable.html&test=accessibility%2Fmac%2Faria-tree-activedescendant.html&test=accessibility%2Fmac%2Faria-tree.html&test=accessibility%2Fmac%2Fcaret-browsing-arrow-nav.html&test=accessibility%2Fmac%2Fcaret-browsing-attribute.html&test=accessibility%2Fmac%2Fcaret-browsing-tab-selection.html&test=accessibility%2Fmac%2Fcombobox-activedescendant-notifications.html&test=accessibility%2Fmac%2Fcrash-in-element-for-text-marker.html&test=accessibility%2Fmac%2Fcrash-invalid-text-marker-node.html&test=accessibility%2Fmac%2Fdocument-attributes.html&test=accessibility%2Fmac%2Fdocument-title-used-for-description.html&test=accessibility%2Fmac%2Feditable-webarea-context-menu-point.html&test=accessibility%2Fmac%2Ffigure-element.html&test=accessibility%2Fmac%2Ffocus-setting-selection-syncronizing-not-clearing.html&test=accessibility%2Fmac%2Fframe-with-title.html&test=accessibility%2Fmac%2Fhtml-with-aria-label.html&test=accessibility%2Fmac%2Finput-slider.html&test=accessibility%2Fmac%2Finternal-link-when-document-has-fragment.html&test=accessibility%2Fmac%2Fnative-vs-nonnative-checkboxes.html&test=accessibility%2Fmac%2Foption-with-arialabel.html&test=accessibility%2Fmac%2Fpress-action-not-exposed-for-event-delegation.html&test=accessibility%2Fmac%2Fprevent-keyboard-event-dispatch.html&test=accessibility%2Fmac%2Fprimary-screen-height.html&test=accessibility%2Fmac%2Fpseudo-element-text-markers.html&test=accessibility%2Fmac%2Fradio-button-checkbox-size.html&test=accessibility%2Fmac%2Fruby-hierarchy-roles.html&platform=Mac

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=accessibility%2Fmac%2Fsearch-predicate-from-ignored-element.html&test=accessibility%2Fmac%2Fsearch-predicate-immediate-descendants-only.html&test=accessibility%2Fmac%2Fsearch-predicate.html&test=accessibility%2Fmac%2Fselection-value-changes-for-aria-textbox.html&test=accessibility%2Fmac%2Fset-value-editable-types.html&test=accessibility%2Fmac%2Fsetting-attributes-is-asynchronous.html&test=accessibility%2Fmac%2Ftest-convenience-methods.html&test=accessibility%2Fmac%2Ftext-marker-paragraph-nav.html&test=accessibility%2Fmac%2Ftext-marker-sentence-nav.html&test=accessibility%2Fmac%2Ftext-marker-word-nav.html&test=accessibility%2Fmac%2Ftextmarker-routines.html&test=accessibility%2Fmac%2Ftitle-attribute-used-as-last-resort.html&test=accessibility%2Fscroll-to-global-point-iframe-nested.html&test=accessibility%2Fscroll-to-global-point-iframe.html&test=accessibility%2Fscroll-to-global-point-main-window.html&test=accessibility%2Fscroll-to-global-point-nested.html&test=accessibility%2Fscroll-to-make-visible-div-overflow.html&test=accessibility%2Fscroll-to-make-visible-iframe-offscreen.html&test=accessibility%2Fscroll-to-make-visible-iframe.html&test=accessibility%2Fscroll-to-make-visible-nested-2.html&test=accessibility%2Fscroll-to-make-visible-nested.html&test=accessibility%2Fscroll-to-make-visible-with-subfocus.html&test=accessibility%2Fselect-element-at-index.html&test=accessibility%2Fset-value-not-work-for-disabled-sliders.html&test=accessibility%2Fslider-with-lost-renderer.html&test=accessibility%2Fsvg-remote-element.html&test=accessibility%2Ftext-alternative-calculation-from-listbox.html&test=accessibility%2Ftext-alternative-calculation-hidden-nodes.html&test=accessibility%2Ftextarea-insertion-point-line-number.html&test=accessibility%2Ftitle-ui-element-correctness.html&platform=mac

Diff:
--- /Volumes/Data/worker/Apple-Monterey-Release-WK2-Accessibility-Isolated-Tree-Tests/build/layout-test-results/accessibility/aria-controls-with-tabs-expected.txt
+++ /Volumes/Data/worker/Apple-Monterey-Release-WK2-Accessibility-Isolated-Tree-Tests/build/layout-test-results/accessibility/aria-controls-with-tabs-actual.txt
@@ -11,10 +11,10 @@
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-PASS tab2.isSelected is true
+FAIL tab2.isSelected should be true. Was false.
 PASS tab1.isSelected is false
 PASS tab2.isSelected is false
-PASS tab1.isSelected is true
+FAIL tab1.isSelected should be true. Was false.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Radar WebKit Bug Importer 2022-08-20 12:30:27 PDT 
<rdar://problem/98929884>
Comment 2 Karl Rackler 2022-08-20 12:44:17 PDT 
Created attachment 461770 [details]
test list
Comment 3 Karl Rackler 2022-08-20 12:44:45 PDT 
I have marked this test as a flaky failure while this issue is investigated.
Comment 4 EWS 2022-08-20 12:58:41 PDT 
Test gardening commit 253622@main (5a67bbeb7234): <https://commits.webkit.org/253622@main>

Reviewed commits have been landed. Closing PR #3507 and removing active labels.