Bug 237372

<rdar://problem/89689914>

Created attachment 453618 [details] Results page It looks like the majority of these failures happen in tandem, which would suggest that this happens when either the simulator or test harness gets into a bad state. It also looks like these failures really started happening a couple of weeks ago, around 2/12..

(In reply to Wenson Hsieh from comment #2) > Created attachment 453618 [details] > Results page > > It looks like the majority of these failures happen in tandem, which would > suggest that this happens when either the simulator or test harness gets > into a bad state. > > It also looks like these failures really started happening a couple of weeks > ago, around 2/12.. I am seeing the exact same thing. Regression appears to be around r289700. But could be as you said due to the simulator being in a bad state.

(In reply to Matteo Flores from comment #3) > (In reply to Wenson Hsieh from comment #2) > > Created attachment 453618 [details] > > Results page > > > > It looks like the majority of these failures happen in tandem, which would > > suggest that this happens when either the simulator or test harness gets > > into a bad state. > > > > It also looks like these failures really started happening a couple of weeks > > ago, around 2/12.. > > I am seeing the exact same thing. > > Regression appears to be around r289700. But could be as you said due to the > simulator being in a bad state. No changes near r289700 seem like they could feasibly cause these failures. Did any major configuration changes to WebKit iOS simulator bots land around that time?

(In reply to Wenson Hsieh from comment #2) > Created attachment 453618 [details] > Results page > > It looks like the majority of these failures happen in tandem, which would > suggest that this happens when either the simulator or test harness gets > into a bad state. > > It also looks like these failures really started happening a couple of weeks > ago, around 2/12.. I am seeing the exact same thing. Regression appears to be around r289

(In reply to Matteo Flores from comment #5) > (In reply to Wenson Hsieh from comment #2) > > Created attachment 453618 [details] > > Results page > > > > It looks like the majority of these failures happen in tandem, which would > > suggest that this happens when either the simulator or test harness gets > > into a bad state. > > > > It also looks like these failures really started happening a couple of weeks > > ago, around 2/12.. > > I am seeing the exact same thing. > > Regression appears to be around r289 Disregard, duplicate message from page refresh... To answer your question, I have reached out to the team about possible configuration changes in that timeframe.

Marking expectations until issue is resolved.

Created attachment 453641 [details] expectations

Comment on attachment 453641 [details] expectations Clearing flags on attachment: 453641 Committed r290750 (?): <https://commits.webkit.org/r290750>

(In reply to Matteo Flores from comment #6) > Disregard, duplicate message from page refresh... > > To answer your question, I have reached out to the team about possible > configuration changes in that timeframe. I'm curious if you have any new info!

Created attachment 453753 [details] Apple-iOS-15-Simulator-Debug-WK2-Tests From examining recent runs in the Apple-iOS-15-Simulator-Debug-WK2-Tests queue, it looks like bot653 is uniquely encountering these mass pasteboard-related timeouts, and not bot654: bot653: https://build.webkit.org/#/workers/27 bot654: https://build.webkit.org/#/workers/75 I think there's a similar pattern on the other iOS sim testing queues. Is there any configuration difference between 653 and 654 that might help explain this? (perhaps a difference in SDK version, for instance?) For internal folks, this seems somewhat reminiscent of rdar://86327357, wherein the iTunes Store TCC prompt causes widespread layout test failures because the keyboard and context menus fail to appear.

Listing more layout tests that seem to suffer from this bug as well. editing/deleting/smart-delete-paragraph-001.html editing/deleting/smart-delete-paragraph-002.html editing/deleting/smart-delete-paragraph-004.html editing/deleting/smart-delete-paragraph-005.html editing/execCommand/insert-newline-in-quoted-content-crash.html editing/execCommand/outdent-cut-crash.html editing/execCommand/paste-as-quotation-disconnected-paragraph-ancestor-crash.html editing/execCommand/selectAll-copy-crash.html editing/pasteboard/19644-1.html editing/pasteboard/19644-2.html editing/pasteboard/3976872.html editing/pasteboard/4076267-2.html editing/pasteboard/4076267-3.html editing/pasteboard/4076267.html editing/pasteboard/4242293-1.html editing/pasteboard/4242293.html editing/pasteboard/4641033.html editing/pasteboard/4922709.html editing/pasteboard/4930986-1-paste-as-quotation.html editing/pasteboard/4930986-2-paste-as-quotation.html editing/pasteboard/4930986-3-paste-as-quotation.html editing/pasteboard/4944770-1.html editing/pasteboard/4944770-2.html editing/pasteboard/4989774.html editing/pasteboard/5006779.html editing/pasteboard/5028447.html editing/pasteboard/5065605.html editing/pasteboard/5761530-1.html editing/spelling/editing-word-with-marker-1.html editing/style/smoosh-styles-003.html editing/undo-manager/undo-manager-item-labels.html And these have a slightly different "regression" point but also appear related: editing/pasteboard/dom-paste/dom-paste-same-origin.html fast/css/border-radius-invalid-pair.html fast/events/before-input-prevent-paste.html fast/forms/ios/inputmode-removing-none.html fast/forms/plaintext-mode-2.html fast/images/image-copy-memory-usage.html fast/images/text-recognition/image-overlay-line-wrapping.html fast/table/table-split-on-insert-with-section-crash.html http/tests/security/clipboard/copy-paste-html-across-origin-sanitizes-html.html http/tests/security/clipboard/copy-paste-html-across-origin-strips-mso-list.html http/tests/security/clipboard/copy-paste-html-cross-origin-iframe-across-origin.html http/tests/security/clipboard/copy-paste-html-cross-origin-iframe-in-same-origin.html http/tests/security/clipboard/copy-paste-url-across-origin-sanitizes-url.html fast/editing/insert-paragraph-with-text-in-table-crash.html fast/editing/mutation-with-display-contents-container-crash.html fast/editing/paste-and-indent-microtask-crash.html fast/editing/paste-and-match-style-with-table-2-crash.html fast/events/before-input-prevent-cut.html fast/events/before-input-prevent-paste.html fast/events/input-events-paste-data.html fast/events/input-events-paste-rich-datatransfer.html editing/selection/ios/avoid-showing-callout-menu-over-controls.html editing/selection/ios/dispatch-mouse-events-when-modifying-selection-quirk.html editing/selection/ios/scrolling-with-fixed-selection-does-not-unselect-native-text-field.html editing/selection/ios/selection-after-changing-text-with-callout-menu.html editing/selection/ios/show-callout-bar-after-selecting-word.html editing/selection/ios/show-callout-bar-after-tap.html editing/selection/ios/update-selection-after-iframe-scroll.html editing/pasteboard/paste-without-nesting.html editing/selection/ios/update-selection-after-overflow-scroll.html editing/selection/programmatic-selection-on-mac-is-directionless.html editing/selection/replace-selection-crash-02.html editing/selection/select-crash-001.html editing/selection/select-crash-002.html editing/selection/setSelection-shadow-dom-crash.html Also slightly different regression point but appear related: editing/pasteboard/contenteditable-pre.html editing/pasteboard/copy-backslash-with-euc.html editing/pasteboard/copy-crash-with-extraneous-attribute.html editing/pasteboard/copy-cut-paste-events-fired-when-disabled.html editing/pasteboard/copy-display-none.html editing/pasteboard/copy-element-with-conflicting-background-color-from-rule.html editing/pasteboard/copy-image-with-alt-text.html editing/pasteboard/copy-in-password-field.html editing/pasteboard/copy-inside-h1-preserves-h1.html editing/pasteboard/copy-null-characters.html editing/pasteboard/copy-paste-across-shadow-boundaries-1.html editing/pasteboard/copy-paste-across-shadow-boundaries-2.html editing/pasteboard/copy-paste-across-shadow-boundaries-3.html editing/pasteboard/copy-paste-across-shadow-boundaries-4.html editing/pasteboard/copy-paste-across-shadow-boundaries-5.html

(In reply to Matteo Flores from comment #12) > Listing more layout tests that seem to suffer from this bug as well. > > editing/deleting/smart-delete-paragraph-001.html > editing/deleting/smart-delete-paragraph-002.html > editing/deleting/smart-delete-paragraph-004.html > editing/deleting/smart-delete-paragraph-005.html > editing/execCommand/insert-newline-in-quoted-content-crash.html > editing/execCommand/outdent-cut-crash.html > editing/execCommand/paste-as-quotation-disconnected-paragraph-ancestor-crash. > html > editing/execCommand/selectAll-copy-crash.html > editing/pasteboard/19644-1.html > editing/pasteboard/19644-2.html > editing/pasteboard/3976872.html > editing/pasteboard/4076267-2.html > editing/pasteboard/4076267-3.html > editing/pasteboard/4076267.html > editing/pasteboard/4242293-1.html > editing/pasteboard/4242293.html > editing/pasteboard/4641033.html > editing/pasteboard/4922709.html > editing/pasteboard/4930986-1-paste-as-quotation.html > editing/pasteboard/4930986-2-paste-as-quotation.html > editing/pasteboard/4930986-3-paste-as-quotation.html > editing/pasteboard/4944770-1.html > editing/pasteboard/4944770-2.html > editing/pasteboard/4989774.html > editing/pasteboard/5006779.html > editing/pasteboard/5028447.html > editing/pasteboard/5065605.html > editing/pasteboard/5761530-1.html > editing/spelling/editing-word-with-marker-1.html > editing/style/smoosh-styles-003.html > editing/undo-manager/undo-manager-item-labels.html > > And these have a slightly different "regression" point but also appear > related: > > editing/pasteboard/dom-paste/dom-paste-same-origin.html > fast/css/border-radius-invalid-pair.html > fast/events/before-input-prevent-paste.html > fast/forms/ios/inputmode-removing-none.html > fast/forms/plaintext-mode-2.html > fast/images/image-copy-memory-usage.html > fast/images/text-recognition/image-overlay-line-wrapping.html > fast/table/table-split-on-insert-with-section-crash.html > http/tests/security/clipboard/copy-paste-html-across-origin-sanitizes-html. > html > http/tests/security/clipboard/copy-paste-html-across-origin-strips-mso-list. > html > http/tests/security/clipboard/copy-paste-html-cross-origin-iframe-across- > origin.html > http/tests/security/clipboard/copy-paste-html-cross-origin-iframe-in-same- > origin.html > http/tests/security/clipboard/copy-paste-url-across-origin-sanitizes-url. > html > fast/editing/insert-paragraph-with-text-in-table-crash.html > fast/editing/mutation-with-display-contents-container-crash.html > fast/editing/paste-and-indent-microtask-crash.html > fast/editing/paste-and-match-style-with-table-2-crash.html > fast/events/before-input-prevent-cut.html > fast/events/before-input-prevent-paste.html > fast/events/input-events-paste-data.html > fast/events/input-events-paste-rich-datatransfer.html > editing/selection/ios/avoid-showing-callout-menu-over-controls.html > editing/selection/ios/dispatch-mouse-events-when-modifying-selection-quirk. > html > editing/selection/ios/scrolling-with-fixed-selection-does-not-unselect- > native-text-field.html > editing/selection/ios/selection-after-changing-text-with-callout-menu.html > editing/selection/ios/show-callout-bar-after-selecting-word.html > editing/selection/ios/show-callout-bar-after-tap.html > editing/selection/ios/update-selection-after-iframe-scroll.html > editing/pasteboard/paste-without-nesting.html > editing/selection/ios/update-selection-after-overflow-scroll.html > editing/selection/programmatic-selection-on-mac-is-directionless.html > editing/selection/replace-selection-crash-02.html > editing/selection/select-crash-001.html > editing/selection/select-crash-002.html > editing/selection/setSelection-shadow-dom-crash.html > > Also slightly different regression point but appear related: > > editing/pasteboard/contenteditable-pre.html > editing/pasteboard/copy-backslash-with-euc.html > editing/pasteboard/copy-crash-with-extraneous-attribute.html > editing/pasteboard/copy-cut-paste-events-fired-when-disabled.html > editing/pasteboard/copy-display-none.html > editing/pasteboard/copy-element-with-conflicting-background-color-from-rule. > html > editing/pasteboard/copy-image-with-alt-text.html > editing/pasteboard/copy-in-password-field.html > editing/pasteboard/copy-inside-h1-preserves-h1.html > editing/pasteboard/copy-null-characters.html > editing/pasteboard/copy-paste-across-shadow-boundaries-1.html > editing/pasteboard/copy-paste-across-shadow-boundaries-2.html > editing/pasteboard/copy-paste-across-shadow-boundaries-3.html > editing/pasteboard/copy-paste-across-shadow-boundaries-4.html > editing/pasteboard/copy-paste-across-shadow-boundaries-5.html We know that this is happening because the PasteboardAutomaticSync default either isn't being set properly when booting the simulator, or is being flipped on during the course of the layout test run. Writing this default should work around the problem on affected bots: ``` defaults write com.apple.iphonesimulator PasteboardAutomaticSync -bool NO ```

I think we should revert the expectations previously set in r290750, and work around these failures for the time being by manually writing the default on all affected test runners. There are likely too many affected tests to set expectations for across the board. To actually address this, we'll need to either figure out what is causing this default to be set after booting the simulator, or add a way to enforce this default to be set to NO when running layout tests.

It's passed on command line, so setting it in on-disk defaults seems unlikely to change the behavior. Tools/Scripts/webkitpy/xcode/simulated_device.py: SimulatedDeviceManager._managing_simulator_app = not host.executive.run_command(['open', '-g', '-b', SimulatedDeviceManager.simulator_bundle_id, '--args', '-PasteboardAutomaticSync', '0'], return_exit_code=True)

(In reply to Alexey Proskuryakov from comment #15) > It's passed on command line, so setting it in on-disk defaults seems > unlikely to change the behavior. > > Tools/Scripts/webkitpy/xcode/simulated_device.py: > SimulatedDeviceManager._managing_simulator_app = not > host.executive.run_command(['open', '-g', '-b', > SimulatedDeviceManager.simulator_bundle_id, '--args', > '-PasteboardAutomaticSync', '0'], return_exit_code=True) My understanding was that `--args PasteboardAutomaticSync 0` would only affect the `Managed X` simulators that the test runner boots up. For some reason, this seems to (sometimes) be ineffective in overriding the default PasteboardAutomaticSync behavior. Perhaps we might see different results by simply writing the default globally.

Seems like an easy enough check to try it out, to isolate possibilities.

(In reply to Alexey Proskuryakov from comment #17) > Seems like an easy enough check to try it out, to isolate possibilities. Indeed! (And even if it doesn't work, that will be an interesting datapoint in this investigation).

(In reply to Wenson Hsieh from comment #18) > (In reply to Alexey Proskuryakov from comment #17) > > Seems like an easy enough check to try it out, to isolate possibilities. > > Indeed! (And even if it doesn't work, that will be an interesting datapoint > in this investigation). I ssh'd into bot651 and bot664 (both of which are consistently hitting these timeouts), and wrote the default. Let's see if it makes any difference... <https://build.webkit.org/#/workers/98> <https://build.webkit.org/#/workers/11>

Pull request: https://github.com/WebKit/WebKit/pull/149

Pull request: https://github.com/WebKit/WebKit/pull/191

(In reply to Jonathan Bedard from comment #20) > Pull request: https://github.com/WebKit/WebKit/pull/149 Wrong PR, mistakingly proposed this on an existing branch. https://github.com/WebKit/WebKit/pull/191 is the right PR.

Landed https://commits.webkit.org/248136@main (r290905)!

Reverted r290750 for reason: Bug has been fixed, expectations are no longer needed. Committed r291066 (248238@trunk): <https://commits.webkit.org/248238@trunk>