Bug 237552

Summary: [iOS Mac ] imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash.html is a flaky text failure
Product: WebKit Reporter: Dawn Morningstar <Morningstar>
Component: New BugsAssignee: Karl Rackler <rackler>
Status: NEW ---    
Severity: Normal CC: beidson, cdumez, rackler, simon.fraser, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=239571

Description Dawn Morningstar 2022-03-07 13:29:21 PST 
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash.html 
Is a flaky text failure on iOS and wk1 queues since the test was introduced.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fbrowsers%2Fhistory%2Fthe-location-interface%2Fsame-hash.html

DIFF:
--- /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-actual.txt
@@ -1,6 +1,6 @@
 
 
-PASS Using location.hash = "#te<st" must not reset scroll position
+FAIL Using location.hash = "#te<st" must not reset scroll position assert_greater_than: First hash assignment scrolls the iframe expected a number greater than 150 but got 0
 PASS Using location.hash = "te<st" must not reset scroll position
 PASS Using location.hash = "#te%3Cst" must not reset scroll position
 PASS Using location.hash = "te%3Cst" must not reset scroll position

DIFF-URL:
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/r290885%20(1859)/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-diff.txt
Comment 1 Radar WebKit Bug Importer 2022-03-07 13:30:34 PST 
<rdar://problem/89926580>
Comment 2 Dawn Morningstar 2022-03-15 12:33:36 PDT 
Correction, this is not solely happening on iOS queues.
Comment 3 Dawn Morningstar 2022-03-15 12:42:07 PDT 
Marking expectations until further investigation can occur.
Comment 4 Chris Dumez 2022-03-15 12:44:43 PDT 
Not truly a regression, just a WPT test update from upstream.

Looking at the output, this likely has to do with some scrolling happening asynchronously when the test expects it not too. As a result, when the test checks the scroll position, sometimes it has scrolled and sometimes it hasn't.
Comment 5 Dawn Morningstar 2022-03-15 12:53:30 PDT 
(In reply to Chris Dumez from comment #4)
> Not truly a regression, just a WPT test update from upstream.
> 
> Looking at the output, this likely has to do with some scrolling happening
> asynchronously when the test expects it not too. As a result, when the test
> checks the scroll position, sometimes it has scrolled and sometimes it
> hasn't.

Ah I see, incorrect starting position, this would mostly be a test order issue then? Or is modifying the test to reset starting position possible?
Comment 6 Chris Dumez 2022-03-15 12:57:31 PDT 
(In reply to Matteo Flores from comment #5)
> (In reply to Chris Dumez from comment #4)
> > Not truly a regression, just a WPT test update from upstream.
> > 
> > Looking at the output, this likely has to do with some scrolling happening
> > asynchronously when the test expects it not too. As a result, when the test
> > checks the scroll position, sometimes it has scrolled and sometimes it
> > hasn't.
> 
> Ah I see, incorrect starting position, this would mostly be a test order
> issue then? Or is modifying the test to reset starting position possible?

Either the test needs to be modified to not wait a little bit for the scrolling position to get updated (which would mean an upstream PR since this is an imported W3C/WPT test, not a WebKit test). Or we really do have a bug and the scroll position is supposed to get updated synchronously here (which would be a WebKit bug but likely not a recent regression).
Comment 7 Karl Rackler 2022-05-04 13:30:38 PDT 
I have marked this test as a flaky failure while this issue is investigated.
Comment 8 Karl Rackler 2022-05-04 13:34:32 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/501
Comment 9 EWS 2022-05-04 13:37:28 PDT 
Test gardening commit r293787 (250266@main): <https://commits.webkit.org/250266@main>

Reviewed commits have been landed. Closing PR #501 and removing active labels.