Bug 241991

Summary:	REGRESSION (251792@main?): [ macOS ][ iOS ] TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture is a flaky failure and crash
Product:	WebKit	Reporter:	Karl Rackler <rackler>
Component:	New Bugs	Assignee:	Chris Dumez <cdumez>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	cdumez, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=241932

Description Karl Rackler 2022-06-24 16:11:57 PDT

Description:
TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture

The first failure that I saw on the dashboard was on 6/23/2022 at 251792@main.  The test is flaky crashing on iOS arm64 and failing on iOS X86_64.  The flaky crashing on iOS arm64 may be bot related because other test are failing similarly. The test is flaky failing on macOS. 
The flakiness dashboard indicates that 251792@main may be the commit causing the issue.  There is a good possibility that the test landed this way and the bots did not show the results until later on the timeline.

REPRODUCTION STEPS
I can reproduce this on 251793@main, but unable to reproduce it on 251789@main or earlier but, I was not able to find a commit that seemed relevant; due to the lack of ability to test commits in that range, I cannot pinpoint what change could have caused this.  I was also able to reproduce on 251793@main and ran again test again and it did not reproduce, which adds to the mystery.

Command: 
run-api-tests --debug --iterations 200 TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture

Result: 
Failed
    TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:405
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html"
          url1.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html"
        
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:410
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a"
          url2.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html"
        
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:415
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html"
          lastURL.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a"

History:
https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture&limit=50000

run-api-tests stdio: 
Failed
    TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:405
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html"
          url1.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html"
        
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:410
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a"
          url2.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html"
        
        
        /Volumes/Data/worker/Apple-BigSur-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKit/WKBackForwardList.mm:415
        Expected equality of these values:
          [webView URL].absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple.html"
          lastURL.absoluteString.UTF8String
            Which is: "file:///Volumes/Data/worker/Apple-BigSur-Release-AppleSilicon-WK1-Tests/build/WebKitBuild/Release/TestWebKitAPI.resources/simple2.html#a"

iOS arm64 results: 
https://build.webkit.org/#/builders/529/builds/108
    TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture
        An error was encountered processing the command (domain=NSMachErrorDomain, code=24):
        The operation couldn’t be completed. (Mach error 24 - (os/kern) memory data moved)

Comment 1 Radar WebKit Bug Importer 2022-06-24 16:12:19 PDT

<rdar://problem/95886849>

Comment 2 Chris Dumez 2022-06-24 16:18:20 PDT

Hmm, `run-api-tests --debug --no-build --iterations 200 TestWebKitAPI.WKBackForwardList.BackForwardNavigationDoesNotSkipItemsWithUserGesture` didn't reproduce for me.

This test was modified in https://commits.webkit.org/251837@main. I wonder if 251837@main fixed it or if I'm just unable to reproduce due to timing issues.

I guess the bots will tell me.

Comment 3 Chris Dumez 2022-06-24 16:25:57 PDT

Ok, I managed to reproduce on an M1 machine. I am investigating. Odds are that the test was already flaky when introduced.

Comment 4 Chris Dumez 2022-06-24 16:56:24 PDT

Pull request: https://github.com/WebKit/WebKit/pull/1789

Comment 5 EWS 2022-06-27 16:36:07 PDT

Committed 251894@main (2a7a51012dfc): <https://commits.webkit.org/251894@main>

Reviewed commits have been landed. Closing PR #1789 and removing active labels.