Bug 207197

Summary: Regression(?): [ iOS wk2 ] http/tests/resourceLoadStatistics/cookie-deletion.html is flaky failing and crashing.
Product: WebKit Reporter: Jason Lawrence <Lawrence.j>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED CONFIGURATION CHANGED    
Severity: Normal CC: tsavell, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Update Test Expectations none

Description Jason Lawrence 2020-02-04 07:28:45 PST 
http/tests/resourceLoadStatistics/cookie-deletion.html

Description:
This test is flaky failing and crashing on iOS wk2.

History:
https://results.webkit.org/?platform=ios&suite=layout-tests&test=http%2Ftests%2FresourceLoadStatistics%2Fcookie-deletion.html&limit=50000

Diff:
--- /Volumes/Data/slave/ipados-simulator-13-release-tests-wk2/build/layout-test-results/http/tests/resourceLoadStatistics/cookie-deletion-expected.txt
+++ /Volumes/Data/slave/ipados-simulator-13-release-tests-wk2/build/layout-test-results/http/tests/resourceLoadStatistics/cookie-deletion-actual.txt
@@ -1,73 +1,5 @@
 CONSOLE MESSAGE: line 117: Test is beginning. document.location.hash is empty.
 CONSOLE MESSAGE: line 123: About to call runTest() for the first time.
 CONSOLE MESSAGE: line 62: step1. About to set a cookie
-CONSOLE MESSAGE: line 127: About to call runTest() after cookie was set. document hash is #step2
-CONSOLE MESSAGE: line 67: step2. About to open an iFrame to test for third party cookie access (should be successful)
-CONSOLE MESSAGE: line 73: step3. About to classify localhost as prevalent
-CONSOLE MESSAGE: line 77: step3. In the callback for testRunner.setStatisticsPrevalentResource
-CONSOLE MESSAGE: line 84: step4. About to open an iFrame to test for third party cookie access (should not be successful)
-CONSOLE MESSAGE: line 90: step5. About to open an iFrame to try to set a cookie as a third party (should fail)
-CONSOLE MESSAGE: line 96: step6. About to open an iFrame and fireDataModificationHandlerAndContinue
-CONSOLE MESSAGE: line 40: In fireDataModificationHandlerAndContinue
-CONSOLE MESSAGE: line 45: Calling statisticsProcessStatisticsAndDataRecords
-CONSOLE MESSAGE: line 42: In callback function for installStatisticsDidScanDataRecordsCallback
-CONSOLE MESSAGE: line 102: step7. About to open an iFrame and setAsNonPrevalentAndContinue
-CONSOLE MESSAGE: line 50: In setAsNonPrevalentAndContinue
-CONSOLE MESSAGE: line 52: In callback function for setStatisticsPrevalentResource
-CONSOLE MESSAGE: line 108: step8. About to open an iFrame and confirm third party has no cookie access
-CONSOLE MESSAGE: line 25: in finishTest
-Test for partitioned and unpartitioned cookie deletion.
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-PASS successfullyParsed is true
-
-TEST COMPLETE
-  
-
---------
-Frame: '<!--frame1-->'
---------
-Should receive first-party cookie.
-Received cookie named 'firstPartyCookie'.
-Did not receive cookie named 'thirdPartyCookie'.
-Client-side document.cookie: firstPartyCookie=value
-
---------
-Frame: '<!--frame2-->'
---------
-Should receive no cookies.
-Did not receive cookie named 'firstPartyCookie'.
-Did not receive cookie named 'thirdPartyCookie'.
-Client-side document.cookie:
-
---------
-Frame: '<!--frame3-->'
---------
-Try to set third party cookie.
-
-
---------
-Frame: '<!--frame4-->'
---------
-Should receive no cookies.
-Did not receive cookie named 'firstPartyCookie'.
-Did not receive cookie named 'thirdPartyCookie'.
-Client-side document.cookie:
-
---------
-Frame: '<!--frame5-->'
---------
-After removal, should receive no cookies.
-Did not receive cookie named 'firstPartyCookie'.
-Did not receive cookie named 'thirdPartyCookie'.
-Client-side document.cookie:
-
---------
-Frame: '<!--frame6-->'
---------
-After declassified, should receive no cookies.
-Did not receive cookie named 'firstPartyCookie'.
-Did not receive cookie named 'thirdPartyCookie'.
-Client-side document.cookie:
Comment 1 Radar WebKit Bug Importer 2020-02-04 07:29:05 PST 
<rdar://problem/59148643>
Comment 2 Jason Lawrence 2020-02-04 07:46:18 PST 
I am not able to reproduce this issue with run-webkit-tests after 2500 iterations.
Comment 3 Jason Lawrence 2020-02-04 08:01:27 PST 
Created attachment 389660 [details]
Update Test Expectations
Comment 4 Truitt Savell 2020-02-04 08:09:04 PST 
Comment on attachment 389660 [details]
Update Test Expectations

Clearing flags on attachment: 389660

Committed r255676: <https://trac.webkit.org/changeset/255676>
Comment 5 Jason Lawrence 2020-02-04 08:09:52 PST 
This appears to only be flaky crashing in debug. It is flaky failing otherwise.
Comment 6 Alexey Proskuryakov 2020-02-04 11:52:03 PST 
I see failures in mid-September (from 11th to 19th), then a long green streak, and then failures, timeouts and assertion failures begin on November 6th. Hard to tell how real this is, but results definitely got worse.

The assertion failure is this, which we've been seeing for a long time on other tests:

ASSERTION FAILED: m_uncommittedState.state == State::Committed
/Volumes/Data/slave/ios-simulator-13-debug/build/Source/WebKit/UIProcess/PageLoadState.cpp(322) : void WebKit::PageLoadState::didFinishLoad(const Transaction::Token &)
1   0x10c353309 WTFCrash
2   0x11ab4da6b WTFCrashWithInfo(int, char const*, char const*, int)
3   0x11b59b140 WebKit::PageLoadState::didFinishLoad(WebKit::PageLoadState::Transaction::Token const&)
4   0x11b6fe4b4 WebKit::WebPageProxy::didFinishLoadForFrame(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&)
5   0x11c1e5782 void IPC::callMemberFunctionImpl<WebKit::WebPageProxy, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&), std::__1::tuple<WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData>, 0ul, 1ul, 2ul>(WebKit::WebPageProxy*, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&), std::__1::tuple<WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData>&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul>)
6   0x11c1e4ab0 void IPC::callMemberFunction<WebKit::WebPageProxy, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&), std::__1::tuple<WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData>, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul> >(std::__1::tuple<WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData>&&, WebKit::WebPageProxy*, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&))
7   0x11c1b30bd void IPC::handleMessage<Messages::WebPageProxy::DidFinishLoadForFrame, WebKit::WebPageProxy, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&)>(IPC::Decoder&, WebKit::WebPageProxy*, void (WebKit::WebPageProxy::*)(WTF::ObjectIdentifier<WebCore::FrameIdentifierType>, unsigned long long, WebKit::UserData const&))
Comment 7 Jason Lawrence 2020-05-26 15:03:12 PDT 
I have removed the expectation because the test appears to be passing consistently now.
https://trac.webkit.org/changeset/262164/webkit