Bug 206834

Summary:

[ Macos iOS ] http/tests/workers/service/basic-timeout.https.html is flaky failing

Product:

WebKit

Reporter:

Jacob Uphoff <jacob_uphoff>

Component:

Tools / Tests

Assignee:

Nobody <webkit-unassigned>

Status:

NEW ---

Severity:

Normal

CC:

Hironori.Fujii, tsavell, webkit-bot-watchers-bugzilla, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Attachments:

Description	Flags
Update TestExpectations	none

Description Jacob Uphoff 2020-01-27 10:28:38 PST

http/tests/workers/service/basic-timeout.https.html is flakey failing on macOS and iOS 

I was able to reproduce the failure by running the test in iterations x5000, it reproduced multiple times.

The test has been flaky for the whole visible history, so no regression point.

History:
https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fworkers%2Fservice%2Fbasic-timeout.https.html&platform=ios&platform=mac

Diff:

--- /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/tests/workers/service/basic-timeout.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/http/tests/workers/service/basic-timeout.https-actual.txt
@@ -2,9 +2,9 @@
 URL 0 - https://127.0.0.1:8443/workers/service/resources/timeout-fallback.html
 Status code 0 - 200
 Source' header 0 - null
-URL 1 - https://127.0.0.1:8443/workers/service/resources/timeout-no-fallback.html
-Status code 1 - 404
-Source' header 1 - null
+URL 1 - undefined
+Status code 1 - undefined
+Source' header 1 - undefined
 URL 2 - https://127.0.0.1:8443/workers/service/resources/succeed-fallback-check.php
 Status code 2 - 200
 Source' header 2 - network

Comment 1 Radar WebKit Bug Importer 2020-01-27 10:31:51 PST

<rdar://problem/58923921>

Comment 2 Jacob Uphoff 2020-01-27 13:09:19 PST

Created attachment 388887 [details]
Update TestExpectations

Comment 3 Truitt Savell 2020-01-27 13:25:16 PST

Comment on attachment 388887 [details]
Update TestExpectations

Clearing flags on attachment: 388887

Committed r255164: <https://trac.webkit.org/changeset/255164>

Comment 4 Fujii Hironori 2024-01-04 18:41:16 PST

It is still a flaky failure.
253540@main changed the test. There are current failure diffs.

Buildbot: builder Apple-Ventura-Release-AppleSilicon-WK2-Tests build 6816 : 272652@main
https://build.webkit.org/#/builders/699/builds/6816

https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK2-Tests/272652@main%20(6816)/http/tests/workers/service/basic-timeout.https-diff.txt

--- /Volumes/Data/worker/Apple-Ventura-Release-AppleSilicon-WK2-Tests/build/layout-test-results/http/tests/workers/service/basic-timeout.https-expected.txt
+++ /Volumes/Data/worker/Apple-Ventura-Release-AppleSilicon-WK2-Tests/build/layout-test-results/http/tests/workers/service/basic-timeout.https-actual.txt
@@ -5,9 +5,13 @@
 
 PASS frameText(testFrame1) is "Timeout resource fetched from network"
 PASS frameText(testFrame2) is "Not Found"
+FAIL frameText(testFrame3) should be Success!. Was .
+PASS frameText(testFrame1) is "Timeout resource fetched from network"
+PASS frameText(testFrame2) is "Not Found"
 PASS frameText(testFrame3) is "Success!"
-PASS frameText(testFrame4) is "Success!"
+FAIL frameText(testFrame4) should be Success!. Was .
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE


Buildbot: builder Apple-iOS-17-Simulator-Release-WK2-Tests build 1262 : 272498@main
https://build.webkit.org/#/builders/925/builds/1262

https://build.webkit.org/results/Apple-iOS-17-Simulator-Release-WK2-Tests/272498@main%20(1262)/http/tests/workers/service/basic-timeout.https-diff.txt

--- /Volumes/Data/worker/Apple-iOS-17-Simulator-Release-WK2-Tests/build/layout-test-results/http/tests/workers/service/basic-timeout.https-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-17-Simulator-Release-WK2-Tests/build/layout-test-results/http/tests/workers/service/basic-timeout.https-actual.txt
@@ -6,8 +6,9 @@
 PASS frameText(testFrame1) is "Timeout resource fetched from network"
 PASS frameText(testFrame2) is "Not Found"
 PASS frameText(testFrame3) is "Success!"
-PASS frameText(testFrame4) is "Success!"
+FAIL frameText(testFrame4) should be Success!. Threw exception TypeError: null is not an object (evaluating 'testFrame.contentDocument.body.innerText')
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE

Comment 5 Fujii Hironori 2024-01-04 18:44:16 PST

WPE and WinCairo ports are also flaky failures.

History:
  https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fworkers%2Fservice%2Fbasic-timeout.https.html