Bug 245725 - REGRESSION (254766main): [ macOS ] http/wpt/webauthn/public-key-credential-get-failure.https.html is a flaky failure
Summary: REGRESSION (254766main): [ macOS ] http/wpt/webauthn/public-key-credential-ge...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-09-27 06:31 PDT by Karl Rackler
Modified: 2022-09-27 16:38 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2022-09-27 06:31:43 PDT
Description:
http/wpt/webauthn/public-key-credential-get-failure.https.html

The test was consistently passing until at 254766@main it began to flaky fail on Monterey wk2.  The test is a consistent failure on Monterey wk2 arm64.

Reproduction steps:
I can reproduce this on 254766main but cannot reproduce it on 254765@main or earlier.

Command: 
run-webkit-tests --iterations 50 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 -f --no-retry --release  http/wpt/webauthn/public-key-credential-get-failure.https.html

History:
https://results.webkit.org/?suite=layout-tests&test=http%2Fwpt%2Fwebauthn%2Fpublic-key-credential-get-failure.https.html

DIFF:
--- /Volumes/Data/worker/Apple-Monterey-Release-WK2-Tests/build/layout-test-results/http/wpt/webauthn/public-key-credential-get-failure.https-expected.txt
+++ /Volumes/Data/worker/Apple-Monterey-Release-WK2-Tests/build/layout-test-results/http/wpt/webauthn/public-key-credential-get-failure.https-actual.txt
@@ -4,7 +4,6 @@
 CONSOLE MESSAGE: User gesture is not detected. To use the WebAuthn API, call 'navigator.credentials.create' or 'navigator.credentials.get' within user activated events.
 CONSOLE MESSAGE: User gesture is not detected. To use the WebAuthn API, call 'navigator.credentials.create' or 'navigator.credentials.get' within user activated events.
 CONSOLE MESSAGE: User gesture is not detected. To use the WebAuthn API, call 'navigator.credentials.create' or 'navigator.credentials.get' within user activated events.
-CONSOLE MESSAGE: Unhandled Promise Rejection: NotAllowedError: Operation timed out.
 CONSOLE MESSAGE: Unhandled Promise Rejection: NotAllowedError: Operation timed out.
 
 PASS PublicKeyCredential's [[get]] with timeout
Comment 1 Radar WebKit Bug Importer 2022-09-27 06:32:04 PDT
<rdar://problem/100456553>
Comment 2 Karl Rackler 2022-09-27 06:42:08 PDT
I have marked this test as a flaky failure while this issue is investigated.
Comment 3 EWS 2022-09-27 06:44:36 PDT
Test gardening commit 254908@main (12e012da9a05): <https://commits.webkit.org/254908@main>

Reviewed commits have been landed. Closing PR #4746 and removing active labels.
Comment 4 Karl Rackler 2022-09-27 07:34:08 PDT
Adding test:
http/wpt/webauthn/public-key-credential-create-failure.https.html

The test was consistently passing until at 254766@main it began to flaky fail on macOS wk2.  The test is a consistent failure on Monterey wk2 arm64.

Same Diff.
Comment 5 EWS 2022-09-27 07:41:32 PDT
Test gardening commit 254912@main (fb2d813dee6a): <https://commits.webkit.org/254912@main>

Reviewed commits have been landed. Closing PR #4750 and removing active labels.
Comment 6 Chris Dumez 2022-09-27 08:18:26 PDT
Pull request: https://github.com/WebKit/WebKit/pull/4751
Comment 7 EWS 2022-09-27 16:38:35 PDT
Committed 254934@main (49a6e960281b): <https://commits.webkit.org/254934@main>

Reviewed commits have been landed. Closing PR #4751 and removing active labels.