Bug 209646

Summary:	Regression(STP 103): 6 webdriver tests with new failures between STP 102 and 103
Product:	WebKit	Reporter:	Philip Jägenstedt <philip>
Component:	WebDriver	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED MOVED
Severity:	Normal	CC:	bburg, bfulgham, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	Safari Technology Preview
Hardware:	Unspecified
OS:	Unspecified

Description Philip Jägenstedt 2020-03-27 02:25:44 PDT

In upgrading from STP 102 to 103 we see new failures in the webdriver tests, which is often bad news even though in this case it doesn't seem to lead to widespread failures in other tests.

The regressions can be seen here:
https://wpt.fyi/results/webdriver/tests?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002

See https://github.com/web-platform-tests/wpt/pull/22475#issuecomment-604899967 for the details.

Comment 1 Radar WebKit Bug Importer 2020-03-27 16:48:50 PDT

<rdar://problem/60990710>

Comment 2 BJ Burg 2020-03-30 16:02:37 PDT

There is some known flakiness in user prompt handling tests. I'll re-run these singly to see if these are failing in a uniquely bad way but it seems unlikely.

Comment 3 BJ Burg 2020-03-30 16:11:57 PDT

https://wpt.fyi/results/webdriver/tests/find_element/user_prompts.py?q=none%28status%3Amissing%29%20and%20seq%28status%3Apass%20status%3A%21pass%29&run_id=448800002&run_id=462410002



test_ignore[capabilities0-alert]	PASS	ERROR message: setup error
test_ignore[capabilities0-confirm]	PASS	ERROR message: setup error
test_ignore[capabilities0-prompt]	PASS	ERROR message: setup error
test_default[alert-None]	PASS	ERROR message: setup error
test_default[confirm-False]	PASS	ERROR message: setup error


I'm not sure what setup error is, but I'm assuming it's because the Safari is already paired with another session and is unable to create a new session. This is being tracked internally as a separate issue.

Comment 4 Philip Jägenstedt 2020-04-28 01:42:37 PDT

Adding STP 105 results to the original comparison shows that there are only two remaining regressions:
https://wpt.fyi/results/webdriver/tests?q=seq%28status%3Apass%20status%3A%21pass%20status%3A%21pass%29&run_id=448800002&run_id=462410002&run_id=501500002

They are:
test_link_closes_window in element_click/navigate.py
test_default[alert-None] in get_active_element/user_prompts.py

Comment 5 Brent Fulgham 2022-02-10 21:46:22 PST

The fix for this issue was needed outside the WebKit project, therefore this is being resolved as 'Moved'.

This should now be fixed in shipping software.