Bug 207617

Summary:	Regression(macOS Catalina): sudo safaridriver --enable not working
Product:	WebKit	Reporter:	Philip Jägenstedt <philip>
Component:	WebDriver	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	bburg
Priority:	P2
Version:	Safari Technology Preview
Hardware:	Unspecified
OS:	macOS 10.15

Description Philip Jägenstedt 2020-02-12 00:54:48 PST

The safaridriver that's part of STP is used by web-platform-tests to run the test suite on macOS VMs on Azure Pipelines.

On macOS 10.14 `sudo "/Applications/Safari Technology Preview.app/Contents/MacOS/safaridriver" --enable` is called and this works. However, upgrading to 10.15 https://github.com/web-platform-tests/wpt/pull/21628 this no longer works. The error is "SessionNotCreatedException: session not created (500): Could not create a session: You must enable the 'Allow Remote Automation' option in Safari's Develop menu to control Safari via WebDriver." STP 100 was used in both cases.

Something has also changed for Safari stable. `sudo safaridriver --enable` already didn't work, in what seemed to be the same problem that affected STP 91 and was later fixed. However, the plist workaround added in https://github.com/web-platform-tests/wpt/pull/19846 also stopped working with macOS Catalina.

In summary, there's no apparent way to enable safaridriver for Safari stable or STP, and we can't upgrade to macOS Catalina. This is a risky situation to be in, because Azure Pipelines only supports two versions of macOS at a time, so if the problem isn't fixed we may be left with no way to run the tests.

Comment 1 Philip Jägenstedt 2020-02-12 01:07:57 PST

I've filed https://github.com/web-platform-tests/wpt/issues/21751 to summarize the issue on the wpt side. That's blocked on this issue, but maybe there's logging we could add to help diagnose the problem.

Comment 2 BJ Burg 2020-02-12 13:57:38 PST

Marking resolved/fixed as this is not a WebKit change.

I would highly advise you to try out the recently released update Seed builds as these should include the fix that's been in trunk for a while. If `sudo safaridriver --enable` does not work on the seed, we need to know ASAP.

Comment 3 BJ Burg 2020-02-12 14:15:24 PST

Eg. https://www.macrumors.com/2020/02/10/apple-seeds-macos-catalina-10-14-5-public-beta-1/

Comment 4 Philip Jägenstedt 2020-02-13 03:31:14 PST

I'm afraid I'm not able to try a beta version of macOS, my MPB is my work machine and on CI systems like Azure Pipelines you can only use the versions the currently support.

Is the problem that's been fixed in the recent Seed build one that was introduced in 10.15 and broke `sudo safaridriver --enable` for both Safari stable and STP? If so it's probably the same issue, but otherwise this might be different.

If I did have access, the way I'd test it is to start from a fresh OS install and run `sudo safaridriver --enable` without ever having launched the browser before. If when the starting the browser the "remote automation" option is checked in the context menu, I'd bet it will also work in a CI setup.