Bug 242730

Summary: Xcode 14 beta: `error: An empty identity is not valid when signing a binary for the product type 'Command-line Tool'.`
Product: WebKit Reporter: Elliott Williams <emw>
Component: Tools / TestsAssignee: Elliott Williams <emw>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, fred.wang, jbedard, rackler, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=243934
Bug Depends on: 244043    
Bug Blocks: 242429    

Description Elliott Williams 2022-07-13 17:41:08 PDT
I noticed while working on the open-source Ventura bringup that TestWebKitAPI and WebKitTestRunner have different signing setting than any other project, resulting in them using a "default" code-signing identity when building with the public SDK. This results in build failures like:

    error: An empty identity is not valid when signing a binary for the product type 'Command-line Tool'. (in target 'WebKitTestRunner' from project 'WebKitTestRunner')
Comment 1 Elliott Williams 2022-07-13 17:45:11 PDT
Pull request: https://github.com/WebKit/WebKit/pull/2396
Comment 2 Radar WebKit Bug Importer 2022-07-20 17:42:16 PDT
<rdar://problem/97350087>
Comment 3 EWS 2022-08-12 14:29:58 PDT
Committed 253391@main (3a3ad5eba5c5): <https://commits.webkit.org/253391@main>

Reviewed commits have been landed. Closing PR #2396 and removing active labels.
Comment 4 Karl Rackler 2022-08-15 09:33:57 PDT
This reverts commit 3a3ad5e.

Unreviewed revert
This reverts because causing 121 api test failures.

Canonical link: https://commits.webkit.org/253426@main
Comment 5 Elliott Williams 2022-08-15 18:04:53 PDT
Re-opening for pull request https://github.com/WebKit/WebKit/pull/3334
Comment 6 EWS 2022-08-16 12:11:03 PDT
Committed 253474@main (d245aed03e29): <https://commits.webkit.org/253474@main>

Reviewed commits have been landed. Closing PR #3334 and removing active labels.
Comment 7 Elliott Williams 2022-08-17 11:36:56 PDT
Reverted by https://github.com/WebKit/WebKit/pull/3414
Comment 8 EWS 2022-08-17 11:39:26 PDT
Committed 253527@main (808d943f6997): <https://commits.webkit.org/253527@main>

Reviewed commits have been landed. Closing PR #3414 and removing active labels.
Comment 9 Elliott Williams 2022-08-17 11:40:48 PDT
Reverted in https://commits.webkit.org/253527@main
Comment 10 Elliott Williams 2022-08-17 14:19:49 PDT
Pull request: https://github.com/WebKit/WebKit/pull/3428
Comment 11 Elliott Williams 2022-09-02 12:32:38 PDT
Since this has gone on for awhile, I should summarize the problem.

First, Xcode 14 now defaults to requiring Mac executables to be signed. Other projects have had to adapt to this change by defaulting to ad-hoc signing where they could previously opt out. (e.g. https://gitlab.kitware.com/cmake/cmake/-/merge_requests/7350)

I've been trying to use ad-hoc signing for open-source builds of TestWebKitAPI and WebKitTestRunner, but that has caused trusted execution problems. Here's my best understanding as to why.

- When EWS builds for Mac: On Intel, we DO NOT sign these TestWebKitAPI and WebKitTestRunner. On Apple Silicon, we "linker sign" them, which is a minimal signing technique that doesn't call `codesign` and ignores entitlements.

- When EWS builds for iOS (and other embedded Apple platforms): We ad-hoc sign. This is Xcode's default, as far as I am aware, because the simulators have stricter enforcement of signed binaries.

- When Apple engineers build TestWebKitAPI and WebKitTestRunner internally, they sign with the WK_ENGINEERING_CODE_SIGN_IDENTITY. When Apple builds these projects in production, they are ad-hoc signed. Both methods use `codesign` and embed entitlements.

- TestWebKitAPI and WebKitTestRunner both have entitlements which require their binaries to have a valid signature, notably NOT an ad-hoc signature. But, since open-source builds are not formally signed, these entitlements are never present on EWS.

I have tried to fix the build issue two separate ways:

- Disable code signing altogether: When I tried this, iOS test runs started breaking, because of the simulator's signature enforcement.

- Use ad-hoc signing for Mac WKTR and TestWebKitAPI: Doing this embeds the sensitive entitlements, which prevent execution on testers. Notably, it's not machine-dependent. I can re-sign these binaries on the test machines with the same entitlement plist, and they will still fail to launch. The kernel logs this message:

    2022-09-02 10:58:36.789 Df kernel.development[0:1d5aaf] mac_vnode_check_signature: /Users/emw/Downloads/Release/TestWebKitAPI: code signature validation failed fatally: When validating /Users/emw/Downloads/Release/TestWebKitAPI:
      Code has restricted entitlements, but the validation of its code signature failed.
    Unsatisfied Entitlements:

--

With this in mind, the next thing I want to try to do is turning off these entitlements for open-source builds. Hopefully that can produce ad-hoc-signed executables that will pass the signature check.

Alternatively, we could try disabling code signing again, but this time only for Mac builds.

Ultimately, it seems like a liability that we do not have a real signing identity for EWS. We should consider creating one and installing it on all EWS builders and testers so that we can do more advanced signing with our open-source build.
Comment 12 EWS 2022-09-06 11:58:45 PDT
Committed 254192@main (e347e3e762ae): <https://commits.webkit.org/254192@main>

Reviewed commits have been landed. Closing PR #3428 and removing active labels.
Comment 13 Elliott Williams 2023-01-13 15:14:10 PST
Re-opening for pull request https://github.com/apple/WebKit/pull/288
Comment 14 EWS 2023-01-13 18:37:15 PST
Committed 252432.1028@safari-7614-branch (aa02d3f68425): <https://commits.webkit.org/252432.1028@safari-7614-branch>

Reviewed commits have been landed. Closing PR #288 and removing active labels.