Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hardened Runtime is not enabled for Sparkle. macOS Mojave #1266

Closed
brendand opened this issue Jul 16, 2018 · 29 comments
Closed

Hardened Runtime is not enabled for Sparkle. macOS Mojave #1266

brendand opened this issue Jul 16, 2018 · 29 comments

Comments

@brendand
Copy link

When I attempt to upload a Developer ID Mac app to Apple using Xcode 10 on Mojave (beta 3), I get the following error:

Hardened Runtime is not enabled.

"org.sparkle-project.InstallerConnection.xpc", "org.sparkle-project.InstallerLauncher.xpc", "org.sparkle-project.InstallerStatus.xpc", "Autoupdate", "Updater.app", "Autoupdate", and "Updater.app" must be rebuilt with support for the Hardened Runtime. Enable the Hardened Runtime capability in the project editor, then test your app, rebuild your archive, and upload again.

I'm wondering if there's any instructions that would help to resolve this situation? I've enabled Hardened Runtime in my own app, but I'm not sure how to get it all working properly for Sparkle as well.

@kornelski
Copy link
Member

We haven't looked at it yet. If you'd like to help, clone Sparkle's source code and see what it takes to enable this mode for Sparkle itself.

@brendand
Copy link
Author

I attempted to enable Hardened Runtime for the above items, but then I received errors:

Ad Hoc code signing is not compatible with the Hardened Runtime. Sign with a Mac Developer certificate or disable the Hardened Runtime by setting the ENABLE_HARDENED_RUNTIME build setting to NO. (in target 'SparkleInstallerConnection')

for each of the items I enabled it on.

I'll keep trying different things though.

@brendand
Copy link
Author

I've also tried hardening the Sparkle Test App and uploading that, but I get the following error:

"Sparkle Test App.app", "TestAppHelper.xpc", "org.sparkle-project.Downloader.xpc", "org.sparkle-project.InstallerConnection.xpc", "org.sparkle-project.InstallerLauncher.xpc", "org.sparkle-project.InstallerStatus.xpc", "Autoupdate", "Updater.app", "Autoupdate", and "Updater.app" must be rebuilt with support for the Hardened Runtime. Enable the Hardened Runtime capability in the project editor, then test your app, rebuild your archive, and upload again.

@brendand
Copy link
Author

So it looks like the XPC Services Script that I had to sign the XPC Services is no longer needed in Xcode. By commenting that all out, I was able to upload to Apple's Notary service. Although I did enable hardened runtime on all the targets in Sparkle, then built the project and copied the XPC and Sparkle.framework to my project. I used the "Automatically manage signing" setting in Sparkle before archiving. I also had to comment out the code inside "$SRCROOT/Configurations/set-agent-signing.sh" because it got confused with ambiguous certificates for "3rd party Mac Developer" and "Mac Developer..." in my keychain.

But I was successfully able to upload so that's good.

@brendand
Copy link
Author

I think it's safe to close this issue since the upload and notarization worked fine after doing what I mentioned in the previous comment.

@mackworth
Copy link

@brendand (and @kornelski), do you think everyone will need to rebuild the project themselves for Mojave, or will there be still be a binary downloadable version?

I started to follow the notes above, but not having built Sparkle before, I ran into problems at your first step of "enabling hardened runtimes". I don't see a Capabilities tab anywhere for Sparkle framework. I set it for Autoupdate.app, but when I generated the framework and copied it into my app, attempting to archive still said "Autoupdate.app" and "fileop" must be rebuilt with support for the Hardened Runtime. Could you give a bit more detail?

@kornelski
Copy link
Member

I can't find information whether hardened runtime is related to sandboxing. If it requires sandboxing, then it won't be supported in Sparkle 1.x.

@mackworth
Copy link

mackworth commented Aug 14, 2018

Mojave provides the option for a hardened runtime, which protects apps against certain attacks such as code injection, etc. AFAICT this is enabling SIP at the app level. It's optional, but it's required for Apple's new notarization service, where you submit the app for notarization to Apple, but then you distribute it yourself. In addition to enhanced security, running a notarized apps can be run by the user, without being warned off by the Gatekeeper.

I'm sure delta sparkle upgrades won't work<<Incorrect; see below>>, but I don't think there'll be any other issue for Sparkle support, other than signing process as discussed here.

@michelf
Copy link
Contributor

michelf commented Aug 14, 2018

It'd be nice if notarization could prevent app translocation in some way.

@mackworth: Why would this break delta updates? If reconstructed properly, shouldn't the resulting app match the notarized signature? Notarization authenticating the downloaded archive instead of the signature for the code-signed app would be weird.

@mackworth
Copy link

Sorry, just ignorance on my part of how deltas work.

BTW, @kornelski, I should mention: the hardened runtime does not restrict which files you can access, so that’s why I don’t think there’ll be any sandbox-related issue.

@mackworth
Copy link

So, after much trial and error, at least in my circumstance, all I needed to do was add:

codesign --verbose --force --deep -o runtime --sign "$IDENTITY" "$LOCATION/Sparkle.framework/Versions/A/Resources/AutoUpdate.app"
to my Signed Bundled Frameworks phase.
(FYI, the -o runtime turns on the hardened runtime, and the --deep is necessary to sign the fileop binary.)

Turns out I didn't need to recompile, I could just use the current downloaded Sparkle Framework binary.

So, I'd suggest adding this to the documentation somewhere, but otherwise things look good to go. (Although I admit I haven't run the autoupdate on the hardened binary yet)

@cyphers
Copy link

cyphers commented Sep 13, 2018

Any chance we can get a little bit more of layman's guide to how to get Sparkle fixed for Mojave? As someone who has simply used the built frameworks, opening the Sparkle Xcode project doesn't even seem to build properly, using Archive produces no products, there are a billion warnings, using the All target fails unit tests, etc. I tried turning on Hardened mode but at this point can't even find built binaries of Sparkle.framework to try in my app. This is with Xcode 10 GM. Any pointers?

@kornelski
Copy link
Member

Archive in Xcode is some weird thing that has never correctly worked for me for any project, ever.

For me Sparkle builds cleanly without warnings on Mojave beta, and Xcode 10. But we only support Build option in Xcode. If the normal build option doesn't work flawlessly, please file bugs.

@mackworth
Copy link

Archive works great for me, and is mandatory to submit for the new Hardened/Notarized option.

@cyphers: so the project builds fine with Xcode 9, but not with 10? I had no problem with Sparkle on Xcode 10 until I tried hardening.

Definitely, don't turn on hardening until you've built properly without it. Once you have, then you can turn it on, and sign both the framework and the AutoUpdate App with the -o option.

To be specific, my final build phase is Sign Bundled Frameworks:

LOCATION="${BUILT_PRODUCTS_DIR}"/"${FRAMEWORKS_FOLDER_PATH}"

# By default, use the configured code signing identity for the project/target
IDENTITY="${CODE_SIGN_IDENTITY}"
if [ "$IDENTITY" == "" ]
then
    # If a code signing identity is not specified, use ad hoc signing
    IDENTITY="-"
fi
codesign --verbose --force --deep -o runtime --sign "$IDENTITY" "$LOCATION/Sparkle.framework/Versions/A/Resources/AutoUpdate.app"
codesign --verbose --force -o runtime --sign "$IDENTITY" "$LOCATION/Sparkle.framework/Versions/A"

@cyphers
Copy link

cyphers commented Sep 14, 2018

Using Archive on Xcode 10 results in an Archive that has an empty Products folder... That's for the Framework target and every other target I tried. The dSYMs are present, Products is empty. That's from a fresh checkout of master, no modifications.

@mackworth
Copy link

That's fascinating. I just did the same thing (Github; Master Branch; Main screen; Clone or Download; Open in Xcode 10 GM (10A254a), Clone in New Folder, Selected Distribution Scheme, Archive (ignored 92 warnings)

Then Organizer, Distribute Content, Built Products, Select new Folder on Desktop, Export. )

I got the applications: Sparkle Test App, Binary Delta and generate_appcast.

Tried again with All, and did get a linker failure in the Unit Tests. Not sure why that was.

@mackworth
Copy link

I note also PR #1244 , commit 0973abf, which changes line 11 in Configurations/ConfigFramework.xcconfig
from SKIP_INSTALL = YES to SKIP_INSTALL = NO. This causes the framework to be included in the Archive and exported cleanly.

@brendand
Copy link
Author

You can also find the built production release in the build folder, even if they don't get included in the archive export.

@cyphers
Copy link

cyphers commented Sep 14, 2018

Confirming @mackworth, doing everything in his last 3 comments here seems to result in success. Or at least it provides a build that Xcode says is notarized but then Xcode doesn't highlight the export notarized app button. I think there are some Xcode bugs here probably unrelated to any of this, but I did get a notification of success FWIW.

@dxdc
Copy link

dxdc commented Oct 6, 2018

@mackworth Confirming that I was able to get this to result in success also. Thanks for sharing.

On some of my apps, I didn't even get the option in Organizer to send these to Apple for notary services. After some troubleshooting, I learned this was because of the SKIP_INSTALL setting on the QuickLook module. Modifying this, then archiving again, restored the upload option.

I had to modify your script slightly to get it to work. Using CODE_SIGN_IDENTITY reported as ambiguous. I suspect this is because I have multiple certificates installed on my system. If EXPANDED_CODE_SIGN_IDENTITY_NAME works in your case as well, it may be a more general solution.

IDENTITY="${EXPANDED_CODE_SIGN_IDENTITY_NAME}"

@adib
Copy link

adib commented Oct 22, 2018

From reading the conversation above, I take it the only way to support notarization is to build Sparkle from source and enable hardening on it?

@dxdc
Copy link

dxdc commented Oct 22, 2018

@adib I didn't have to build from source. Just apply the code signing script I mentioned and the 'standard' Sparkle configuration worked. Received notice from Apple that the app was notarized (took several days).

@mackworth
Copy link

mackworth commented Oct 22, 2018 via email

@JamesBucanek
Copy link

Thanks, @mackworth, I'm going through the same pain right now.

I added your script as a new Run Script phase for my top-level app target and was successful in archiving and uploading it for notarization. Now to figure out how to do this using xcodebuild ...

For anyone else struggling with notarization, I also discovered that Apple will only notarize an archive that contains a single product. So you need to set SKIP_INSTALL=YES on every product target except your final one.

In our case, where our final build actually produces three separate apps for distribution, it looks like we're going to have to re-factor our production scripts to archive and notarize each app separately.

@kolpanic
Copy link
Contributor

kolpanic commented Nov 21, 2018

I got our sandboxed app built with Sparkle and the hardened runtime enabled, and it was accepted by Apple's notarization service. In case anyone else finds this looking for a more concise procedure, here are the steps I followed.

Build

  1. Get the ui-separation-and-xpc branch.
  2. In Terminal, make release in the Sparkle project directory. Upon success, a Finder window will open revealing an archive containing the newly-built Sparkle binaries. Expand the archive (e.g., to the Desktop).
  3. Edit the XPC code signing script bin/codesign_xpc that was copied into the build archive to enable the hardened runtime by changing the command in the codesign script to:
    • command = ["codesign", "-o", "runtime", "-fs", identity, path] + ([] if entitlements_path is None else ["--entitlements", entitlements_path])
  4. Use this modified script to sign the XPCs. In Terminal, cd to the expanded Sparkle directory and:
    • ./bin/codesign_xpc "Developer ID Application: XXXX" ./XPCServices/*.xpc (use an appropriate code signing identity here and below)
  5. Sign the two binaries embedded within Sparkle.framework with the hardened runtime option:
    • codesign -o runtime -fs "Developer ID Application: XXXX" Sparkle.framework/Versions/A/Resources/Autoupdate
    • codesign -o runtime -fs "Developer ID Application: XXXX" Sparkle.framework/Versions/A/Resources/Updater.app/
  6. Copy the signed Sparkle bundles to the app project:
    • Sparkle.framework
    • org.sparkle-project.InstallerConnection.xpc
    • org.sparkle-project.InstallerLauncher.xpc
    • org.sparkle-project.InstallerStatus.xpc
    • org.sparkle-project.Downloader.xpc (required only if the app doesn't have the network client entitlement)
  7. Add them to the appropriate copy phases and link settings in the app target. Check the "Code Sign on Copy" option for the FW phase (it's not offered for the XPC copy phase).
  8. Enable the Hardened Runtime setting on the Capabilities tab for relevant targets that have it—apps, share extensions, etc. (If you miss one, you won't be able to upload to the notary service; however, it'll tell you which item is incorrectly signed. Also, don't use the option on Build Settings.)

Test

In the past, I tested Sparkle updates by temporarily lowering the CURRENT_PROJECT_VERSION, launching the app, and using the Check for Updates… menu item. This no longer works when running a debug build from Xcode—the "Extracting Update" phase either hangs or fails. Instead, I have to do an archive build and export a Developer ID-signed app.

I built our app with Xcode 10.1 on macOS 10.14.1, but the deployment target is set to 10.12. I successfully tested updates on both 10.14.1 and 10.12.

Notarize

At this point, Apple's notary service accepts the app as having the hardened runtime enabled. It took a few minutes to negotiate the connection and complete the upload, and a few minutes more for the app to be "Ready to Distribute". At that point, there was an "Export Notarized App" button visible at the bottom-right corner of the Organizer window. Running command xcrun stapler validate MyApp.app on the exported notarized app showed the app as valid.

@kolpanic
Copy link
Contributor

I asked for some clarification at Apple's developer forums. Quinn “The Eskimo!” replied with some typically great information. https://forums.developer.apple.com/message/340768#340768

It turns out you only have to enable the hardened runtime for apps and app-like targets—e.g., share extensions. It pretty much boils down to this: if a target has a Capabilities tab—apps, share extensions, etc.—then enable it there. If not, then don't worry.

My only remaining question there is how to handle the code signing of deeply-embedded apps, like Sparkle's Updater. Specifically, I want to know if the --deep flag is necessary.

I'll update my previous post once I have that cleared up, and I get our apps finalized.

@kolpanic
Copy link
Contributor

kolpanic commented Nov 23, 2018

@kornelski It looks like the --deep option isn't necessary when signing the two executables inside Sparkle.framework. I've updated my previous comment to reflect that. In addition, I noted that the Hardened Runtime setting should be enabled on the Capabilities tab (not Build Settings). I was able to get an app notarized with the steps now listed above.

A couple of suggestions:

  1. Please update the codesign_xpc script to add the -o runtime flag to the codesign command, perhaps optionally.
  2. Please add a similar script to codesign the executables inside Sparkle.framework. Alternatively, add the functionality to the codesign_xpc script (and probably rename the script too).

I'd tackle it, but I'm not proficient in Python. If you like, I can open an issue.

@kornelski
Copy link
Member

Yes, please open an isssue. And give it a try at implementation — at this point you know this better than I do :)

@kolpanic
Copy link
Contributor

kolpanic commented Nov 23, 2018

@kornelski I opened #1319.

It occurs to me that just adding "-o runtime" and renaming the script is sufficient. The instructions can be then be updated to reflect the new name and the step of signing the embedded executables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants