Why Bother With All This?
In other words: if you want to run on an Apple platform, why not just write everything in an Apple programming language, like Swift? If you need to ship to multiple platforms, you might have to rewrite it all anyway, so why not give up?
Despite the significant investment that platform vendors make in their tools, I fundamentally believe that the core logic in any software application ought to be where its most important value lies. For small, independent developers, having portable logic that can be faithfully replicated on every platform without massive rework might be tricky to get started with, but if you can’t do it, it may not be cost effective to support multiple platforms at all.
So, it makes sense for me to write my applications in Python to achieve this sort of portability, even though on each platform it’s going to be a little bit more of a hassle to get it all built and shipped since the default tools don’t account for the use of Python.
But how much more is “a little bit” more of a hassle? I’ve been slowly learning about the pipeline to ship independently-distributed1 macOS applications for the last few years, and I’ve encountered a ton of annoying roadblocks.
Didn’t You Do This Already? What’s New?
So nice of you to remember. Thanks for asking. While I’ve gotten this to mostly work in the past, some things have changed since then:
- the notarization toolchain has been updated (
- I’ve had to ship libraries other than just PyGame,
- Apple Silicon launched, necessitating another dimension of build complexity to account for multiple architectures,
- Perhaps most significantly, I have written a tool that attempts to encode as much of this knowledge as possible, Encrust, which I have put on PyPI and GitHub. If this is of interest to you, I would encourage you to file bugs on it, and hopefully add in more corner cases which I have missed.
I’ve also recently shipped my first build of an end-user application that successfully launches on both Apple Silicon and Intel macs, so here is a brief summary of the hoops I needed to jump through, from the beginning, in order to make everything work.
Wait did you say you wrote a tool? Is this fixed, then?
Encrust is, I hope, a temporary stopgap on the way to a much better comprehensive solution.
Specifically, I believe that Briefcase is a much more holistic solution to the general problem being described here, but it doesn’t suit my very specific needs right now4, and it doesn’t address a couple of minor points that I was running into here.
It is mostly glue that is shelling out to other tools that already solve portions of the problem, even when better APIs exist. It addresses three very specific layers of complexity:
- It enforces architecture independence, so that your app built on an M1 machine will still actually run on about half of the macs remaining out there2.
- It remembers tricky nuances of the notarization submission process, such as
the highly specific way I need to generate my
zipfiles to avoid mysterious notarization rejections3.
- Providing a common and central way to store the configuration for these things across repositories so I don’t need to repeat this process and copy/paste a shell script every time I make a tiny new application.
It only works on Apple Silicon macs, because I didn’t bother to figure out how
pip actually determines which architecture to download wheels for.
As such, unfortunately, Encrust is mostly a place for other people who have already solved this problem to collaborate to centralize this sort of knowledge and share ideas about where this code should ultimately go, rather than a tool for users trying to get started with shipping an app.
- I want to help Python developers ship their Python apps to users who are not also Python developers.
- macOS is an even trickier platform to do that on than most.
- It’s now easy for me to sign, notarize, and release new applications reliably
If you have an open source Python application that runs on macOS5 but can’t ship to macOS — either because:
- you’ve gotten stuck on one of the roadblocks that this post describes,
- you don’t have $100 to give to Apple, or because
- the app is using a cross-platform toolkit that should work just fine and you don’t have access to a mac at all, then
Send me an email and I’ll sign and post your releases.
What’s this post about, then?
People still frequently complain that “Python
packaging” is really bad. And I’m on record
that packaging Python (in the sense of “code”) for Python (in the sense of
“deployment platform”) is actually kind of fine right now; if what you’re
trying to get to is a package that can be
pip installed, you can have a
reasonably good experience modulo a few small onboarding hiccups that are
well-understood in the community and fairly easy to overcome.
However, it’s still unfortunately hard to get Python code into the hands of users who are not also Python programmers with their own development environments.
My goal here is to document the difficulties themselves to try to provide a snapshot of what happens if you try to get started from scratch today. I think it is useful to record all the snags and inscrutable error messages that you will hit in a row, so we can see what the experience really feels like.
I hope that everyone will find it entertaining.
- Other Mac python programmers might find pieces of trivia useful, and
- Linux users will have fun making fun of the hoops we have to jump through on Apple platforms,
but the main audience is the maintainers of tools like Briefcase and py2app to evaluate the new-user experience holistically, and to see how much the use of their tools feels like this. This necessarily includes the parts of the process that are not actually packaging.
This is why I’m starting from the beginning again, and going through all the stuff that I’ve discussed in previous posts again, to present the whole experience.
So, with no further ado, here is a non-exhaustive list of frustrations that I have encountered in this process:
- Okay. Time to get started. How do I display a GUI at all? Nothing happens when I call some nominally GUI API. Oops: I need my app to exist in an app bundle, which means I need to have a framework build. Time to throw those partially-broken pyenv pythons in the trash; best to use the official python.org from here on out.
- Bonus Frustration since I’m using AppKit directly: why is my app
segfaulting all the time? Oh,
targetis a weak reference in objective C, so if I make a window and put a button in it that points at a Python object, the Python interpreter deallocates it immediately because only the window (which is “nothing” as it’s a weakref) is referring to it. I need to start stuffing every Python object that talks to a UI element like a window or a button into a global list, or manually calling
.retain()on all of them and hoping I don’t leak memory.
- Everything seems to be using the default Python Launcher icon, and the app menu says “Python”. That wouldn’t look too good to end users. I should probably have my own app.
- I’ll skip the part here where the author of a new application might have to investigate py2app, briefcase, pyoxidizer, and pyinstaller separately and try to figure out which one works the best right now. As I said above, I started with py2app and I’m stubborn to a fault, so that is the one I’m going to make work.
- Now I need to set up py2app. Oops, I can’t use
pyproject.tomlany more, time to go back to
- Now I built it and the the app is crashing on startup when I click on it. I
can’t see a traceback anywhere, so I guess I need to do something in the
- Wow; the console is an unusable flood of useless garbage. Forget that.
- I guess I need to run it in the terminal somehow. After some googling I
figure out it’s
./dist/MyApp.app/Contents/Resources/MacOS/MyApp. Aha, okay, I can see the traceback now, and it’s … an import error?
- Ugh, py2app isn’t actually including all of my code, it’s using some
magic to figure out which modules are actually used, but it’s doing it
importstatements, which means I need to put a bunch of fake static
importstatements for everything that is used indirectly at the top of my app’s main script so that it gets found by the build. I experimentally discover a half a dozen things that are dynamically imported inside libraries that I use and jam them all in there.
- Okay. Now at least it starts up. The blank app icon is uninspiring, though,
time to actually get my own icon in there. Cool, I’ll make an icon in my
favorite image editor, and save it as... icons must be PNGs, right?
Uhh... no, looks like they have to be
.icnsfiles. But luckily I can convert the PNG I saved with a simple 12-line shell script that invokes
At this point I have an app bundle which kinda works. But in order to run on anyone else’s computer, I have to code-sign it.
- In order to code-sign anything, I have to have an account with Apple that costs $99 per year, on developer.apple.com.
- The easiest way to get these certificates is to log in to Xcode itself.
There’s a web
but using it appears to involve a lot more manual management of key material,
so, no thanks. This requires the full-fat Xcode.app though, not just the
command-line tools that come down when I run
xcode-select --install, so, time to wait for an 11GB download.
- Oops, I made the wrong certificate type. Apparently the only right answer here is a “Developer ID Application” certificate.
- Now that I’ve logged in to Xcode to get the certificate, I need to figure out
how to tell my command-line tools about it (for starters, “
codesign”). Looks like I need to run
security find-identity -v -p codesigning.
- Time to sign the application’s code.
codesigntool has a
--deepoption which can sign the whole bundle. Great!
- Except, that doesn’t work, because Python ships shared libraries in
locations that macOS doesn’t automatically expect, so I have to manually
locate those files and sign them, invoking
codesignonce for each.
--deepis deprecated. There’s no replacement.
- Logically, it seems like I still need
--deep, because it does some poorly-explained stuff with non-code resource files that maybe doesn’t happen properly if I don’t? Oh well. Let's drop the option and hope for the best.8
- With a few heuristics I think we can find all the relevant files with a little script7.
Now my app bundle is signed! Hooray. 12 years ago, I’d be all set. But today I need some additional steps.
- After I sign my app, Apple needs to sign my app (to indicate they’ve checked
it for malware), which is called “notarization”.
- In order to be eligible for notarization, I can’t just code-sign my app. I have to code-sign it with entitlements.
- Also I can’t just code sign it with entitlements, I have to sign it with the hardened runtime, or it fails notarization.
- Oops, out of the box, the hardened runtime is incompatible with a bunch
of stuff in Python, including cffi and ctypes, because nobody has
implemented support for
MAP_JITyet, so it crashes at startup. After some thrashing around I discover that I need a legacy “allow unsigned executable memory” entitlement. I can’t avoid importing this because a bunch of things in py2app’s bootstrapping code import things that use ctypes, and probably a bunch of packages which I’m definitely going to need, like
- In order to set up notarization external to Xcode, I need to create an App Password which is set up at appleid.apple.com, not the developer portal.
- Bonus Frustration since I’ve been doing this for a few years:
Originally this used to be even more annoying as I needed to wait for an
altool), and so I couldn’t script it directly. Now, at least, the new
notarytool(which will shortly be mandatory) has a
- Although the tool is documented under
man notarytool, I actually have to run it as
xcrun notarytool, even though
codesigncan be run either directly or via
- Great, we’re ready to zip up our app and submit to Apple. Wait, they’re rejecting it? Why???
- Aah, I need to manually copy and paste the UUID in the console output
xcrun notarytool submitinto
xcrun notarytool logto get some JSON that has some error messages embedded in it.
- Oh. The bundle contains internal symlinks, so when I zipped it without
-yoption, I got a corrupt archive.
- Great, resubmitted with
- Oops, just kidding, that only works sometimes. Later, a different submission with a different hash will fail, and I’ll learn that the correct command line is actually
ditto -c -k --sequesterRsrc --keepParent MyApp.app MyApp.app.zip.
- Note that, for extra entertainment value, the position of the archive
itself and directory are reversed on the command line from
tar, and every other archive tool).
- Note that, for extra entertainment value, the position of the archive itself and directory are reversed on the command line from
notarytooldoesn’t record anything in my app though; it puts the “notarization ticket” on Apple's servers. Apparently, I still need to run
staplerfor users to be able to launch it while those servers are inaccessible, like, for example, if they’re offline.
- Oops, not
xcrun stapler. Whatever.
notarytooloperates on a zip archive, but
stapleroperates on an app bundle. So we have to save the original app bundle, run
stapleron it, then re-archive the whole thing into a new archive.
Hooray! Time to release my great app!
- Whoops, just got a bug report that it crashes immediately on every Intel mac. What’s going on?
- Turns out I’m using a library whose authors distribute both
pipwill prefer single-architecture wheels even if
universal2wheels are also available, so I’ve got to somehow get fat binaries put together. Am I going to have to build a huge pile of C code by myself? I thought all these Python hassles would at least let me avoid the C hassles!
Whew, okay, no need for that: there’s an amazing Swiss-army knife for macOS binary wheels, called
delocatethat includes a
delocate-fusetool that can fuse two wheels together. So I just need to figure out which binaries are the wrong architecture and somehow install my fixed/fused wheels before building my app with
- except, oops, this tool just rewrites the file in-place without even changing its name, so I have to write some janky shell scripts to do the reinstallation. Ugh.
OK now that all that is in place, I just need to re-do all the steps:
universal2-ize my virtualenv!
And we have an application bundle we can ship to users.
It’s just that easy.
As long as I don’t need sandboxing or Mac App Store distribution, of course. That’s a challenge for another day.
So, that was terrible. But what should be happening here?
Some of this is impossible to simplify beyond a certain point - many of the things above are not really about Python, but are about distribution requirements for macOS specifically, and we in the Python community can’t affect operating system vendors’ tooling.
What we can do is build tools that produce clear guidance on what step is required next, handle edge cases on their own, and generally guide users through these complex processes without requiring them to hit weird binary-format or cryptographic-signing errors on their own with no explanation of what to do next.
I do not think that documentation is the answer here. The necessary steps
should be discoverable. If you need to go to a website, the tool should use
module to open a website. If you need to launch an app, the tool should launch
Encrust, I am hoping to generalize
the solutions that I found while working on this for this one specific slice of
the app distribution pipeline — i.e. a macOS desktop application desktop, as
distributed independently and not through the mac app store — but other
platforms will need the same treatment.
However, even without really changing
py2app or any of the existing tooling,
we could imagine a tool that would interactively prompt the user for each
manual step, automate as much of it as possible, verify that it was performed
correctly, and give comprehensible error messages if it was not.
For a lot of users, this full code-signing journey may not be necessary; if you just want to run your code on one or two friends’ computers, telling them to right click, go to ‘open’ and enter their password is not too bad. But it may not even be clear to them what the trade-off is, exactly; it looks like the app is just broken when you download it. The app build pipeline should tell you what the limitations are.
Other parts of this just need bug-fixes to address. py2app specifically, for example, could have a better self-test for its module-collecting behavior, launching an app to make sure it didn’t leave anything out.
Interactive prompts to set up a Homebrew tap, or a Flatpak build, or a Microsoft Store Metro app, might be similarly useful. These all have outside-of-Python required manual steps, and all of them are also amenable to at least partial automation.
Thanks to my patrons on Patreon for supporting this sort of work, including development of Encrust, of Pomodouroboros, of posts like this one and of that offer to sign other people’s apps. If you think this sort of stuff is worthwhile, you might want to consider supporting me over there as well.
I am not even going to try to describe building a sandboxed, app-store ready application yet. ↩
At least according to the Steam Hardware Survey, which as of this writing in March of 2023 pegs the current user-base at 54% apple silicon and 46% Intel. The last version I can convince the Internet Archive to give me, from December of 2022, has it closer to 51%/49%, which suggests a transition rate of 1% per month. I suspect that this is pretty generous to Apple Silicon as Steam users would tend to be earlier adopters and more sensitive to performance, but mostly I just don’t have any other source of data. ↩
It is truly remarkable how bad the error reporting from the notarization service is. There are dozens of articles and forum posts around the web like this one where someone independently discovers this failure mode after successfully notarizing a dozen or so binaries and then suddenly being unable to do so any more because one of the bytes in the signature is suddenly not valid UTF-8 or something. ↩
A lot of this is probably historical baggage; I started with py2app in 2008 or so, and I have been working on these apps in fits and starts for… ugh… 15 years. At some point when things are humming along and there are actual users, a more comprehensive retrofit of the build process might make sense but right now I just want to stop thinking about this. ↩
If your application isn’t open source, or if it requires some porting work, I’m also available for light contract work, but it might take a while to get on my schedule. Feel free to reach out as well, but I am not looking to spend a lot of time doing porting work. ↩
I find this particular detail interesting; it speaks to the complexity and depth of this problem space that this has been a known issue for several years in Briefcase, but there’s just so much other stuff to handle in the release pipeline that it remains open. ↩
I forgot both
.afiles and the py2app-included
pythonexecutable itself here, and had to discover that gap when I signed a different app where that made a difference. ↩
Thus far, it seems to be working. ↩