Skip to main content

DIY: Core Animation

ยท 6 min read

(Check out the project over here.)

After having been a consumer of the Core Animation (originally LayerKit) framework for about a decade now, I began to wonder how it was designed and implemented under the hood. Not simply the surface details such as "it's rendered out of process" or "it handles the animation interpolation for you" or whatnot, but more specifically, HOW all of its beautiful bells and whistles translate into nitty gritty graphics ideas and associated draw calls. I figured the best way was to try to build it myself from scratch!

So I took iron to the anvil and decided to reimplement Core Animation in near-pure Swift -- that is, avoiding NSObject/Cocoa wherever possible. I didn't attempt to implement the entire API surface, such as CAAtom, or the layout managers API (for that matter, layout managers have been remarked by Core Animation engineers as a bad idea anyways). I chose to design a closely matched, but overall Swift-ier API, such as Render.Value. I haven't yet, however, implemented wrappers for all Codable types, partially because Core Animation is implemented in C++ and some internal types such as CA::Render::Vector are actually wrappers around float[], for example. Implementing serialization for a [Float] in Swift is free, so we don't necessarily need these wrapper types. Some facilities like fences to synchronize rendering between threads or processes, or slots to share drawable items (if I recall correctly), aren't implemented yet simply because they're not required in DIYAnimation yet. Ultimately, only the Objective-C CA* and the internal CA::Render::* API surfaces are the same, as I've reimplemented the actual rendering (CA::OGL::*) in an entirely different way atop Metal.

Another major departure from the C++ implementation is the use of XPC serialization and transport over raw mach_msg types and calls. While XPC is best known for its service and connection facilities, DIYAnimation uses the xpc_pipe_t object atop raw mach ports, a transport medium for serialized XPC objects to be passed between threads and processes which underlies xpc_connection_t. After writing an XPC encoder and decoder pair in Swift, the XPCPipe wrapper works with any Codable with some notable exceptions involving patched implementations. Though I've got a mach shared memory object in Swift matching the original implementation, I'm instead using IOSurface in a few places simply because it's easier to work with at the moment. I'll talk a bit more about the XPC internals later, but for now I'll share a few tidbits I came across while implementing parts of DIYAnimation.

Since Core Animation doesn't deal with accelerated 2D graphics (i.e. the now-defunct QuartzGL from Core Graphics), text rendering, for example, is CPU-bound -- it's implemented in an identical manner in DIYAnimation. However, while QuartzGL is indeed defunct, Core Animation does actually indirectly support accelerated drawing! To make a very long story short, non-accelerated/CPU-bound drawing is performed through CGContext from Core Graphics, which is actually just a wrapper around CGGState, which manages the transparency layers and graphics state stack, and CGContextDelegate, which handles conversion of Quartz draw calls into actual rasterization. (As a side-side-note, CGLayers and transparency layers are essentially identical, but capture context state at different points in the drawing lifecycle.) CAWindowContextDelegate is the delegate used by all CGContexts drawing to a Cocoa window, and CAIOSurfaceContextDelegate is the delegate used by all CGContexts attached to a CALayer that's currently accelerating drawing within -drawInContext:. It accelerates drawing by buffering its queued draw calls until its associated layer needs to draw to screen, and subsequently thereafter, its associated IOSurface needs to draw to screen.

I've attempted to replicate window server functionality as seen in CAWindowServer and CADisplay, using CGS windows (CoreGraphicsServices, now known as SkyLight in macOS 11+), though it's done somewhat strangely at the moment. Not only does SkyLight on macOS handle event routing and layer compositing like Core Animation on iOS, but also desktop management, spaces, the menu bar, cursors, and controlling the dock, and more on macOS. That's a lot of additional functionality that I'll be ignoring. In AppKit, for example, invoking the modern drag and drop API eventually calls into HIServices, the vestigial Carbon.framework remnant underlying most of AppKit, which then calls into the CoreDrag.framework, which then creates a new CGSWindow with the SkyLight window server, a new CAContext to go with it (therefore enabling Core Animation embedding in that window), and a new root CALayer. That layer is then what you see while dragging a file or anything else. The actual movement of data between one process to another (NSPasteboard in AppKit) is actually implemented using CFMessagePort in HIServices - every AppKit-linked process has this port pair allocated for drag-and-drop or other pasteboard synchronization.

Speaking of CGSWindow and CAContext, by the way, these two API are actually somewhat identical in some ways. AppKit's NSWindow wraps a CGSWindow, and supports on-screen drawing through providing a CGContext to its children NSViews. System events are passed from the SkyLight window server to the most appropriate CGSWindow (through a callback into the application's run loop) which then is translated and passed to the appropriate NSView by that NSWindow. The connection to the window server, CGSConnection is actually explicit in all these API, and must be created by AppKit for each thread that wants to manage on-screen elements. Each connection is recorded by the server in an identifier-based lookup table, and upon each call to the server, its originating connection's rights are validated for each resource modified or accessed by that call. SkyLight actually uses Mach Interface Generator (MIG) to marshall these calls, and connections are actually just composed of a server request mach port and an on-demand client notification/event port. In Core Animation, however, CAContext doesn't handle any drawing or rendering control itself, instead delegating this to its root CALayer. It does control inter-context ordering (i.e. ordering root layers relative to all other on-screen root layers) and event routing when initialized connected to a remote window server instead of initialized as a local rendering context. So, CA windows (contexts) are equivalent to SkyLight windows, but while one SkyLight connection can control many windows, one CA connection may only control one CA window (context). In a way though, for quite some time now, AppKit internal/private API has slowly been modernized to match Core Animation, with functions like NSWindowMark mimicking CALayerMark along with the recent addition of transactions. (With CGPixelAccess and direct CGContext drawing, SkyLight actually has an analogous concept to CABackingStore; CA::Shape too is essentially the same as CGSRegionObj/CGRegion.)

Anyways, the next steps for this project include adding CADynamicsBehavior (probably hard, requiring box2d or PhysicsKit integration), CAStateController (likely easy), CAPresentationModifier (likely easy), and CAML and CAPackage support (likely easy... if using NSXMLParser). Though they've since been removed from Core Animation, CALight and related lighting filters did used to exist, roughly matching SVG's feDiffuseLighting and feSpecularLighting, for example. They were meant to be used with CAMeshTransform, which is also currently unimplemented. In summary, as you can see, there's a lot of private and some public API that are yet unimplemented, even though the groundwork has been laid. Take a look at the implementation details -- I'm sure you'll find them interesting, as they're mostly faithful to the original (decompiled) C++ implementation!