iPad drop shadow performance

by Tim Wood on May 21, 2010

Update: I've added some comments about the -[CALayer shadowPath] approach that I missed before.

I've just pushed a new sample to the OmniGroup? source on github which shows several drop shadow approaches, with an eye towards performance, particularly while the shadow casting object's frame is being animated. One of these is a pair of functions vended by OmniUI, OUIViewAddShadowEdges?() and OUIViewLayoutShadowEdges?(), which originate from writing our document picker. In this case we want to be able to smoothly animate between two apparent view sizes as you open and close documents.

Opening up OmniGroup/Frameworks/OmniUI/iPad/Examples/DropShadowOptions? and building, you'll see a set of options that control the testing:

 

DropShadowOptions.png

 

Animation Type

  • Resize: The size of the object changes, possibly leading to it being redrawn.

  • Slide: The object simply move, which may allow optimized screen rendering by compositing a previously cached backing store.

 

Animation Driver

  • One-time change: Some one-time programatic change is being made to the object that has a starting and ending state. These are driven by UIView animation.

  • User interaction: This simulates multiple successive small changes, as if the user were performing a sequence of dragging touches (like resizing a shape). These are driven by a timer, simulating events from the user.

 

Shadow Options

  • CoreGraphics, resampled: This renders the shadow into the view's content area with normal CoreGraphics calls. When the view's size changes, the content is not redrawn, but the previously created backing store is just scaled. If the change you are making is temporary (like a small zoom in/out bounce animation), this could be imperceptible to the user. One problem with the CoreGraphics-based approaches is that some of your content area is taken up by the shadow and positioning elements within your view needs to take this into account.

  • CoreGraphics, redrawn: The shadow is drawn into the backing store, as above. But, each time the view's bounds changes, a new backing store is generated. Note however, that UIView animations only generate a backing store image for the starting and ending frame and interpolate between them. There is not a new image generated for each frame of animation. This yields good performance for one-time, animations.

  • CALayer Shadow: CALayer provides shadowing options with the shadows being cast outside your content. This makes performing other geometry calculations simpler (like positioning sublayers) since you can turn on the shadow and forget about it.

  • CALayer Shadow, rasterized: Normally, CALayer shadows are built at the time that the layer is composited to screen. That means, each time you even move a layer, the full shadow needs to be recomputed. But, CALayer provides an option to request rasterization, which is enabled in this case.

  • CALayer, shadow path:(not shown in screenshot above since this was added to the test later) This uses the shadowPath property on CALayer, which improves the performance of shadow rendering by allowing the layer to assume that the interior of the path is opaque (rather than having to convolve the alpha channel of the layer's contents). Sadly, UIView seems to disable implicit animations for this property, and you need to be careful to share or reuse CGPathRefs. With this extra bookkeeping, it is a bit of a pain to use, but nice and speedy.

  • OmniUI Shadow Edge Views: Given a UIView, these OmniUI functions add four thin views along the edges of your view. Each has a three-part stretchable image that renders the shadow. This takes advantage of the CALayer contentsCenter property to avoid even needing to re-fill the shadow edge contents on a resize. Like the CALayer approach, these views lie outside the view itself, simplifying the positioning of the content within the view (or other geometry calculations, in our case, calculating the animation parameters when opening or closing documents).

 

Performance

Running this app under Instruments with the Core Animation tool, we can check out the relative advantages of each approach. Instruments doesn't give a precise frame rate over time, but really we don't care. In the real world, you'd have other work to do during a live-resize of an object, so anything here less than 60fps (for just the shadowing) is likely too slow.

 

 

Resize/Once

Slide/Once

Resize/User

Slide/User

CoreGraphics, resampled

Fast

Fast

Fast

Fast

CoreGraphics, redrawn

Fast

Fast

Slowish

Fast

CALayer Shadow

Slow

Slow

Slow

Slow

CALayer Shadow, rasterized

Slow

Fast

Slow

Fast

CALayer, shadow path

Fast

Fast

Fast

Fast

OmniUI Shadow Edge Views

Fast

Fast

Fast

Fast

 

I'll call out the one "Slowish" result; this timed at ~45fps on my iPad. CoreGraphics shadowing is amazingly fast — nice work. Still, depending on the rest of your workload during the resize, "pretty fast" may not be good enough.

 

Conclusions

The default behavior of UIView/CALayer is great for simple situations. If, however, a user is dynamically interacting with an object that casts shadows and you want that object to not get blurry as it resizes, CoreGraphics may not be as fast as you'd like (obviously the rasterization or relayout of the inner content matters too).

Don't turn on CALayer shadows unless either enable rasterization on and aren't going to be changing the view's size, or are willing to do the extra bookkeeping to maintain a shadowPath. Generic CALayer shadows are stunningly slow if all you need is a rectangle, but the shadowPath hint makes them very zippy.

Finally, what does the OmniUI shadow edges approach tell us? This particular hack may not be useful in your app, but the core idea is that you shouldn't do work you don't need to do. When resizing axially aligned, opaque objects, we don't need a fully general shadowing algorithm.

We can deceive the user into thinking a real shadowed object is floating in their view, but we don't have to implement it that way. On a "magical" device like an iPad, you need to act like a magician: as long as it looks like something visually complicated is really happening, then your job as illusionist is complete.

 

?

New Omni framework source release

by Tim Wood on May 13, 2010

We've pushed new versions of our public framework source to github, featuring a bunch of work from our iPad apps, OmniGraffle and OmniGraphSketcher. Some of the highlights include our document picker, inspectors, a bunch of controls, and a start on a CoreText-based text editor, but there is much more.

For a quick iPad example, open up OmniGroup/Frameworks/OmniUI/iPad/Examples/TextEditor and build it. The text editing bit of this sample is a little rough, but this shows off some of the document picker.

Even if you aren't interested in adopting the frameworks directly, there are a bunch of little gems that you can use. One such feature is our Objective-C runtime sanity checker (in OmniGroup/Frameworks/OmniBase/OBRuntimeCheck.m).

The runtime checker helps avoid problems with mismatched method signatures between classes and their superclasses and between classes and conformed protocols. These can creep in as ABI changes from 32- to 64-bit support is added, or as subtle shifts in API happen in the system frameworks or your own class hierarchies. Some of these clang will check for you, but not all.

For example, having built OmniBase (with the Debug configuration, which enables this checking support) we can run Xcode with checking enabled (not to pick on Xcode, lots of apps have these problems):

% OBPerformRuntimeChecksOnLoad=1 DYLD_INSERT_LIBRARIES=/Users/Shared/bungi/Source/PROJ/Products/Debug/OmniBase.framework/OmniBase /Developer/Applications/Xcode.app/Contents/MacOS/Xcode?

... <br /> 2010-05-13 13:36:10.205 Xcode[19717:903] Method scriptWorkingDirectory has conflicting type signatures between class and its superclass:<br /> signature i16@0:8 for class XCUserScript has imp -[XCUserScript scriptWorkingDirectory] at 0x100874d40 in DevToolsInterface.framework<br /> signature @16@0:8 for class XCUserScriptNode has imp -[XCUserScriptNode scriptWorkingDirectory] at 0x100c8f870 in DevToolsInterface.framework<br /> ?...

?

Here we can see that a method's signature has changed — in one case it is returning an integer and in the other an object. Probably not a good sign.

By default, we silence problems where both sides of the conflict are in system frameworks (since we can't do anything about them). We've logged Radars on the in-system conflicts, but if you'd like to see them, you can also define OBReportWarningsInSystemLibraries in the environment:

2010-05-13 15:19:05.652 Xcode[21608:903] Method hash has conflicting type signatures between class and its superclass:

signature I16@0:8 for class CIVector has imp -[CIVector hash] at 0x7fff81e27e84 in QuartzCore.framework<br /> signature Q16@0:8 for class NSObject has imp -[NSObject(NSObject) hash] at 0x7fff84862e80 in CoreFoundation.framework

Intern at Omni!

by Tim Wood on February 6, 2009

We're looking for one or two software development interns for this summer! Check out our open positions for more info on internships or full-time employment.

Animating CALayer content

by Tim Wood on November 14, 2008

Out of the box, CALayer supports many animatable properties where a change will automatically create a CAAnimation for that property. But, if you look at the list of properties, it is clear that they are all of the form “stuff that OpenGL does while compositing” and have nothing to do with the inner texture representing your content. You are entirely responsible for telling CoreAnimation when your content has changed.

Recently I wanted to write a layer that had its content animate, but as far as I could determine CALayer doesn't support this out of the box, though it is really pretty close. If you add an @dynamic property to a CALayer subclass and set it, your layer will be sent -actionForKey: and the returned animation will be passed to your -addAnimation:forKey:.

The problem is that CALayer doesn't know that this change means it needs to update your contents property too, so the animation will happen but no display change will happen.

The API on CALayer really isn't sufficient for this task, so it isn't too surprising it isn't supported. Since CALayer is a generic KVC container you can squirrel away random keys/value pairs in it and use them for whatever purposes you want. But it has no way of knowing that a particular property change should provoke an update to the content, and updating the content unnecessarily would seriously hurt performance.

I've whipped up some sample code with a superclass that shows one approach to handling this.

The sample subclass looks like:

@interface WaveLayer : ContentAnimatingLayer

@property CGFloat phase, frequency, amplitude;

@end

@implementation WaveLayer

@dynamic phase, frequency, amplitude;

+ (NSSet *)keyPathsForValuesAffectingContent;

{

static NSSet *keys = nil;

if (!keys)

keys = [[NSSet alloc] initWithObjects:@"phase",

@"frequency", @"amplitude", nil];

return keys;

}

- (void)drawInContext:(CGContextRef)ctx;

{

… draw a sine wave …

}

@end

 

The ContentAnimatingLayer superclass provides some bookkeeping and basic support for updating the content when the content-affecting properties are changed. First, it adds support for -actionFor; like many other key-based Cocoa protocols (Radar #6372335). On top of this, it adds support for determining which properties are content-affecting with +keyPathsForValuesAffectingContent and -isContentAnimation:. Active content-affecting animations are then tracked and an update timer is fired when there is at least one such animation.

This makes it as easy as I'd hoped it would be to write content animating layers; hopefully you'll find this useful too! I've asked Apple to add this to CALayer in the future. If you'd like it too, you can reference my Radar #6372372.

The sample code is just that, my first cut at a sample. Some possible issues/improvements:

  • The -actionFor support may or may not be fast enough (or really not even necessary).
  • Instead of following the class-based KVO customization pattern, CALayers are often customized by the delegate. So, the NSSet-returning method should maybe be an instance method or maybe the -isContentAnimation: should be split up and call a new -layer:isContentAnimatingKey: delegate method.
  • It would be nicer to tie into the normal CoreAnimation timing mechanism instead of creating a per-instance NSTimer. Even without that, it might be a bit more efficient to have a single animation timer for all instances of ContentAnimatingLayer. Also, right now the timer doesn't get registered for the event tracking runloop mode, though it maybe should.
  • This approach assumes that a change to a content-affecting key requires the entire content be redrawn. This may or may not be the case for any particular layer. A change to a content-animating property might want to specify an NSRect-based animation that describes the dirty area.

Using frameworks in iPhone applications

by Tim Wood on October 1, 2008

Now that the iPhone NDA is being lifted, we can share a few of the lessons we've learned while working with the SDK.

Since OmniFocus makes extensive use of the Omni Frameworks, one of the very first challenges we hit when starting with the SDK was how to structure our source and Xcode projects so that we could re-use some of our battle-tested common code between OmniFocus on the Mac and on the iPhone. Some people like splitting up their source into frameworks and some don't. That's all good, but on the iPhone 3rd party developers have no supported way to build their own frameworks.

Many limitations also come with a benefit, and in this case it is smaller application packages. A framework may include classes or categories that are used in several clients, but not all of them. On the iPhone, we want fast downloads from the App Store (possibly over the cell network) and we want fast startup times. Both of these require us to not bloat our executable with code not specifically used by our app.

Given that we can't use real frameworks bundles, we need to directly include our framework source in our iPhone project. I'll describe an approach that's worked well for us and might suit you too.

First, we want to be sure that we don't pick up any extra headers that we didn't intend to use. For example, if our OmniFoundation NSSet category imports another extension header, we want to be sure that we consider that change rather than just letting the dependency creep in (if nothing else, we need to build the corresponding .m file). One approach that helps with this is to use <…> instead of “…” imports, so we have:

#import

This ensures that Xcode finds our header from a “public” location rather than finding it relative to the file being compiled. Now, if we do have an internal header that shouldn't be published, we can leave it in an “…” import, but we have to take care to add those .m files to the target.

Second, we want to “publish” the headers for the public headers necessary to build the subset of the framework we'll be using. We accomplish this with a sequence of Build Phases in our OmniFocus target in Xcode:

OmniFocus for iPhone Target List.png


The first interesting phase, Create Header Directories, gives us a place in our build area for the headers:

FRAMEWORK_HEADER_BASE="$DERIVED_SOURCES_DIR/FrameworkHeaders"

mkdir -p "$FRAMEWORK_HEADER_BASE"

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniBase

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFoundation

mkdir -p "$FRAMEWORK_HEADER_BASE"/XMLData

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFocusModel

mkdir -p "$FRAMEWORK_HEADER_BASE"/OODataTypes

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniDataObjects


Then, for each “framework” we have a Copy Files build phase that is set to copy to the header directory in question. For example, Copy OmniBase Headers looks like:

Copy OmniBase Headers.png


Finally, we need to let Xcode find the headers when it builds our target. Opening the Target Info editor for our iPhone app, we can double-click the Header Search Path entry and add the top level FrameworkHeaders directory:

Header Search Path.png


Now we can add just the framework headers and source files we need to our iPhone project. Each “public” header can then be dragged into the appropriate Copy Files build phase for installation during builds.

This isn't the end of the road, but it is a good start. Other issues involve making sure that your NSError domain strings aren't dependent on your bundle identifier (since on the Mac they'll be in different bundles and on the iPhone not), resource location (again, due to the one bundle vs. many difference on the platforms), and generating strings files when your sources are spread around (there are a couple scripts floating around for doing this; once we clean ours up we'll hopefully publish it).