The Omni Group

Update: I've added some comments about the -[CALayer shadowPath] approach that I missed before.

I've just pushed a new sample to the OmniGroup? source on github which shows several drop shadow approaches, with an eye towards performance, particularly while the shadow casting object's frame is being animated. One of these is a pair of functions vended by OmniUI, OUIViewAddShadowEdges?() and OUIViewLayoutShadowEdges?(), which originate from writing our document picker. In this case we want to be able to smoothly animate between two apparent view sizes as you open and close documents.

Opening up OmniGroup/Frameworks/OmniUI/iPad/Examples/DropShadowOptions? and building, you'll see a set of options that control the testing:

Animation Type

• Resize: The size of the object changes, possibly leading to it being redrawn.

• Slide: The object simply move, which may allow optimized screen rendering by compositing a previously cached backing store.

Animation Driver

• One-time change: Some one-time programatic change is being made to the object that has a starting and ending state. These are driven by UIView animation.

• User interaction: This simulates multiple successive small changes, as if the user were performing a sequence of dragging touches (like resizing a shape). These are driven by a timer, simulating events from the user.

Shadow Options

• CoreGraphics, resampled: This renders the shadow into the view's content area with normal CoreGraphics calls. When the view's size changes, the content is not redrawn, but the previously created backing store is just scaled. If the change you are making is temporary (like a small zoom in/out bounce animation), this could be imperceptible to the user. One problem with the CoreGraphics-based approaches is that some of your content area is taken up by the shadow and positioning elements within your view needs to take this into account.

• CoreGraphics, redrawn: The shadow is drawn into the backing store, as above. But, each time the view's bounds changes, a new backing store is generated. Note however, that UIView animations only generate a backing store image for the starting and ending frame and interpolate between them. There is not a new image generated for each frame of animation. This yields good performance for one-time, animations.

• CALayer Shadow: CALayer provides shadowing options with the shadows being cast outside your content. This makes performing other geometry calculations simpler (like positioning sublayers) since you can turn on the shadow and forget about it.

• CALayer Shadow, rasterized: Normally, CALayer shadows are built at the time that the layer is composited to screen. That means, each time you even move a layer, the full shadow needs to be recomputed. But, CALayer provides an option to request rasterization, which is enabled in this case.

• CALayer, shadow path:(not shown in screenshot above since this was added to the test later) This uses the shadowPath property on CALayer, which improves the performance of shadow rendering by allowing the layer to assume that the interior of the path is opaque (rather than having to convolve the alpha channel of the layer's contents). Sadly, UIView seems to disable implicit animations for this property, and you need to be careful to share or reuse CGPathRefs. With this extra bookkeeping, it is a bit of a pain to use, but nice and speedy.

• OmniUI Shadow Edge Views: Given a UIView, these OmniUI functions add four thin views along the edges of your view. Each has a three-part stretchable image that renders the shadow. This takes advantage of the CALayer contentsCenter property to avoid even needing to re-fill the shadow edge contents on a resize. Like the CALayer approach, these views lie outside the view itself, simplifying the positioning of the content within the view (or other geometry calculations, in our case, calculating the animation parameters when opening or closing documents).

Performance

Running this app under Instruments with the Core Animation tool, we can check out the relative advantages of each approach. Instruments doesn't give a precise frame rate over time, but really we don't care. In the real world, you'd have other work to do during a live-resize of an object, so anything here less than 60fps (for just the shadowing) is likely too slow.

I'll call out the one "Slowish" result; this timed at ~45fps on my iPad. CoreGraphics shadowing is amazingly fast — nice work. Still, depending on the rest of your workload during the resize, "pretty fast" may not be good enough.

Conclusions

The default behavior of UIView/CALayer is great for simple situations. If, however, a user is dynamically interacting with an object that casts shadows and you want that object to not get blurry as it resizes, CoreGraphics may not be as fast as you'd like (obviously the rasterization or relayout of the inner content matters too).

Don't turn on CALayer shadows unless either enable rasterization on and aren't going to be changing the view's size, or are willing to do the extra bookkeeping to maintain a shadowPath. Generic CALayer shadows are stunningly slow if all you need is a rectangle, but the shadowPath hint makes them very zippy.

Finally, what does the OmniUI shadow edges approach tell us? This particular hack may not be useful in your app, but the core idea is that you shouldn't do work you don't need to do. When resizing axially aligned, opaque objects, we don't need a fully general shadowing algorithm.

We can deceive the user into thinking a real shadowed object is floating in their view, but we don't have to implement it that way. On a "magical" device like an iPad, you need to act like a magician: as long as it looks like something visually complicated is really happening, then your job as illusionist is complete.

We've pushed new versions of our public framework source to github, featuring a bunch of work from our iPad apps, OmniGraffle and OmniGraphSketcher. Some of the highlights include our document picker, inspectors, a bunch of controls, and a start on a CoreText-based text editor, but there is much more.

For a quick iPad example, open up OmniGroup/Frameworks/OmniUI/iPad/Examples/TextEditor and build it. The text editing bit of this sample is a little rough, but this shows off some of the document picker.

Even if you aren't interested in adopting the frameworks directly, there are a bunch of little gems that you can use. One such feature is our Objective-C runtime sanity checker (in OmniGroup/Frameworks/OmniBase/OBRuntimeCheck.m).

The runtime checker helps avoid problems with mismatched method signatures between classes and their superclasses and between classes and conformed protocols. These can creep in as ABI changes from 32- to 64-bit support is added, or as subtle shifts in API happen in the system frameworks or your own class hierarchies. Some of these clang will check for you, but not all.

For example, having built OmniBase (with the Debug configuration, which enables this checking support) we can run Xcode with checking enabled (not to pick on Xcode, lots of apps have these problems):

% OBPerformRuntimeChecksOnLoad=1 DYLD_INSERT_LIBRARIES=/Users/Shared/bungi/Source/PROJ/Products/Debug/OmniBase.framework/OmniBase /Developer/Applications/Xcode.app/Contents/MacOS/Xcode?

...
2010-05-13 13:36:10.205 Xcode[19717:903] Method scriptWorkingDirectory has conflicting type signatures between class and its superclass:
signature i16@0:8 for class XCUserScript has imp -[XCUserScript scriptWorkingDirectory] at 0x100874d40 in DevToolsInterface.framework
signature @16@0:8 for class XCUserScriptNode has imp -[XCUserScriptNode scriptWorkingDirectory] at 0x100c8f870 in DevToolsInterface.framework
?...

?

Here we can see that a method's signature has changed — in one case it is returning an integer and in the other an object. Probably not a good sign.

By default, we silence problems where both sides of the conflict are in system frameworks (since we can't do anything about them). We've logged Radars on the in-system conflicts, but if you'd like to see them, you can also define OBReportWarningsInSystemLibraries in the environment:

2010-05-13 15:19:05.652 Xcode[21608:903] Method hash has conflicting type signatures between class and its superclass:

signature I16@0:8 for class CIVector has imp -[CIVector hash] at 0x7fff81e27e84 in QuartzCore.framework
signature Q16@0:8 for class NSObject has imp -[NSObject(NSObject) hash] at 0x7fff84862e80 in CoreFoundation.framework

We're looking for one or two software development interns for this summer! Check out our open positions for more info on internships or full-time employment.

Out of the box, CALayer supports many animatable properties where a change will automatically create a CAAnimation for that property. But, if you look at the list of properties, it is clear that they are all of the form “stuff that OpenGL does while compositing” and have nothing to do with the inner texture representing your content. You are entirely responsible for telling CoreAnimation when your content has changed.

Recently I wanted to write a layer that had its content animate, but as far as I could determine CALayer doesn't support this out of the box, though it is really pretty close. If you add an @dynamic property to a CALayer subclass and set it, your layer will be sent -actionForKey: and the returned animation will be passed to your -addAnimation:forKey:.

The problem is that CALayer doesn't know that this change means it needs to update your contents property too, so the animation will happen but no display change will happen.

The API on CALayer really isn't sufficient for this task, so it isn't too surprising it isn't supported. Since CALayer is a generic KVC container you can squirrel away random keys/value pairs in it and use them for whatever purposes you want. But it has no way of knowing that a particular property change should provoke an update to the content, and updating the content unnecessarily would seriously hurt performance.

I've whipped up some sample code with a superclass that shows one approach to handling this.

The sample subclass looks like:

@interface WaveLayer : ContentAnimatingLayer


@property CGFloat phase, frequency, amplitude;

@end


@implementation WaveLayer

@dynamic phase, frequency, amplitude;


+ (NSSet *)keyPathsForValuesAffectingContent;

{

    static NSSet *keys = nil;

    if (!keys)

        keys = [[NSSet alloc] initWithObjects:@"phase",

                   @"frequency", @"amplitude", nil];

    return keys;

}


- (void)drawInContext:(CGContextRef)ctx;

{

    … draw a sine wave …

}

@end

The ContentAnimatingLayer superclass provides some bookkeeping and basic support for updating the content when the content-affecting properties are changed. First, it adds support for -actionFor; like many other key-based Cocoa protocols (Radar #6372335). On top of this, it adds support for determining which properties are content-affecting with +keyPathsForValuesAffectingContent and -isContentAnimation:. Active content-affecting animations are then tracked and an update timer is fired when there is at least one such animation.

This makes it as easy as I'd hoped it would be to write content animating layers; hopefully you'll find this useful too! I've asked Apple to add this to CALayer in the future. If you'd like it too, you can reference my Radar #6372372.

The sample code is just that, my first cut at a sample. Some possible issues/improvements:

• The -actionFor support may or may not be fast enough (or really not even necessary).
• Instead of following the class-based KVO customization pattern, CALayers are often customized by the delegate. So, the NSSet-returning method should maybe be an instance method or maybe the -isContentAnimation: should be split up and call a new -layer:isContentAnimatingKey: delegate method.
• It would be nicer to tie into the normal CoreAnimation timing mechanism instead of creating a per-instance NSTimer. Even without that, it might be a bit more efficient to have a single animation timer for all instances of ContentAnimatingLayer. Also, right now the timer doesn't get registered for the event tracking runloop mode, though it maybe should.
• This approach assumes that a change to a content-affecting key requires the entire content be redrawn. This may or may not be the case for any particular layer. A change to a content-animating property might want to specify an NSRect-based animation that describes the dirty area.

Now that the iPhone NDA is being lifted, we can share a few of the lessons we've learned while working with the SDK.

Since OmniFocus makes extensive use of the Omni Frameworks, one of the very first challenges we hit when starting with the SDK was how to structure our source and Xcode projects so that we could re-use some of our battle-tested common code between OmniFocus on the Mac and on the iPhone. Some people like splitting up their source into frameworks and some don't. That's all good, but on the iPhone 3rd party developers have no supported way to build their own frameworks.

Many limitations also come with a benefit, and in this case it is smaller application packages. A framework may include classes or categories that are used in several clients, but not all of them. On the iPhone, we want fast downloads from the App Store (possibly over the cell network) and we want fast startup times. Both of these require us to not bloat our executable with code not specifically used by our app.

Given that we can't use real frameworks bundles, we need to directly include our framework source in our iPhone project. I'll describe an approach that's worked well for us and might suit you too.

First, we want to be sure that we don't pick up any extra headers that we didn't intend to use. For example, if our OmniFoundation NSSet category imports another extension header, we want to be sure that we consider that change rather than just letting the dependency creep in (if nothing else, we need to build the corresponding .m file). One approach that helps with this is to use <...> instead of "..." imports, so we have:

#import 

This ensures that Xcode finds our header from a “public” location rather than finding it relative to the file being compiled. Now, if we do have an internal header that shouldn't be published, we can leave it in an "..." import, but we have to take care to add those .m files to the target.

Second, we want to “publish” the headers for the public headers necessary to build the subset of the framework we'll be using. We accomplish this with a sequence of Build Phases in our OmniFocus target in Xcode:

The first interesting phase, Create Header Directories, gives us a place in our build area for the headers:

FRAMEWORK_HEADER_BASE="$DERIVED_SOURCES_DIR/FrameworkHeaders"

mkdir -p "$FRAMEWORK_HEADER_BASE"

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniBase

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFoundation

mkdir -p "$FRAMEWORK_HEADER_BASE"/XMLData

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFocusModel

mkdir -p "$FRAMEWORK_HEADER_BASE"/OODataTypes

mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniDataObjects

Then, for each “framework” we have a Copy Files build phase that is set to copy to the header directory in question. For example, Copy OmniBase Headers looks like:

Finally, we need to let Xcode find the headers when it builds our target. Opening the Target Info editor for our iPhone app, we can double-click the Header Search Path entry and add the top level FrameworkHeaders directory:

Now we can add just the framework headers and source files we need to our iPhone project. Each “public” header can then be dragged into the appropriate Copy Files build phase for installation during builds.

This isn't the end of the road, but it is a good start. Other issues involve making sure that your NSError domain strings aren't dependent on your bundle identifier (since on the Mac they'll be in different bundles and on the iPhone not), resource location (again, due to the one bundle vs. many difference on the platforms), and generating strings files when your sources are spread around (there are a couple scripts floating around for doing this; once we clean ours up we'll hopefully publish it).

Dave Dribin posted a nice article on Proper Key-Value Observer Usage that you should definitely check out. Something he mentioned in passing has been bugging me for a while:

You could check the object that’s passed in as well. Unfortunately, this doesn’t help the case where a subclass or superclass has registered a notification on the same key path of the same object.

If, however, you specify something unique for the context, you can use that to correctly identify your notifications.

Proper use of the KVO context will allow a single object to subscribe to a key across a parent and child class in some common cases, but there is a hidden danger. If your two classes both follow the basic pattern of adding an observation when your object is created and removing it when your object is dying, all will be well. But if you have a class that may toggle its observation on and off based on some varying criteria, you can get in trouble.

The core of the problem is an API flaw in KVO; -removeObserver:forKeyPath: doesn't include a context argument. So, if you have an object that is subscribed to a key twice, with two different context pointers, there is no way to specify which one is to be removed. If your object is dying and cleaning up all observations, this isn't a big deal. But if your subclass logic wants to temporarily turn off its observation, it has no way to make sure that's what happens — it might instead end up removing the superclass observation.

Now, granted this is a relatively rare case that can be accounted for in your design. But, this takes an otherwise general and extremely useful API and adds a bit of worry to it every time you use it. You can't use it locally in a class without worrying about what your super- and subclasses are doing.

I've written up Radar 6244260 on this, including a test case. Hopefully they'll add a proper -removeObserver:forKeyPath:context: and deprecate the current method for some future release.

Bill Bumgarner has posted a few great articles on Objective-C method tracing using dtrace:

• Objective-C: Printing Class Name from Dtrace
• Objective-C: Using Instruments to trace messages-to-nil

I have a internal tool at Omni that traces message sends by creating new method IMPs to wrap the existing ones in a trampoline, but it requires some nasty assembly stubs and has only ever worked on Sparc, ppc and x86. It is very fast, but hard to maintain and a little fragile (and doesn't handle the nil case). A 90% solution can be obtained by extending Bill and other's work.

Ken sent out a partial tracing solution this morning and finally got me off my keister. Here is a D script that catches ObjC message sends and dumps a OPML fragment (which can then be wrapped in the XML goo via an external shell script or whatever) and then viewed in OmniOutliner.

Download: objc-trace-opml.d.gz

This isn't a perfect solution by any means. Some of the issues:

• This has to be run in an essentially single-threaded program; any background threads must be totally quiescent or you'll get a mixture of output from multiple threads. It wouldn't be too hard to modify this script to include a thread identifier or maybe just filter out anything from background threads.
• This won't handle exceptions correctly; they'll break the OPML nesting. Our internal tool handles this, but since NSError arrived on the scene, this is less of an issue. Just avoid tracing code with exceptions.
• dtrace is slow when matching all entries in the objc provider. The startup time on this script is something like 40 seconds on my machine. Annoying, but not the worst thing ever, since you are likely to trace once and then spend a fair bit of time examining the results.
• This particular variant of the script will only work on x86 due to the hacky way I check for objc_msgSend_stret.
• dtrace can overflow its buffer and drop events when there are massive numbers of probe hits. Run with a big buffer and trace only the exact set of methods you need. Ideally you'll be running in the debugger, stop right before where you need to trace turn on tracing and then 'next' over one or two lines of code. The '-b' flag to dtrace can also be used to increase the buffer size.

Some of the nice bits:

• No assembly-fu required.
• Call hierarchy is preserved and can be expanded/hoisted and such in OmniOutliner.
• Both the class of the receiver and the class of the method implementation are included. So, you can see that +[MySuperclass initialize] is getting called on MySubcass, for example.
• All method invocations are shown, so you can see nested calls to super instead of just the initial call to objc_msgSend. I've not tested whether cached method IMPs get traced too; normal code should get traced as expected. Swizzling, IMP caching or dynamically registering methods will possibly not work as well.
• The pointer value of the receiver is emitted. By the time you examine the output, it might be dead and gone, but a surprising amount of the time it isn't. For example, things like static NSStrings, NSScriptClassDescriptions, interface widgets and other cache-once data may still be around for submitting to 'po' in the debugger.

We finally have space to grow, now that we've moved to our new office! There is no specific position to fill, but rather we are content to wait until we find someone who will fit well into our team. Take a look at what working at Omni is like and the general description of who we are seeking to see if this is the right place for you!

I failed to find any way to do this via the internets; but gdb, and some kindly soul inside Apple, met the challenge:

(gdb) b -[NSUserDefaults objectForKey:]

Breakpoint 17 at 0x94172fa4

(gdb) c

Continuing.

... invoke service, hit breakpoint ...

(gdb) po *(id *)($fp + 16)

NSShouldActivateForServiceRequest

(gdb)    

Is there a better way to do this? Google and mdls don't seem to know about NSShouldActivateForServiceRequest at all, so I'm guessing not.

Page 1 of 2 pages  1 2 >