The challenge of syncing OmniOutliner

by Ken Case on April 16, 2012

Hi, all!

It’s been almost one year since we shipped OmniOutliner for iPad. Every day since we shipped we’ve received feedback along the lines of “This looks great! But when will it automatically sync with OmniOutliner for Mac?”—and each time we reply back “We’re working on it!” But it’s been almost a year now, and I know you all must be wondering: Why is it taking so long to add automatic syncing to OmniOutliner?

To make documents with large embedded attachments more efficient, OmniOutliner 3 stores its documents as “file wrappers”, which behind the scenes are simply directories holding separate files which represent different parts of the document. (The outline itself is one file, and each attachment is its own separate file.) Unfortunately, most sync services don’t support syncing a directory as an atomic operation.

Wait, hold on, what does “atomic operation” mean? Well, in computer speak, an atomic operation is something that gets done all at once. In this case, we want all of the parts of the document to get synchronized to or from the cloud at the same time, otherwise different parts of the document might not be in sync with itself: it might be missing some attachments, or perhaps some of them will be out of date, or maybe there will be some extra attachments that shouldn’t be there.

In other words, when syncing OmniOutliner, it’s not just important to have all the parts eventually arrive on the other end; we want all of the parts of the document to show up on the other end at the very same time (and we want to know when they’ve all finished arriving) so that we know it’s safe to open and you won’t end up with a corrupt document.

One way to solve this syncing challenge is to change our document format—to stop storing OmniOutliner documents as directories behind the scenes. This change would make it much less efficient for attachments, but would also make it much more compatible with many file servers, email, web forms, source control systems, and so on. We can do that—but such a big change to the OmniOutliner file format won’t work with the currently shipping apps, and doesn’t really make sense to do before shipping OmniOutliner 4. So taking this route to solving the problem means for us to try to ship OmniOutliner 4 as soon as possible (which we’re certainly working very hard to do).

Another way to solve this problem is to help syncing solution providers (such as DropBox and iCloud) improve their support for atomic directory operations so we can sync using their solutions. The most promising option here by far is iCloud, since iCloud’s engineers explicitly do want to support syncing of file wrappers. We’ve had some of our most experienced developers spend much of the past year working on this approach—but much of that time was spent blazing new territory, and most issues we encountered weren’t within our control to fix because we don’t control the technology and servers. Fortunately, we think we’ve pushed through most of the critical issues and we’re hoping this approach will bear fruit soon.

Of course, if it takes too long to change our document format or to get atomic directory syncing working with other people’s cloud servers, another approach would be for us to add syncing support for our own cloud servers. We recently brought our Omni Sync Server out of beta, and we’re using it now to automatically sync OmniFocus and OmniPlan.

One way or another, we’re working very hard to bring automatic syncing to OmniOutliner as soon as possible: we know it’s absolutely critical for anyone who wants to use OmniOutliner on more than one device. We’re very sorry it’s taking so long!

As always, we welcome your feedback! Please feel free to leave a comment here or on our forums, or by sending me a message on twitter (where you’ll find me at @kcase).