The Blog

Recently Featured

Complete Archives

Articles by App

OmniDazzle
OmniDiskSweeper
OmniFocus
OmniGraffle
OmniGraphSketcher
OmniObjectMeter
OmniOutliner
OmniPlan
OmniWeb

OmniFocus crash statistics

by Tim Wood on July 25, 2007

For many years now we've had an integrated crash reporting system.  This has helped improve the stability of our applications immensely (often report now start out with “Wow, this is the first crash I've seen…”).  But, it hasn't always been clear (especially in the alpha or beta timeframe) how well we were doing on overall stability.  We could guess by counting the number of crash reports vs. an estimate of the number of active users, but that wasn't very convincing.

Near the beginning of June, I added some support to our software update and crash logging frameworks to keep track of things like:

  • total times the application has been launched

  • total number of crashes

  • total amount of time the application has been running

(As always, our software update system reports its information without including any personal details, and can be disabled entirely if so desired.)

Using this, we can now chart the total number of hours OmniFocus has been running vs. the total number of crashes (reported or not!).  As the pool of people testing OmniFocus goes up, or some testers go idle, or some user with large number of crashes isn't reporting them, we don't have to wonder as much how that affects our average crash rate.

OmniFocus Hours per Crash 20070725

After my latest crash fix, our rate has improved to about 8000 hours per crash.  We aren't sure yet what constitutes a reasonable lower limit for hours/crash, but this does let us notice when a fix we've made actually is addressing the issue.  We aren't yet tracking the number of hours that the application is active (an hour spent hidden counts the same as an hour spent in full use).  Whether this matters, when averaged across a large number of users, is open to question.

Still, there are only 8760 hours in a year, so if we can get above that, we'll be feeling pretty good.

 

 

Comments

I cannot think of a single time where OmniFocus has crashed on me. There are glitches, sure, but not one crashes.


And I grab just about every single new build as they come out, so I would think that the chance of getting bitten by a bad build would be higher than the people who are updating only once a day or every couple of days.


That's pretty damned impressive for alpha builds of an incomplete application.

Jay Elmore

07.25.07 4:43 AM

Strangely enough, I have yet to see a single crash, and I was in the first group of alpha invites! So IME, the program is perfectly stable! :)

SkylarP

07.25.07 5:47 AM

So….this means the beta's almost ready, right?

alexandra

07.25.07 8:03 AM

I've been using it every day for at least 2-3 months (can't remember that clearly) and haven't had one crash… and the average is once every 8000 hours? You Omni folks must be really good with that computer code and stuff.

coffee_shop_theologian

07.26.07 2:47 PM

I have thought several times how impressively stable this app is. If it crashes, I'll let you know!

Paul Walton

07.27.07 6:45 PM

It would be interesting to the see the distribution of crashes per hour over all people (instead of just the mean) - e.g. with a histogram.  An average crash rate of 1 / 8000 hours could mean for some people it crashes every hour, while for others it never crashes.

Hadley

07.27.07 11:56 PM

Well I'm glad OmniWeb is so stable for you folks, but for me it crashes almost daily. I always download the latest SneakyPeak and am an advanced Mac user.


OmniWeb has never been that stable for me, whatever version I have used.

Phillip Deackes

07.28.07 4:49 AM

Deackes

Then you have something wrong with your *machine*. I'm a Mac consultant - want I should look at it?

Ron

07.29.07 1:25 AM

Phillip; note that this refers to OmniFocus, not OmniWeb.  We know that OmniWeb 5.6 continues to need improvement—in fact one of the more common crashers was fixed on Friday (something to do with the intersection of ad blocking and plugins).

Tim

07.29.07 4:57 AM

Doh! Profuse apologies, Tim. I assumed the 'focus' meant you were focussing on OmniWeb. Sorry, guys!

Phillip Deackes

07.29.07 5:41 AM

I haven't experienced any crashes, but on my newest Mac Book, I consistently get the “Can't push calendar” when trying to sync with iCal. I don't get that message with my older G4 notebook.


These latest alphas are getting good enough that I'm leaning toward using OmniFocus instead of other GTD programs (most of which are buggy in their post-beta versions).

Jeffery Smith

07.29.07 5:52 AM

I would like to echo what has already been said, I have had no problems with crashing. I can't believe this is an “alpha” build. It's so dependable that I actually depend on it for my day-to-day GTD needs. It's awesome.

Chris

07.30.07 1:45 AM

Please, please, please consider looking at your recent mailing list subscriptions and inviting newbies to participate in the tasty alpha peeks? I just bought OO pro (OG pro too) and am using kGTD, which is very cool, but I'd love to kick the tires with OF, which you can definitely put me down for right now. :)

Steve Linberg

08.06.07 6:25 AM
Team Member

On our forums, we post weekly status updates about our sneaky peek invitations.  Today's update reads:


10,625 invitations have been sent (to 93% of 11,382 subscribers), bringing us up to July 24 (two weeks ago today). At our current rate, someone who subscribes today will receive an invitation within five days.

Ken Case

08.07.07 12:40 AM

I subscribed on the 24th, so I'll be watching my inbox breathlessly. :) Thanks!

Steve Linberg

08.07.07 2:57 AM

Instead of (or in addition to) counting wall time, why not count CPU time?  This would allow you to differentiate between time spent in the background and time spent in active use.

Ben Rosengart

08.07.07 10:38 AM

I subscribed to the mailing list back in March or April, but didn't receive a preview email.  It may have landed in the spam folder.  Any chance I could get it resent?

Michael Third

08.08.07 6:51 AM

Yeah, wow.


I have had Focus for nearly a month now, and not a single crash.  Obviously some (obvious) bugs, but never a crash.


Regularly signing up for Alpha tests, I would have expected many - like, for example, the MySQL GUI tools that have been in Alpha release for a couple of years and crash every time you do any of four or five things.


You guys continue to hold a special place in my heart as my favorite Mac development team. :-)


I can't believe I'm about to type this… I can't wait until the final version comes out so that I can buy it. :-)


Great work so far!

Chris Ostmo

08.08.07 9:47 AM

the problem with this system is, as you get more users that are using the sofware flipantly, your hours will go up.  You'll have someone look at it, add a few things to it, and become disinterested, so they won't get into the hardcore use.


Also, as time goes on, alpha testers will run out of personal tricks for breaking it.


Graphs are cool though.

scot

08.13.07 3:35 AM
Commenting is not available in this channel entry.