The Blog

Recently Featured

Complete Archives

Articles by App

OmniDazzle
OmniDiskSweeper
OmniFocus
OmniGraffle
OmniGraphSketcher
OmniObjectMeter
OmniOutliner
OmniPlan
OmniWeb

OmniFocus and the Way of the (Support) Ninja

by Brian on October 2, 2007

Linda asked lil' ol' me to provide the second post in our ongoing OmniFocus: What We've Learned So Far series. Whether she will come to regret this, only time can tell. Without further ado, the post beginneth thusly:Okay, I'm long-winded; sue me. Before I can really tell you how well the OmniFocus test process is going, though, I feel like I have to supply a bit of background on how we've handled this process for previous projects. It went a little something like this:

  1. Development team produces build of application OmniFoo they're mostly happy with.
  2. Marketing Weasel writes press release, webmaster nee Web Editor updates site with brand-new OmniFoo Beta page. We push new version of website.
  3. VersionTracker, etc. pick up on new build.
  4. Support Ninjas are crushed under a big pile of electronic mail for the next three months. All that is heard from them is a soft but desperate honking noise. Think baby penguin at the bottom of a deep ice-crevasse.
This has a couple of negative effects: first of all, when you're buried in raw oscillating electrons up to your neck, it's really hard on the skin; not at all like that 'drink from the glowy pool of water' scene in Tron would have you believe.More seriously: we get plenty of eyeballs on the new application, which is a good thing. Unfortunately we also got all those eyeballs on the new app at the exact same time. So that thing that's forehead-smackingly obviously broken (which, of course, we failed to catch in the gajillions of hours we spent staring at the app before we pushed it out) gets reported 200 times.Now, the first report of an issue? Good. The tenth? Gold - then we know that this isn't random gremlin activity; there is something here we need to figure out. This holds up till, oh, somewhere between 30 and 50 reports. Beyond that, it's a problem we know about, that we know we need to fix, but from there on out the additional utility of the reports drops off fast. The time it takes the Ninjas to process them doesn't, however.Result: stressed-out ninjas, frustrated engineers (because they're not getting reports of the problems in the newest builds; we're still looking at launch day reports), and the folks with the test builds are having to wade through issues that we haven't fixed because we're still sorting and writing up reports.In short, it works, but it's painful for everyone involved.  So this time, we did something better. Process this time:
  1. A couple months before we're ready to start testing, let folks know we'll be ready to start testing in a couple months. Set up an email list to join if they'd like to participate.
  2. Produce build of application we're ready to start testing.
  3. Make build available to some of the folks on said list.
  4. Fix the problems they find, including forehead-smackers.
  5. Return to step 3, above.
Advantages: Many. We get feedback in manageable quantities. Testers get fixes that bear some resemblance to their reports. Support ninjas get fewer ulcers. World shares cola beverage, sings in perfect harmony.What do we need to do differently next time? To begin with, we need to give customers the ability to help us prioritize their mail, by at the very least sorting it into “bug report”, “feature request”, and “oh god, where is my download login” buckets. The other thing? If we choose to implement any more apps based on the current 800-pound gorilla of personal productivity methodologies, I'm just going to start hiring and never, ever stop. ;-)Which, of course, provides me with a perfect opportunity to link point interested parties over to our Want Work? page, newly updated as of yesterday.

 

Comments

One suggestion to your testing process.  Automatically have your feedback (at the app level?) filter out builds that are greater than 48 hours or so (maybe 3-4 builds back.)


When someone goes to give feedback (believe me, they'll mark a feature that is missing as a 'bug'), the app checks the 'recentness' of the alpha/beta build: too old, and it tells them to download the current version and THEN give the feedback.

jeffG

10.02.07 2:01 PM

I feel your pain!  I've been in similar positions before, myself.


Here's one suggestion: make a public web page (or one hidden behind the password used for the OF downloads) that lists all the major known bugs.


Strongly and repeatedly encourage everyone to check that before sending you e-mail with bug reports.


I know it's always a bit uncomfortable airing your dirty laundry, but you'll probably save yourself a lot of work in the long run.

zedboy

10.03.07 1:01 AM

Jeff: Not a bad idea - we post several builds a day, but it would definitely be helpful if we encouraged folks sending us feedback to update if they were more than a couple days behind.


And yeah, some minority of folks will definitely abuse any priority system we give them, but it's still better than the current system, where we have no way to prioritize at all. Folks that try to bump to the top of the list will just get kicked back down.


Zedboy: We've got the known issues posted on the site, but yeah, making it easier for folks to check that would be good. Filing that away in the mental rolodex. Thanks!

Brian

10.03.07 6:35 AM

I'm just wondering if you got the memo about how we're handling TPS now.

chris

10.03.07 12:58 PM

I'm an alpha-tester wondering whether the feedback mechanism should be used for “voting” for particular development. For example, if I read on the OF forum that somebody has an idea for a feature and has submitted to Omni, how welcome or unwelcome would additional feedback be? If I send additional feedback myself, is that helpful in generating a good picture of the desires of the userbase in aggregate, or is it unhelpfully clogging up the system?

Craig

10.04.07 5:39 AM

Craig: Don't hesitate - just go ahead and send the feedback in. For feature requests, additional votes always help.


Customer demand is plays a tie-breaker role when we have two features we're considering implementing. Given equal difficulty/time required to add them, the one with more demand behind it is more likely to make it onto the schedule.


All of my theorycrafting above is about what we can do on the back end to get responses (email and/or code changes) out to you guys as fast as possible. Just tell us what you want/need; we'll take it from there. ;-)

Brian

10.04.07 6:10 AM

Hey, Brian…“Filing that away in the mental rolodex.” You guys will never get GTD right if you keep thinking like this!

Patrick

10.05.07 9:12 AM

LOL @ chris ;)


Funny, I just watched that movie yesterday.

Rob Record

10.07.07 7:34 AM

I really like the OmniFocus (and now OmniWeb) rapid release cycle, especially now that it's automated.


My vision of alpha tester heaven:


Each feedback report is entered a tracking database and is classified as new problem, duplicate report, user error, what???, out-in-left field, ...


Testers are encouraged to classify their feedback:  bug report vs. feature request, priority/impact (e.g., showstopper, important, moderate, minor).


When a problem is resolved, a message is sent to those who submitted feedback.  Although I scan all of the release notes (aka messages of the day) for fixes to the bugs I've reported or features I've requested,  I know I've missed some because I've scanned too quickly or because I didn't recognize that a fix/feature corresponds to my feedback.

Ward

10.14.07 5:06 AM

Perhaps all that you Ninjas need to help with prioritizing/organizing all the Alpha feedback is a nifty new program called OmniFocus…

Erik

10.15.07 8:57 AM

http://www.apple.com/hotnews/


A real iPhone SDK is scheduled for February! Great news for OmniFocus, I'd think…

Dave

10.17.07 6:04 AM

Someone told me that OmniFocus 1.0 would be more like 1.5 and that it could have shipped already. If true, aren't you far enough along that you can announce a ship date?

Neil J. Squillante

10.23.07 9:35 AM

We generally don't announce ship dates until the possibility of that announcement being wrong is approaching 0%. We're not quite to that point yet on OmniFocus.

Brian

10.25.07 4:53 AM
Commenting is not available in this channel entry.