Secret Origin of the ‘Super Ultra Critical Showstopper’ Bug

When you first start a project and you set up bug tracking software there tends to be 3 levels of bugs: Low, Medium, and High. After awhile you might notice there are now ‘critical’ bugs. Come back a little while later and there’ll be a ‘showstopper’ or ‘blocker’ bug level. And the funny thing is, most of the bugs will be logged at the highest levels. Literally 50% of bugs are given such a status as to make you think the app is in shambles. Except that when you actually open up qny of them to see what could possibly be so horrible, you find things like: missing fields, pages that should be pop-ups, and misspelled text. How is it that these things are given the highest level of importance?

Today, while I was working on a super ultra critical showstopper bug, I had some time to reflect on this phenomena. The bug was this: One of our models was missing a field. And the business wanted this field to be entered before the user could perform some task. Now, I should point out that nothing in the app was giving an error message, except that the user could get away without entering a field that had no functionality except to be displayed on the screen later. In my humble opinion, the app could have gone live without fixing this bug and nothing terrible would have happened.

But, you might say, it’s not my call to decide that. Absolutely right. But it’s also true that sitting next to it, at the same level of priority, were bugs that really were super ultra critical (such as the temporary inability to edit anything at all in our content management system – yipes!). So how do we let the business decide what gets done first but also have a rational system of ranking bugs? Well you need to have ranking system that actually means something. If high isn’t defined then everything gets rated as ‘high’ (especially cause manager X is gonna freak if it doesn’t get fixed – and every bug has a manager X) so we create ‘critical’ and then everyone calls things critical, and this continues until you have a very long list of bug ratings (and only the top two will ever get used). On a previous project we had the same problem and so moved to this rating system:

  • Priority 1: The app is down. No part of the app runs at all.
  • Priority 2: A large part of the app is not functional, but some parts still work.
  • Priority 3: An important function does not work, but there is a workaround.
  • Priority 4: Satellite functionality does not work.
  • Priority 5: Cosmetic change.

This worked pretty well: important things got fixed and if the client wanted to get something moved up in priority it could always be played as a card (like while technically misspelling the name of the application is a cosmetic change and therefore priority 5, but still…)


Matt McKnight said…
You have to have two fields- priority and severity.
In addition, we make sure we classify anything new as an enhancement or task versus a bug. Enhancement has no severity attached, and doesn't confuse when we try to take a metric oriented look at our quality.

Popular posts from this blog

SICP Wasn’t Written for You

What's a Good Flog Score?

Point Inside a Polygon in Ruby