Either fix it quickly or not at all.
For many, bug tracking software is central to good software practice. It features as number four of the Joel Test and is used extensively in a Lean or Agile context, as well as in more traditional software development approaches.
My company uses bug tracking software, my team used to use bug tracking software, and now we don’t, so what’s going on?
In general, I think we’re doing pretty well, we use a Kanban board to manage our work flow, we safely release to production five or six times a week, use tests to drive our designs, practice pair programming and have an effective Continuous Integration and Black Box testing set up. We definitely still have areas we can improve on but by most people’s standards we’re on the right track.
So why aren’t we using a bug tracker? Well on identifying a bug any team has four choices:-
- fix it immediately
- fix it in the near future
- add it to a list and plan to fix it at some point
- ignore it
Some bugs simply have to be fixed immediately, this is a truism wherever you work. If Google’s search functionality stopped searching, or Microsoft’s Excel stopped multiplying then everything would need to stop in order to get a fix out. Hopefully such occurrences are rare and can be considered a special case.
So we’re left with fixing it in the near future, fixing it at some unspecified point in the future or ignoring it entirely. What I noticed about option three is that low priority bugs are simply never looked at, something more important always gets in the way. So we made a decision, that when we find a bug we either fix it promptly, probably in the next week, or decide simply to ignore it.
This took a some time to get used to. Nobody is comfortable with the idea of dropping bugs, but as soon as we realised that ignoring it and adding it to the bug tracker as a low priority item were pretty much the same thing, we started to feel more comfortable. A bug feels most important on discovery and if we decide not to fix a bug when it’s fresh in our minds and we’re ready to go, realistically, it’s never going to get fixed. If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.
If bugs are to be prioritised over scheduled project work, it’s important to track the amount of effort spent on bug fixing. For my team, this isn’t really a problem since in addition to having responsibility for developing software, we are also responsible for much of our own sys admin, operations and support. Therefore we expect a certain amount of time to be spent on unplanned work and we track time spent on bug fixing as part of this. It turns out that the amount of time spent on unplanned work is fairly predictable and can be accounted for in long term project planning. The only exception to this rule is where the bug is sufficiently large that fixing it will require a significant effort, maybe a week or more’s worth of work. In that case we’ll treat it as a project in its own right and record progress as if it were any other standard planned block of work.
How does this work in practice? The key point is that it encourages us to fix bugs as we find them, and places greater emphasis on not introducing them in the first place. The fact that we know that the bug will be dropped if we don’t fix it in the short term means that we take bugs more seriously. Over the past six months we’ve ignored only two issues, both of which were highly unlikely to recur and both of which resided in third party libraries that we intend to upgrade.
The only downside to not recording bug fixes is that there is then no record of previous faults and no bug driven history of why things are as they are. However, so long as the Black Box tests cover all bugs as they are found the chance of regression is nil. Therefore I don’t really see this as a problem at all.
As an aside, I can see the value in tracking bugs over longer periods in situations where QA and development are separate entities that do not interact as part of the same team, or alternatively where a team is so swamped with bugs that they cannot keep up. However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term rather than addressing the underlying problems.
So in conclusion, by opting to fix bugs quickly or not at all, we force ourselves to fix more bugs and place greater emphasis on not introducing them in the first place. We no longer have to manage a unmaintainable bug list and ensure that our black box test coverage develops over time.
Edit: This post has generated a good deal of debate, in response I have posted a follow up here where I try to address some of the points raised.
Fragile 
That’s a really interesting idea. I think I would have a difficult time wrapping my brain around the idea of ignoring a bug like that, but I think in the long run you actually save time and effort.
LikeLike
It works for you but it won’t when you have many developers and release cycles/milestones. Besides, not all bugs are major and requires immediate fix (hot patch). The majority of bugs are of a medium severity.
LikeLike
I wonder how you document your solution to a problem and how others in your company can find out about it as soon as you are gone (maybe forever).
Your style reminds me of the working style of to former employees: we are now facing a lot of hassle because they were to lazy to document everything.
And its very helpful to be able to reopen closed bugs, not something to consider optionally — regardless if repeated disaster strikes you within weeks or years — btw. I consider searching for related bugs obligatory, not a waste of time.
I rather drop bug reports made by employees who are gone in the meantime anyway, than drop bugs only because they have a certain age.
LikeLike
>> If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it. <> However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term… <<
Doesn't the bug tracker also make blindingly obvious to everyone in the organization that the team is swamped with bugs which are not being addressed?
LikeLike
So, how do you remember that bug that happends 5 years back and the way you solve it?
If you use a bugtracker you have all that info in the ticket.
LikeLike
> “If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.”
Given that you aren’t tracking, how would you know that the same bug has returned at a later date?
> “However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term…”
Doesn’t the bug tracker also make it blindingly obvious to everyone in the organization that the team is swamped with bugs which are not being addressed?
LikeLike
Release to production 5-6 times a week?
No bug tracking?
Ignoring bugs, because they have low priority – (aka. because they are lame and found by some marketing guy who has no serious understanding of how difficult software programming is and that he shouldn’t bother the geniouses with such trivial matter)?
Sounds like naive programming to me. My kid just learned prgramming and he does it just like you.
LikeLike
Replacing bug tracking by a kanban board !!! Thats one way to physically limit the number of bugs you have.
Bug tracker hosts bugs, with time becomes ‘inventory’ which is a waste.
LikeLike
True or False : the reason you can live without bug tracking is because your dev team is small?
If you ever try to live without proper bug tracking system when your team exceeds 10 devs – its just a nightmare.
LikeLike
A couple of questions to understand your seemingly “dangerous” move :
How close are you to the people specifying the product ?
It seems like dropping the bug tracker could only be manageable if you always have the solution to a bug whenever you face one. From what I’ve seen, bugs are more than often the result of a misunderstanding. A bug tracker can help you keep track of the *communication* required to fix the bug. And in conjunction to your VCS, find the line of code that implemented the proper fix.
What’s the size of your team ?
Again, a bug tracker is really a communication facilitator ; if you’re only a couple of coders, and you’re the ones deciding what goes into your product or not, then this is probably very sustainable. I wouldn’t advocate it for any other kind of set ups …
Is your project outside-world facing ? You mentionned you have QA, but do you also get bug reports / reviews from actual users of the productin system, and if so, how ?
Thanks
PH
LikeLike
Echoing Jonathan and going a bit further:
Eh, but what if you use a bug tracker to facilitate a dev record, and reviews on planned features?
Sure you use SCM, but no-one can be bothered reading through necessarily terse commit logs to work out what the pros and cons of a particular design route were seen to be during original implementation.
That’s why we use bugzilla, anyway.
LikeLike
This is a really bad idea. You can fix bugs as they arrive AND keep track of them at the same time. A good bug tracking system isn’t a big hassle to use.
In short, why you would want to throw out a massive amount of potentially useful data is just totally beyond me. There is a lot of good information to be gleaned from bug tracking besides just “which bugs do we need to fix”.
Specifically and in addition to the really good points made in the other comments, here are a few more reasons this is a bad idea:
1. You can’t compare the quality of releases against each other. How many bugs are you shipping with? How many were found?
2. Bug tracking is a useful metric to developers and managers. How many bugs am I creating? Of what severity? Where can I improve?
3. As the person writing the software you WILL have inconsistencies with your customers in what you consider a must-fix bug. If you ignore a bug and then a support call comes in, there is no way to re-evaluate and learn that maybe you shouldn’t have shipped with that known issue.
LikeLike
You present your case well 🙂
LikeLike
A similar approach would be to use bug-tracking software as your Kanban board, and when a new bug is entered triage it immediately into (near future) or (never).
LikeLike
I don’t think “fix it quickly or not at all” can scale over the long term (maybe you can have another post in the future about that). There are legitimate issues that are not “stop everything” types that need to get done within a reasonable duration but not right away. There must be some way to document these; therefore, what may happen is the team will go back to email and instant messaging for issue management. I had a lot of frustrations with getting a team (project managers, managers,technical experts, and programmers ) collaborating in one place; therefore, I created Simplton.
I was fed up with priority levels — everything is a critical you know 😉 — so I got rid of them completely and went to a “bubble up” system. Issues are then bubbled down automatically (15 days is the default) if there is no activity on them. This helps create an environment where stuff that is important stays near the top of the list.
Another thing on “stop everything” issues: I don’t even think these types of issues need to be in the issue tracker right away. Phone, IM, and instant messaging are great tools for truly critical issues. After the resolution the issue can the go into the bug tracker as it also serves as a knowledge base. Very interesting post.
LikeLike
You will end up wasting a lot of time with this approach IMHO
by using a bug tracker, you then know what the status of an known issue is
how does a new staff member find out about the history of a problem?
i love the just fix it or don’t create bugs, but thats a pipe dream in a complex system
with commercial priorities.
By not logging this information, I would expect many hours of your (expensive) staff’s time
to be wasted over and over again
LikeLike
Many thanks for all of your comments, I’ve just posted https://fragile.org.uk/2010/04/why-i-still-dont-use-bug-tracking-software/ which aims to answer some of the points raised here.
LikeLike
Hi there,
I could not agree more with you.
To keep track of bugs automatically (the ones you can detect), with info such as number of occurrences of a particular bug, context…, have a look at ErrorNot (http://github.com/errornot/ErrorNot).
That way, you don’t have the overkill of a bug tracker, but you have history and metrics.
LikeLike
What I’m missing is a solution to “not introducing them in the first place”
Running a project Kanban sounds like a support function for an existing (possibly legacy) system typically requiring one manager and two / three developers. The funding cost will be the same year on year. Eventually, the company enters a cost-cutting phase and looks at the return on investment. It costs X to run the team and the team runs the system — typically not introducing new features; rather fixing bugs. If it’s worth to the company — the support function stays & nothing changes; it’s it’s not — it enters a decomission programme for next year.
I don’t see how management process or development utilities provide a solution and the solution I am still missing is how not to introduce bugs in the first place.
There are two types of bugs that developers introduce: those they know about and those they don’t know about — hence the suport cycle repeats until the business has other interests.
LikeLike
I think you’re missing the point of why bugs need to be tracked. Its not simply a case of ‘will we fix this one or not’.
We need to improve the quality of the software overall. We cant do that if bugs arent logged in a system where they can be analysed & quantified. Im afraid your Kanban board doesn’t allow for this. But if your company is not interested in assessing & looking for ways to improve quality across whole development process then it doesn’t matter.
LikeLike