Either fix it quickly or not at all.
For many, bug tracking software is central to good software practice. It features as number four of the Joel Test and is used extensively in a Lean or Agile context, as well as in more traditional software development approaches.
My company uses bug tracking software, my team used to use bug tracking software, and now we don’t, so what’s going on?
In general, I think we’re doing pretty well, we use a Kanban board to manage our work flow, we safely release to production five or six times a week, use tests to drive our designs, practice pair programming and have an effective Continuous Integration and Black Box testing set up. We definitely still have areas we can improve on but by most people’s standards we’re on the right track.
So why aren’t we using a bug tracker? Well on identifying a bug any team has four choices:-
- fix it immediately
- fix it in the near future
- add it to a list and plan to fix it at some point
- ignore it
Some bugs simply have to be fixed immediately, this is a truism wherever you work. If Google’s search functionality stopped searching, or Microsoft’s Excel stopped multiplying then everything would need to stop in order to get a fix out. Hopefully such occurrences are rare and can be considered a special case.
So we’re left with fixing it in the near future, fixing it at some unspecified point in the future or ignoring it entirely. What I noticed about option three is that low priority bugs are simply never looked at, something more important always gets in the way. So we made a decision, that when we find a bug we either fix it promptly, probably in the next week, or decide simply to ignore it.
This took a some time to get used to. Nobody is comfortable with the idea of dropping bugs, but as soon as we realised that ignoring it and adding it to the bug tracker as a low priority item were pretty much the same thing, we started to feel more comfortable. A bug feels most important on discovery and if we decide not to fix a bug when it’s fresh in our minds and we’re ready to go, realistically, it’s never going to get fixed. If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.
If bugs are to be prioritised over scheduled project work, it’s important to track the amount of effort spent on bug fixing. For my team, this isn’t really a problem since in addition to having responsibility for developing software, we are also responsible for much of our own sys admin, operations and support. Therefore we expect a certain amount of time to be spent on unplanned work and we track time spent on bug fixing as part of this. It turns out that the amount of time spent on unplanned work is fairly predictable and can be accounted for in long term project planning. The only exception to this rule is where the bug is sufficiently large that fixing it will require a significant effort, maybe a week or more’s worth of work. In that case we’ll treat it as a project in its own right and record progress as if it were any other standard planned block of work.
How does this work in practice? The key point is that it encourages us to fix bugs as we find them, and places greater emphasis on not introducing them in the first place. The fact that we know that the bug will be dropped if we don’t fix it in the short term means that we take bugs more seriously. Over the past six months we’ve ignored only two issues, both of which were highly unlikely to recur and both of which resided in third party libraries that we intend to upgrade.
The only downside to not recording bug fixes is that there is then no record of previous faults and no bug driven history of why things are as they are. However, so long as the Black Box tests cover all bugs as they are found the chance of regression is nil. Therefore I don’t really see this as a problem at all.
As an aside, I can see the value in tracking bugs over longer periods in situations where QA and development are separate entities that do not interact as part of the same team, or alternatively where a team is so swamped with bugs that they cannot keep up. However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term rather than addressing the underlying problems.
So in conclusion, by opting to fix bugs quickly or not at all, we force ourselves to fix more bugs and place greater emphasis on not introducing them in the first place. We no longer have to manage a unmaintainable bug list and ensure that our black box test coverage develops over time.
Edit: This post has generated a good deal of debate, in response I have posted a follow up here where I try to address some of the points raised.
61 Responses to “Why I Don’t Use Bug Tracking Software”
Sorry, the comment form is closed at this time.
Additional comments powered byBackType
This ignores a critical use of bug tracking – communication. This approach might work in a small local team, but falls apart in a bigger organization or in a distributed environment. Bug tracking software is one place to store information about the bug that multiple people can comment on. The person discovering the bug might not be a developer, and often not. The person testing the bug shouldn’t be the developer; QA is doing it. If the fix involves several people and they need to add input, they can add in their comments in the bug tracking system. Cvs/Svn/Git can tie with the bug tracking ticket to create a reverse lookup. There are so many benefits to have a bug tracking system.
This comment was originally posted onHacker News
It (obviously) works fine for small to medium projects, but I believe the bug tracker-free approach doesn’t scale at all. I don’t know if the author thinks it does or now, I just want to point it out to prevent people from getting ideas.
Scenario 1: A salesperson or customer advocate reports five bugs found during deployment of application X. Some of the bugs crosses several products, and you have to "pass on" the bug to backend server developer Bob when you find out your code is working as intended. How should you communicate the reported bug and your analysis to Bob? Assign the bug to him in the tracker, and he will get access to the complete bug history – initial report, discussion, reproduction steps, analysis, why-this-isn’t-a-problem-with-module-Q-even-though-it-looks-like-it, new comments, screenshots, log files and configuration file attachments from the customer. How will you (and Bob) communicate progress to the salesman? He is CC-ed to each bug in the tracker.
Scenario 2: Your team is responsible for 4 products, which it switches between focusing on every few weeks. When you’re hard at work on releasing product A version 3.1, and you spot a nasty and subtle problem with the design of the message queue in the upcoming version 2.3 of product C. Immediately drop development on A? No, you have a deadline to respect and the product C bug isn’t affecting anyone yet. Forget about the bug and hope to rediscover it or remember it (and 5 others) in 6 weeks time? That is irresponsible. You put it in the bug tracker, and when product C time comes around, you look at the bug database before starting work.
Or as larsberg says, when your project reaches a certain size and userbase, you will get several hard and timeconsuming bugs at the same time. You can’t always fixemrightaway or drop them.
This comment was originally posted onHacker News
It (obviously) works fine for small to medium projects, but I believe the bug tracker-free approach doesn’t scale at all.Scenario 1: A salesperson or customer advocate reports five bugs found during deployment of application X. Some of the bugs crosses several products, and you have to "pass on" the bug to backend server developer Bob when you find out your code is working as intended. How should you communicate the reported bug and your analysis to Bob? Assign the bug to him in the tracker, and he will get complete history. How will you (and Bob) communicate progress to the salesman? He is CC-ed to each bug in the tracker.
Scenario 2: Your team is responsible for 4 products, which it switches between focusing on every few weeks. When you’re hard at work on releasing product A version 3.1, and you spot a nasty and subtle problem with the design of the message queue in the upcoming version 2.3 of product C. Immediately drop development on A? No, you have a deadline to respect and the product C bug isn’t affecting anyone yet. Forget about the bug and hope to rediscover it or remember it (and 5 others) in 6 weeks time? That is irresponsible. You put it in the bug tracker, and when product C time comes around, you look at the bug database before starting work.
Or as larsberg says, when your project reaches a certain size and userbase, you will get several hard and timeconsuming bugs at the same time. You can’t always fixemrightaway or drop them.
This comment was originally posted onHacker News
I often find bugs when I’m too bleary eyed to care about fixing them. If I just go to bed at that time, I’ll have forgotten about it in the morning. Instead, filing a 2 line bug report serves as a good enough reminder for when I wake up.Also, what everyone else said
This comment was originally posted onHacker News
Assuming you have more work than resources (which is what one should plan for) I would find it hard to sacrifice the ’strategy’ aspect of bug tracking systems. A good bug tracking system allows you to balance quick/low-overhead logging with an ability to create categorical data points that let you analyze you data.Having this data collection & reporting allows you to identify & trend problem areas that can not be seen looking at a single bug at a time. It also allows you collect similar overlapping areas of work to strategically manage your resources and productivity – saving you time and money.
Maybe the case is that the article comes from a company that’s just too small to realize the benefits of planning.
This comment was originally posted onHacker News
Assuming you have more work than resources (which is what one should plan for) I would find it hard to sacrifice the ’strategy’ aspect of bug tracking systems. A good bug tracking system allows you to balance quick/low-overhead logging with an ability to create categorical data points that let you analyze your work & workload.Having this data collection & reporting allows you to identify & trend problem areas that can not be seen looking at a single bug at a time by different people. It also allows you collect similar/overlapping areas of work to strategically manage your resources and productivity – saving you time and money.
Maybe the case is that the article comes from a company that’s just too small to realize the benefits of planning.
This comment was originally posted onHacker News
**_REDICULOUS_**. How many times do you see a comment in the code like "WTF: This is bug #N" and go and look it up only to find that this is the way it is for a reason, or a warning about the possibility of breaking it again, or any number of other things. How many times do you have some bug to fix and remember a former developer had worked on it a few months back but can’t remember what the deal was anymore, so you look it up? How many… **etc**, **etc**, **etc**. Well, I can’t answer that for _you_, but for me it’s _all the time_. It takes mere seconds to enter a bug, and then a few more seconds after you fix it to explain what happened, refer to a commit number, etc and it could save _hours_ in the future for you or colleagues working in the same area. Seriously, do not do this – and if you must, make sure to note it in your job posting so competent developers don’t accidentally apply to work with you.
This comment was originally posted onReddit
I couldn’t agree with you more. I came to post the same thing. This process so completely ignores your users, I cannot fathom how someone could think this is "agile" (what with all that customer-driving-the-process stuff). How in the world do you ship to your users with a "known bugs" list if this is the way you deal with your bugs? Incidentally, wherever I’ve interviewed in recent memory, I always ask what they used to track bugs. At one *big* company, the response I got back was "Whiteboard", which I took to be the name of a software product, and only several questions later did it become obvious that they meant an *actual whiteboard*. I then asked them what they did with the bug if a developer fixed it. "We erase it!" Doh.
This comment was originally posted onReddit
> "We erase it!" /me runs away screaming.
This comment was originally posted onReddit
Come back next week for: *Why version control is a waste of time and resources.* Stupid.
This comment was originally posted onReddit
Thanks for comments, this post has generated a lot of discussion, I’ve tried to answer some of the more common responses in a follow up post here http://fragile.org.uk/2010/04/why-i-still-dont-use-bug-tracking-software/
This comment was originally posted onReddit