Either fix it quickly or not at all.
For many, bug tracking software is central to good software practice. It features as number four of the Joel Test and is used extensively in a Lean or Agile context, as well as in more traditional software development approaches.
My company uses bug tracking software, my team used to use bug tracking software, and now we don’t, so what’s going on?
In general, I think we’re doing pretty well, we use a Kanban board to manage our work flow, we safely release to production five or six times a week, use tests to drive our designs, practice pair programming and have an effective Continuous Integration and Black Box testing set up. We definitely still have areas we can improve on but by most people’s standards we’re on the right track.
So why aren’t we using a bug tracker? Well on identifying a bug any team has four choices:-
- fix it immediately
- fix it in the near future
- add it to a list and plan to fix it at some point
- ignore it
Some bugs simply have to be fixed immediately, this is a truism wherever you work. If Google’s search functionality stopped searching, or Microsoft’s Excel stopped multiplying then everything would need to stop in order to get a fix out. Hopefully such occurrences are rare and can be considered a special case.
So we’re left with fixing it in the near future, fixing it at some unspecified point in the future or ignoring it entirely. What I noticed about option three is that low priority bugs are simply never looked at, something more important always gets in the way. So we made a decision, that when we find a bug we either fix it promptly, probably in the next week, or decide simply to ignore it.
This took a some time to get used to. Nobody is comfortable with the idea of dropping bugs, but as soon as we realised that ignoring it and adding it to the bug tracker as a low priority item were pretty much the same thing, we started to feel more comfortable. A bug feels most important on discovery and if we decide not to fix a bug when it’s fresh in our minds and we’re ready to go, realistically, it’s never going to get fixed. If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.
If bugs are to be prioritised over scheduled project work, it’s important to track the amount of effort spent on bug fixing. For my team, this isn’t really a problem since in addition to having responsibility for developing software, we are also responsible for much of our own sys admin, operations and support. Therefore we expect a certain amount of time to be spent on unplanned work and we track time spent on bug fixing as part of this. It turns out that the amount of time spent on unplanned work is fairly predictable and can be accounted for in long term project planning. The only exception to this rule is where the bug is sufficiently large that fixing it will require a significant effort, maybe a week or more’s worth of work. In that case we’ll treat it as a project in its own right and record progress as if it were any other standard planned block of work.
How does this work in practice? The key point is that it encourages us to fix bugs as we find them, and places greater emphasis on not introducing them in the first place. The fact that we know that the bug will be dropped if we don’t fix it in the short term means that we take bugs more seriously. Over the past six months we’ve ignored only two issues, both of which were highly unlikely to recur and both of which resided in third party libraries that we intend to upgrade.
The only downside to not recording bug fixes is that there is then no record of previous faults and no bug driven history of why things are as they are. However, so long as the Black Box tests cover all bugs as they are found the chance of regression is nil. Therefore I don’t really see this as a problem at all.
As an aside, I can see the value in tracking bugs over longer periods in situations where QA and development are separate entities that do not interact as part of the same team, or alternatively where a team is so swamped with bugs that they cannot keep up. However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term rather than addressing the underlying problems.
So in conclusion, by opting to fix bugs quickly or not at all, we force ourselves to fix more bugs and place greater emphasis on not introducing them in the first place. We no longer have to manage a unmaintainable bug list and ensure that our black box test coverage develops over time.
Edit: This post has generated a good deal of debate, in response I have posted a follow up here where I try to address some of the points raised.
61 Responses to “Why I Don’t Use Bug Tracking Software”
Sorry, the comment form is closed at this time.
Additional comments powered byBackType
That’s a really interesting idea. I think I would have a difficult time wrapping my brain around the idea of ignoring a bug like that, but I think in the long run you actually save time and effort.
It works for you but it won’t when you have many developers and release cycles/milestones. Besides, not all bugs are major and requires immediate fix (hot patch). The majority of bugs are of a medium severity.
Social comments and analytics for this post…
This post was mentioned on Hackernews by larsberg: “Hopefully such occurrences are rare and can be considered a special case.”What if it’s both high-priority and hard? Like you find out that a fundamental part of how your product works causes data t…
[...] This post was mentioned on Twitter by Lars T., Thomas Buck. Thomas Buck said: Why I don't use bug tracking software. http://fragile.org.uk/2010/04/why-i-dont-use-bug-tracking-software/ [...]
I wonder how you document your solution to a problem and how others in your company can find out about it as soon as you are gone (maybe forever).
Your style reminds me of the working style of to former employees: we are now facing a lot of hassle because they were to lazy to document everything.
And its very helpful to be able to reopen closed bugs, not something to consider optionally — regardless if repeated disaster strikes you within weeks or years — btw. I consider searching for related bugs obligatory, not a waste of time.
I rather drop bug reports made by employees who are gone in the meantime anyway, than drop bugs only because they have a certain age.
>> If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it. <> However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term… <<
Doesn't the bug tracker also make blindingly obvious to everyone in the organization that the team is swamped with bugs which are not being addressed?
So, how do you remember that bug that happends 5 years back and the way you solve it?
If you use a bugtracker you have all that info in the ticket.
> “If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.”
Given that you aren’t tracking, how would you know that the same bug has returned at a later date?
> “However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term…”
Doesn’t the bug tracker also make it blindingly obvious to everyone in the organization that the team is swamped with bugs which are not being addressed?
Release to production 5-6 times a week?
No bug tracking?
Ignoring bugs, because they have low priority – (aka. because they are lame and found by some marketing guy who has no serious understanding of how difficult software programming is and that he shouldn’t bother the geniouses with such trivial matter)?
Sounds like naive programming to me. My kid just learned prgramming and he does it just like you.
Replacing bug tracking by a kanban board !!! Thats one way to physically limit the number of bugs you have.
Bug tracker hosts bugs, with time becomes ‘inventory’ which is a waste.
True or False : the reason you can live without bug tracking is because your dev team is small?
If you ever try to live without proper bug tracking system when your team exceeds 10 devs – its just a nightmare.
A couple of questions to understand your seemingly “dangerous” move :
How close are you to the people specifying the product ?
It seems like dropping the bug tracker could only be manageable if you always have the solution to a bug whenever you face one. From what I’ve seen, bugs are more than often the result of a misunderstanding. A bug tracker can help you keep track of the *communication* required to fix the bug. And in conjunction to your VCS, find the line of code that implemented the proper fix.
What’s the size of your team ?
Again, a bug tracker is really a communication facilitator ; if you’re only a couple of coders, and you’re the ones deciding what goes into your product or not, then this is probably very sustainable. I wouldn’t advocate it for any other kind of set ups …
Is your project outside-world facing ? You mentionned you have QA, but do you also get bug reports / reviews from actual users of the productin system, and if so, how ?
Thanks
PH
Echoing Jonathan and going a bit further:
Eh, but what if you use a bug tracker to facilitate a dev record, and reviews on planned features?
Sure you use SCM, but no-one can be bothered reading through necessarily terse commit logs to work out what the pros and cons of a particular design route were seen to be during original implementation.
That’s why we use bugzilla, anyway.
This is a really bad idea. You can fix bugs as they arrive AND keep track of them at the same time. A good bug tracking system isn’t a big hassle to use.
In short, why you would want to throw out a massive amount of potentially useful data is just totally beyond me. There is a lot of good information to be gleaned from bug tracking besides just “which bugs do we need to fix”.
Specifically and in addition to the really good points made in the other comments, here are a few more reasons this is a bad idea:
1. You can’t compare the quality of releases against each other. How many bugs are you shipping with? How many were found?
2. Bug tracking is a useful metric to developers and managers. How many bugs am I creating? Of what severity? Where can I improve?
3. As the person writing the software you WILL have inconsistencies with your customers in what you consider a must-fix bug. If you ignore a bug and then a support call comes in, there is no way to re-evaluate and learn that maybe you shouldn’t have shipped with that known issue.
You present your case well
A similar approach would be to use bug-tracking software as your Kanban board, and when a new bug is entered triage it immediately into (near future) or (never).
I don’t think “fix it quickly or not at all” can scale over the long term (maybe you can have another post in the future about that). There are legitimate issues that are not “stop everything” types that need to get done within a reasonable duration but not right away. There must be some way to document these; therefore, what may happen is the team will go back to email and instant messaging for issue management. I had a lot of frustrations with getting a team (project managers, managers,technical experts, and programmers ) collaborating in one place; therefore, I created Simplton.
— so I got rid of them completely and went to a “bubble up” system. Issues are then bubbled down automatically (15 days is the default) if there is no activity on them. This helps create an environment where stuff that is important stays near the top of the list.
I was fed up with priority levels — everything is a critical you know
Another thing on “stop everything” issues: I don’t even think these types of issues need to be in the issue tracker right away. Phone, IM, and instant messaging are great tools for truly critical issues. After the resolution the issue can the go into the bug tracker as it also serves as a knowledge base. Very interesting post.
You will end up wasting a lot of time with this approach IMHO
by using a bug tracker, you then know what the status of an known issue is
how does a new staff member find out about the history of a problem?
i love the just fix it or don’t create bugs, but thats a pipe dream in a complex system
with commercial priorities.
By not logging this information, I would expect many hours of your (expensive) staff’s time
to be wasted over and over again
[...] here to read the rest: Why I don't use bug tracking software. | fragile Share and [...]
Many thanks for all of your comments, I’ve just posted http://fragile.org.uk/2010/04/why-i-still-dont-use-bug-tracking-software/ which aims to answer some of the points raised here.
Hi there,
I could not agree more with you.
To keep track of bugs automatically (the ones you can detect), with info such as number of occurrences of a particular bug, context…, have a look at ErrorNot (http://github.com/errornot/ErrorNot).
That way, you don’t have the overkill of a bug tracker, but you have history and metrics.
[...] Why I don't use bug tracking software. | fragile [...]
[...] This post was mentioned on Twitter by Sébastien Douche, Yves Hanoulle. Yves Hanoulle said: RT @sdouche: interesting: Why I don't use bug tracking software http://goo.gl/fb/e8h7k [...]
"Hopefully such occurrences are rare and can be considered a special case."What if it’s both high-priority and hard? Like you find out that a fundamental part of how your product works causes data to be miscalculated in a certain scenario, and rewriting it while maintaining backward compatibility is a multi-week task? During which you will get more high-priority bugs…
This article sounds like a fantastic sentiment, but even a small project of a few hundred thousand lines will completely fall over as soon as it’s released into the wild. And if you had a medium-sized project you wouldn’t even make it past the point where internal testing is doing large scenario evaluation.
That said, if it works for your team, go for it. Never throw in more process than you need, but don’t assume that what works for small teams scales. Unless you’re trying to sell a book or consulting services, of course
This comment was originally posted onHacker News
>The only downside to not recording bug fixes is that there is then no > record of previous faults and no bug driven history of why things are > as they are. However, so long as the Black Box tests cover all bugs > as they are found the chance of regression is nil. Therefore I don’t > really see this as a problem at all. Ok, that’s impossible. There’s no way your "Black Box" tests will ever cover every bug you’ll find. Many bugs will be highly data-dependent, often in ways that defy easy replication. Bugs that depend on services being overloaded with network traffic are also very hard to simulate. I think there’s a kernel of truth here though – when someone reports a tiny bug, don’t put it in your bug tracker, just fix it and move on. The time you spend categorizing and describing it could be better used fixing more bugs. And if you’re about to enter a bug report at some ludicrously low priority level, consider not wasting your time.
This comment was originally posted onReddit
Sometimes bug tracking isn’t all about the developer though. What may be a small, ignorable issue to most, won’t be that way to all. By tracking all bugs you at least provide a way of saying "X doesn’t work" and having that so users can find the bug in the tracker to know if it exists and work around it. It’s also important when writing documentation. As far as work flow goes, the article is accurate. From the open source projects I’ve seen, bugs either get fixed immediately, soon, or go years without having a fix (essentially never). There isn’t a lot of middle ground unless an outsider finds it important and contributes a stray patch.
This comment was originally posted onReddit
There’s a major problem with completely ignoring and not tracking ignored bugs:They can come back and bite you. What may seem innocuous in one system now may end up being a huge roadblock later on down the line when you want to extend/modify/update said system with new features. While I understand that some bugs are simply too minor and the edge cases simply too rare to bother fixing, it’s still a good idea to track them.
This also makes certain that they aren’t brought up again as a new bug six months down the line – you can point to the tracking software and say, "Yes. We know about it, and it’s not worth fixing."
This comment was originally posted onHacker News
"What if it’s both high-priority and hard? Like you find out that a fundamental part of how your product works causes data to be miscalculated in a certain scenario, and rewriting it while maintaining backward compatibility is a multi-week task? During which you will get more high-priority bugs…"Perhaps it’s a matter of terminology, if something is going to take a significant block of time and have multiple people working on it then you need some way of planning and tracking progress, your bug tracker may support project planning as well (JIRA perhaps), but used a pure bug tracker will not be as effective in fixing the problem.
"That said, if it works for your team, go for it. Never throw in more process than you need, but don’t assume that what works for small teams scales."
I fully agree with this sentiment, in this case we’re talking about a dev team of 6 working in a broader team of 50 developers. So I wouldn’t describe it as small, though there again if we were 250 developers then certainly, we’d need more process.
This comment was originally posted onHacker News
Bug tracking is about tracking not fixing. So it’s more of a communications tool. If you are solo, you can use more ad hoc systems.
This comment was originally posted onReddit
I’m not sure how much team size is relevant here. You’re still creating project history through a series of discussions and decisions, things that a bug tracker can record quite well if used properly. Every team also has turnover. Having your project’s history recorded in written form can be invaluable for new (and old) developers to research why something was done a certain way, and not have to rely on memory and hearsay to do it.
This comment was originally posted onHacker News
It’s difficult to comment without knowing more about their tests. But I think that if you’re squashing bugs, the first step is replication. If you can’t replicate the bug, whether by data-dependent injection, overloading networks bits, or the like, then how will you know what to fix? Then, if you can replicate the bug, it *shouldn’t* be too difficult to turn that replication code into a test that can be added to the suite. I’m not a TDD fanboy, but I do think that it should cover past bugs to eliminate future changes exposing the same flaws or edge cases.
This comment was originally posted onReddit
It’s good to track low priority bugs; and specifically the customer they affect.In slow weeks (when we have little to do) we pull out the low-pri bugs and fix them – then fire off a personal email to the relevant customer(s). It’s helpful in building an ongoing rapport with your clients.
This comment was originally posted onHacker News
i don’t understand what this has to do with bug tracking software. you can realize the same thing by placing all bugs in the tracker, and still ignoring low-importance bugs.at least that way when the bug re-surfaces, and you discover "its a bigger deal than we thought", you’ll know for sure that you’ve seen it before
This comment was originally posted onHacker News
It’s a good strategy when you’re doing small, quick, low impact releases. When you have to coordinate with other areas of the company about when, and what to release, or when the software becomes so large that there’s a new bug every day, then you need to keep a detailed track of the bugs.
This comment was originally posted onHacker News
What may seem innocuous in one system now may end up being a huge roadblock later on down the line when you want to extend/modify/update said system with new featuresIt may, that’s true. But it’s very unlikely, and what seems like a huge roadblock now in terms of missing functionality is much more likely to be a huge roadblock later as well.
I think the point of this system is not to say that low-priority bugs are pointless – just that there’s almost always going to be something higher priority, so they’re not worth the tracking overhead.
This comment was originally posted onHacker News
You have slow weeks? In a start-up?I’d consider that a very worrying sign.
This comment was originally posted onHacker News
A real benefit I have found in not tracking bugs is that the constraint makes you much less cavalier about introducing complexity in the first place, which is totally a good thing.
This comment was originally posted onHacker News
> If you can’t replicate the bug, whether by data-dependent injection, > overloading networks bits, or the like, then how will you know what to fix? You’ll know when it stops happening. If that’s frequent enough you’ll be pretty sure. If it’s a rare bug it may take weeks or months before you’re sure it’s gone. I fully agree this is non-optimal, but sometimes it’s all you get. You can tell from the error logs what’s going wrong and you have no idea how that could happen. You can either fix it or keep letting it happen. Eventually you’re just going to fix it because allowing a bug to continue affecting your customers is not really an option! One place I see this a lot is parallel programs. Dead-locks are often very hard to simulate in tests, but when they happen you’re definitely going to have to fix them!
This comment was originally posted onReddit
Then you fix them later, when it becomes a roadblock. At that point, you’ll have much more information about why it’s a roadblock and how you should change things so that further development proceeds smoothly. And many bugs may never become roadblocks at all (or may be obsoleted by other changes), so you save all the development effort of fixing things speculatively.I’ve seen a lot of time wasted trying to anticipate what might break later on and bullet-proofing that, when the bullet-proofing simply breaks in new and unforeseeable ways when you actually do try to extend it.
As for running across the same bug six months later – at that point, you probably should reevaluate and decide again whether you want to fix it. The "WONTFIX" designation was made on the basis of information that’s now 6 months out of date – the fact that the bug has come up to bite you again is perhaps a good reason why now it should be fixed.
This comment was originally posted onHacker News
Because you are not big enough to need one?
This comment was originally posted onReddit
I’m a big fan of this approach, but I find that bugtracking is really useful when you want to look back on the project and data-mine to identify problems with your process that you want to fix.For example, if bugs get identified quickly and fixes mailed quickly, but the fixes sit in code review for a long time, maybe it’s time to lean on your code reviewers a bit more. If 50% of your bugs are CSS regressions, maybe it’s time to invest in a CSS-testing framework. If most of your bugs are crashes caused by memory corruption, you need to straighten out object ownership in your C++.
These kind of overall patterns are very hard to discern when you’re considering one bug at a time, yet fixing the root cause of them can lead to big productivity gains.
This comment was originally posted onHacker News
i don’t use bug tracking software because my software is bug free 100% guaranteed all the time coz your love endures forever
This comment was originally posted onReddit
Very true – all the more reason to keep track of it having been reported.
This comment was originally posted onHacker News
Everything is relative. In a startup a slow week might be like a really busy week for a regular 9-5 job.
This comment was originally posted onHacker News
I don’t work in a startup…
This comment was originally posted onHacker News
Because you are stupid.
This comment was originally posted onReddit
I don’t work in a startup… I meant my day job.Regardless, slow weeks are always very useful; time to reassess and plan.
This comment was originally posted onHacker News
I add low-importance bugs that can be done quickly to a "quick" list. If I have some time to spare, I check that list and knock off a couple.Another idea is to simply try find out where a bug might be and add a TODO comment to the file/class. This way, if someone changes that class later they can keep the issue in mind.
This comment was originally posted onHacker News
I can definitely see an argument for logging; particularly as relates to project and developer metrics.But ‘Bug Tracking’ is used to describe much more than ‘Bug Logging’. Bug tracking software packages define themselves and pitch their value based entirely on how they handle that ‘everything else’.
Ergo, if you reject the ‘everything else’ (as the author does), you are rejecting bug tracking software.
This comment was originally posted onHacker News
When you bleed you need band-aid, but it’s best not to bleed in the first place.Ideal situation is where you don’t need a bug tracker.
This comment was originally posted onHacker News