Why I still don’t use bug tracking software

Yesterday I wrote a post entitled ‘Why I don’t use bug tracking software‘. I’ve had some good feedback, and I thought it worth writing a follow up post so that I can respond to common and/or interesting responses without undue repetition.

The first point to make is that the post was not entitled “I don’t use bug tracking software and neither should you”, the purpose of the post was, given the prevalence of bug tracking, and the fact my team have found value in not using it, how many other people have had the same experiences, or is my approach unusual?

Turns out it’s pretty unusual.

So the question really is, what it is that is so unusual about my current context?

First of all some background, my immediate team consists of myself, a tech lead and four developers. We work in a technical team team of ~sixty five of which ~fifty are programmers. There’s a lot of freedom in approach and other teams do use bug tracking to varying degrees.

The company is privately funded and has never taken venture capital,  this means we have grown organically in a calm and conservative manner,  in this context wilfully reckless project management practices simply won’t wash. On the technical side my team has responsibilities for a number of web services, some are very new, others have been in a production environment for almost ten years, all are expected to run continuously.

With that in mind, onto the discussion. I’ve drawn from comments posted on fragile as well as Hacker News. For brevity I’ve taken the liberty of editing some comments down, if you feel misrepresented just let me know.

First up

pht says:
7 April, 2010 at 6:33 pm
A couple of questions to understand your seemingly “dangerous” move :
How close are you to the people specifying the product ?
It seems like dropping the bug tracker could only be manageable if you always have the solution to a bug whenever you face one. From what I’ve seen, bugs are more than often the result of a misunderstanding. A bug tracker can help you keep track of the *communication* required to fix the bug. And in conjunction to your VCS, find the line of code that implemented the proper fix.

This is an excellent point, and one that I ought have brought up in the original post, the system is entirely dependent on the assumption that the team has good visibility over the product, and what’s more good access to those specifying the product such that ambiguities can be ironed out early. I would see this as something that would occur either as part of broader project planning or in the form of feedback post demo. If everyone is face to face then a bug tracker is only one of a number of ways to do this.

What’s the size of your team ?
Again, a bug tracker is really a communication facilitator ; if you’re only a couple of coders, and you’re the ones deciding what goes into your product or not, then this is probably very sustainable. I wouldn’t advocate it for any other kind of set ups …

We’re a team of six working in a broader tech team of sixty five. As you say communication is the limiting factor here. We have mitigated the risk through having a set of well specified APIs and treating internal consumers of the service in the way as for external consumers. Abstracting complexity is hardly a new idea, but it would been very easy to group a bunch of loosely related services together, with a long term consequence of increasing the likelihood of subtle and hard to reproduce bugs.

Is your project outside-world facing ? You mentionned you have QA, but do you also get bug reports / reviews from actual users of the productin system, and if so, how ?

The product is outside world facing, issues are raised through our support and account management teams. As you would expect they have a query tracker, with which we interact in cases where it is not possible for them to answer the query themselves. Only a very small number of these queries can be accurately described as bugs. More often than not they refer to general technical questions about the product, or specific requests about their configuration. Given the nature of our business it ought to be rare for a client to detect a bug, in these cases we would always treat it as a priority.

Benjamin says:
7 April, 2010 at 5:00 pm
I wonder how you document your solution to a problem and how others in your company can find out about it as soon as you are gone (maybe forever).

When the bug is fixed, the integration tests are updated to prevent recurrence, such that should the system regress then it will clear how the behaviour was reintroduced. I consider documentation of how the system works to be a separate problem to bug tracking.

Isaac Gouy says:

7 April, 2010 at 5:10 pm
> “If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.”
Given that you aren’t tracking, how would you know that the same bug has returned at a later date?

That’s good point. Reassessing bugs does require a certain level of continuity, as mentioned above, this approach does require small teams working on largely independent platforms, however in practice we have not found this to be a problem given that the number of bugs we have dropped has been so low. The key point is that even if you had a detailed description of the problem and a justification for not fixing it, all information is now old, perhaps years’ old. The bug could provide a starting point for new investigation, but ultimately the system has changed and old assumptions must be reevaluated.

> “However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term…”
Doesn’t the bug tracker also make it blindingly obvious to everyone in the organization that the team is swamped with bugs which are not being addressed?

Only in terms of the quantity of bugs, which I consider to be a poor metric of load. We do track time spent on unscheduled non project work (including bug fixing) which serves the same purpose.

Chris says:

7 April, 2010 at 7:06 pm
This is a really bad idea. You can fix bugs as they arrive AND keep track of them at the same time. A good bug tracking system isn’t a big hassle to use.
In short, why you would want to throw out a massive amount of potentially useful data is just totally beyond me. There is a lot of good information to be gleaned from bug tracking besides just “which bugs do we need to fix”.
Specifically and in addition to the really good points made in the other comments, here are a few more reasons this is a bad idea:
1. You can’t compare the quality of releases against each other. How many bugs are you shipping with? How many were found?

When you are releasing multiple times a week, I’m not sure how useful this really is. Sure, you want to minimise the amount of time you feel it necessary to spend fixing bugs, but I’d rather spend time understanding how the bug went out in the first and adapting process accordingly rather than tying it back to a specific release.

2. Bug tracking is a useful metric to developers and managers. How many bugs am I creating? Of what severity? Where can I improve?

To me the number of bugs is less valuable tracking the amount of effort spent on bug tracking in total. Severity is interesting, but less so when almost all bugs fit into ‘fix it in the near future’. Continual process improvement is absolutely essential, and aggregation of effort expended unscheduled work is something I already do. Bug tracking software could potentially help with the aggregation of this data, but is certainly not a prerequisite.

3. As the person writing the software you WILL have inconsistencies with your customers in what you consider a must-fix bug. If you ignore a bug and then a support call comes in, there is no way to re-evaluate and learn that maybe you shouldn’t have shipped with that known issue.

Inconsistencies between client and dev team understanding is a good point and covered, in part, above. I would struggle to imagine a situation where we would choose to ignore a bug raised by a client. As part of the resolution we would spend time assessing what we could have differently to prevent the introduction in the first place. Understanding where the communication breakdown occurred is key, improve on that understanding and you reduce likelihood of a similar style of bug being introduced.

nostrademons wrote

I’m a big fan of this approach, but I find that bugtracking is really useful when you want to look back on the project and data-mine to identify problems with your process that you want to fix.
For example, if bugs get identified quickly and fixes mailed quickly, but the fixes sit in code review for a long time, maybe it’s time to lean on your code reviewers a bit more. If 50% of your bugs are CSS regressions, maybe it’s time to invest in a CSS-testing framework. If most of your bugs are crashes caused by memory corruption, you need to straighten out object ownership in your C++.
These kind of overall patterns are very hard to discern when you’re considering one bug at a time, yet fixing the root cause of them can lead to big productivity gains.

I think that this a great point and I can definitely see value in breaking down the amount of effort spent on unscheduled work by category. I don’t think I necessarily need to maintain a bug tracking database to do this if I aggregate on discovery, but as a means of process improvement it could be really powerful in responding to long term trends.

Finally

Ravindra says:
7 April, 2010 at 5:33 pm
Replacing bug tracking by a kanban board !!! Thats one way to physically limit the number of bugs you have.
Bug tracker hosts bugs, with time becomes ‘inventory’ which is a waste.

I agree, there’s definitely a Lean influence in my approach.

So in conclusion, the approach I described works best when the dev team has:-

  • has good access to the client
  • has good visibility over incoming bugs
  • has the freedom to prioritise bugs over scheduled project work
  • runs a system that lends itself to system testing

As a result of feedback I think that, in addition to tracking time spent on unscheduled work, attempting to categorise and aggregate bugs by type could provide interesting process improvement insights.

Why I Don’t Use Bug Tracking Software

Either fix it quickly or not at all.

For many, bug tracking software is central to good software practice. It features as number four of the Joel Test and is used extensively in a Lean or Agile context, as well as in more traditional software development approaches.

My company uses bug tracking software, my team used to use bug tracking software, and now we don’t, so what’s going on?

In general, I think we’re doing pretty well, we use a Kanban board to manage our work flow, we safely release to production five or six times a week, use tests to drive our designs, practice pair programming and have an effective Continuous Integration and Black Box testing set up. We definitely still have areas we can improve on but by most people’s standards we’re on the right track.

So why aren’t we using a bug tracker? Well on identifying a bug any team has four choices:-

  1. fix it immediately
  2. fix it in the near future
  3. add it to a list and plan to fix it at some point
  4. ignore it

Some bugs simply have to be fixed immediately, this is a truism wherever you work. If Google’s search functionality stopped searching, or Microsoft’s Excel stopped multiplying then everything would need to stop in order to get a fix out. Hopefully such occurrences are rare and can be considered a special case.

So we’re left with fixing it in the near future, fixing it at some unspecified point in the future or ignoring it entirely. What I noticed about option three is that low priority bugs are simply never looked at, something more important always gets in the way. So we made a decision, that when we find a bug we either fix it promptly, probably in the next week, or decide simply to ignore it.

This took a some time to get used to. Nobody is comfortable with the idea of dropping bugs, but as soon as we realised that ignoring it and adding it to the bug tracker as a low priority item were pretty much the same thing, we started to feel more comfortable. A bug feels most important on discovery and if we decide not to fix a bug when it’s fresh in our minds and we’re ready to go, realistically, it’s never going to get fixed. If the same bug returns at a later date, well in that case it’s not as small a deal as we thought and we’ll reassess it.

If bugs are to be prioritised over scheduled project work, it’s important to track the amount of effort spent on bug fixing. For my team, this isn’t really a problem since in addition to having responsibility for developing software, we are also responsible for much of our own sys admin, operations and support. Therefore we expect a certain amount of time to be spent on unplanned work and we track time spent on bug fixing as part of this. It turns out that the amount of time spent on unplanned work is fairly predictable and can be accounted for in long term project planning. The only exception to this rule is where the bug is sufficiently large that fixing it will require a significant effort, maybe a week or more’s worth of work. In that case we’ll treat it as a project in its own right and record progress as if it were any other standard planned block of work.

How does this work in practice? The key point is that it encourages us to fix bugs as we find them, and places greater emphasis on not introducing them in the first place. The fact that we know that the bug will be dropped if we don’t fix it in the short term means that we take bugs more seriously. Over the past six months we’ve ignored only two issues, both of which were highly unlikely to recur and both of which resided in third party libraries that we intend to upgrade.

The only downside to not recording bug fixes is that there is then no record of previous faults and no bug driven history of why things are as they are. However, so long as the Black Box tests cover all bugs as they are found the chance of regression is nil. Therefore I don’t really see this as a problem at all.

As an aside, I can see the value in tracking bugs over longer periods in situations where QA and development are separate entities that do not interact as part of the same team, or alternatively where a team is so swamped with bugs that they cannot keep up. However, neither case is particularly desirable and the bug tracker merely helps keep things going in the short term rather than addressing the underlying problems.

So in conclusion, by opting to fix bugs quickly or not at all, we force ourselves to fix more bugs and place greater emphasis on not introducing them in the first place. We no longer have to manage a unmaintainable bug list and ensure that our black box test coverage develops over time.

Edit: This post has generated a good deal of debate, in response I have posted a follow up here where I try to address some of the points raised.

Bad reasons to adopt Kanban

David Anderson recently posted Why do we use Kanban?

He proposes two lists, one entitled ‘Why do we use Kanban’ and a second entitled ‘We do not use Kanban because’, the second list contains complimentary practices that are themselves not a reason to adopt Kanban. For instance it is not necessary to adopt Kanban if all you want to do is dispense with iterations.

The thing I found really interesting about this post is that the reasons my team first adopted a kanban system were drawn almost entirely from the second list. In particular our dissatisfaction with time boxing.

This may, in part, be due to us coming at this very much from a software development perspective, rather than considering the entire value stream.

David lists the following as reasons for practising Kanban

  • Evolutionary, incremental change with minimal resistance
  • Achieve sustainable pace by balance throughput against demand
  • Quantitative Management and emergence of high maturity behavior in alignment with senior management desire to have a highly predictable business
  • Better risk management (the emerging theme in the Kanban community)

It’s only through practising kanban and learning more about the lean principles underpinning it that the real benefits such as improved risk management and greater predictability have become apparent.

Anatomy of a technical Interview

So you decided that you need to hire a new developer , advertised in the right way , vetted the CVs and even thought of some questions . Unless you’re in the 0.1% of people that think that a 90 minute chat is the best way to assess the skills of someone you may end up working with for the next 5 years you’re going to need to think really hard about how to give the candidate the best chance of showcasing their skills in a face to face interview.

This post won’t help with what questions to ask, but it will help with structure and maximising the chance of getting a good performance from a candidate.

Calm them down

First off let’s start right at the beginning, you’ve just met and are about to commence on some small talk, perhaps give an overview of the company – to be honest it really doesn’t matter what you say at this point, the candidate simply isn’t listening. Years of evolution are telling them to fight or flight  and it’s your job to calm them down sufficiently that you can move the interview forward.

Your end goal is to get to the point where you can ask them some really hard questions with a fighting chance of doing themselves justice, but first off there’s ground work to do.

Get them talking

Having helped them settle into their surroundings, it’s time to turn things up a notch and actually get them talking. A pretty good way is to go through their CV, their past work experience, university projects etc. This is stuff that they definitely know more about than you do, and should help them ease into the interview. The key point to remember here is that by asking very open questions and listening to what they consider to be most important or interesting, you can learn a lot about them. Good candidates will start explaining cool and interesting tech that they have used, while others will start talking about co-ordinating this and managaing that. Give people a chance and press them on the tech, but if they’re not confident or worse disinterested, then this is your first warning sign.

During this period you should be thinking about what sort of person you have in front of you, people handle stressful situations in different ways and if you don’t pick up on this you’re likely to miss some good candidates. e.g. if they are shy they’ll need some encouragement or if they are brash and perhaps little arrogant you need to be challenging them and forcing them to justify their assertions.

A final point, and one that I am most guilty of committing, is to make sure that you don’t go too far and overly relax the candidate. Doing so will make it harder for them focus once you start firing technical questions at them.

Getting technical

Before we get into asking any questions, I’d advise the following approach. Present the problem in a form that will probably be too difficult to answer, expect the candidate to need some help and judge their performance on quite how much help they need. This means that you’ll get an idea for how they think, smart candidates will have stretched themselves to solve the problem and poor candidate can at least solve the problem eventually, meaning that they won’t be so traumatised that they can’t concentrate on future questions.

As an initial technical question I’d recommend something pretty straight forward. Something like Fizz Buzz, I find candidates answer these sorts of questions in three different ways.

  • They write out the answer without even having to pause to think and you move on quickly.
  • They struggle a little bit, think things through and get the answer out. This isn’t necessarily a bad thing and might be down to inexperience, but it does mean that they are unlikely to hit the ground running.
  • They really can’t get the answer together and need a lot of leading. Alarm bells are ringing.

Next up I’d raise the difficulty, I’d suggest something wordy and open ended, perhaps some sort of systems level discussion. This means that the candidate can still push the discussion in a direction that they are comfortable with, but make sure that have at least of few things in mind that you will definitely discuss. Again judge them on the amount of leading they need before completing the question.

Now you’re ready to unleash the killer coding challenge, in many respects the whole interview has been leading up to this point, you’ve learnt a lot about them including how to get the best from them, but the next question will really set the best candidates apart. At this point in the interview you also need to consider whether it’s time to bail, if the candidate struggled with the toy programming question and couldn’t impress with your follow up question, then you really have to ask yourself whether it’s worth skipping the final challenge.

I would chose something that is initially tightly bounded but has the potential for significant extension should the candidate be up to it. Exactly what you go for is up to you, but if you don’t stretch the best candidate they won’t want to work for you.

Any questions?

Lastly it’s time to ask the candidate if they have questions, this in itself can be illuminating, though to be fair at this stage, the best they can do is reinforce a good performance. Questions about working attire and hours are generally a bad sign especially if not a final round, questions about version control, project management practices or something that came up as part of the interview are generally good things and provide an opportunity to sell the role.

After the interview

Once the interview is over, it’s time to come up with an decision. How you do is will be entirely down to your specific circumstances. For first rounds I tend to allocate a point for their programming, their systems and general technical common sense and finally a point for how well I think they’ll fit in. Anything over two and half, i.e. a strong performance in two out of the three and a good performance in the third is a thumbs up. For final rounds you can’t afford to let them leave the room if you still have any concerns unanswered. This is someone who you could end up working with for years so err on the side of caution.

Ultimately though, the only way to arrive at a good system is to have a lot of practice. Make sure technical staff are involved at the earliest possible stage and if you make a bad hire, don’t blame them, blame your process and learn from you mistakes.

In Defence of ‘Scrum but…’

Scrum, as a framework, is practiced in many different organisations and can be considered a mainstream approach to software development. To me agile is way of scaling common sense and while I no longer practice Scrum I consider it an excellent implementation and would happily return to it if my circumstances were better suited to time boxing. However as with all great ideas that are adopted widely, there are many instances where people are simply doing it wrong.

Warren Buffett has a neat way of describing this:

First come the innovators, who see opportunities that others don’t. Then come the imitators, who copy what the innovators have done. And then come the idiots, whose avarice undoes the very innovations they are trying to use to get rich.

In the case of software it’s reasonable to replace the goal of getting rich with that of successfully delivering software that is valued by the customer.

As such there are a wealth of posts discouraging the adaption and variation of Scrum, in fact the notion of Scrumbut is so well understood it even has a definition.

scrumbut [skruhmbut] noun.
1. A person engaged in only partially Agile project management or development methodologies
2. One who engages in either semi-agile or quasi-waterfall development methodologies.
3. One who adopts only SOME tenents of the SCRUM methodology.
4. In general, one who uses the word “but” when answering the question “Do you do SCRUM?”

Project management will necessarily vary from company to company. As such the ubiquity of a term like Scrum has less value within a specific company, since the shared understanding of the company’s approach ought to be evident to all. When I talk to people outside of my company it’s useful to reference commonly understood terms as a starting point. For time boxed approaches Scrum is an obvious point of reference, but the specific system that I wish to describe is highly likely to deviate from Scrum in some way and I do not necessarily see this as a bad thing.

The challenge of software development is to produce something that someone will find valuable. There are plenty of different ways to achieve this and all are highly dependent on the context of the project. Generally an iterative and incremental approach that bakes quality in from the start is preferable and a number of ready made implementations exist such as Scrum, Kanban or Evo. In some cases it may be appropriate to lift something like Scrum wholesale, but in many other cases, there are legitimate reasons for adapting the process. For example a team might be writing an implementation of a system where the behaviour is well understood. In fact in order for the system to be compliant and therefore usable it must adhere to a large pre-written spec provided by an external party. Clearly this is at odds with Scrum but there is no need for the team to drop time boxing or stand ups as a result.

So adapting a process is not in itself something to be frowned upon. Clearly bad project management is bad, and if the manager blames their own failings on their inability to implement a methodology then larger problems exist. The key point is that good managers should not worry that they are running Scrumbut, or indeed if they are ‘doing it right’ as the only metric for correctness is that the team maximises the value they create.

How to do Nothing

Doing nothing is harder than it looks. As a developer I sometimes found it hard to understand what on earth my boss did all day. In fact the better the boss, the less they seemed to do.

Fast forward a few years and I found myself in charge of other developers, it quickly became apparent that doing nothing was actually quite a challenge. Try as I might something always came up, and I found myself doing something, which as previously mentioned was not a good sign. It turns out that doing nothing is pretty much a full time job, and I’ve put together some tips on how best to achieve it. Now bear with me, some of these activities do technically involve doing something, but if you are careful not to get caught no one will ever know.

  • Anticipate rather than react.

In some senses dashing in to save the day at the last minute is exciting, all the more so if your hair happens to be on fire at the time. However, there is no surer way to appear to be doing something than by intervening in this way. It is much better to notice that something will need to be done ahead of time and to act before anyone else notices that you are doing anything at all. This does mean that you will need to be constantly on the lookout for things that need doing before anyone else notices, however it is a small price to pay for doing nothing.

  • Maintain relationships outside of the team.

Occasionally, despite your best attempts at anticipating problems, your the team will get blocked and require some action on your part to resolve the matter. This is a dangerous situation since if they have nothing to do they are more likely to notice that you are in fact doing something. Therefore you need to act quickly. At this point it’s important to know the right person to talk to and what’s more have a pre-existing relationship with them. That way it will appear that you are just having a nice chat rather than actually doing something.

  • Big visible task boards.

A sure way to get caught doing something is by having your team ask you about what to do next, you can easily get around this by delegating the responsibility to a whiteboard. That’s right you can use an inanimate object to replace a previously essential function. To make this work, it does mean that you may need to do some planning ahead of time, it might even mean talking to more commercially minded folk on the other side of the office but nobody ever said that doing nothing would be easy.

  • Team collaboration

The more time the team spend working together, sharing ideas and learning form one another the less time they have for catching you doing something.

  • Small incremental changes

If you must be seen to make a change, at least be sure that it feels like a natural progression and sensible to all, that way your doing something won’t be as obvious. Once the change has been made, it is critical that you resist the temptation to tinker, not only will tinkering make it more likely that you will be spotted doing something, you’ll never learn what effect your change brought about, which means you might miss out on discovering new ways to do nothing.

  • Inspect and Adapt

Further to the previous section why not ask the team themselves to contribute ideas in order to refine their working practices, this way you can actively do nothing in their presence.

  • Hire great people

Unfortunately there is no getting around this one, while there are plenty of excellent guides available (1,2,3), sadly all of them advocate that you do something. However since it is likely that much of the process will occur in an interview room you can at least hide the fact that you are doing something. You may also find that you spend much of the time listening which can be easily be mistaken for doing nothing.

  • Commit to personal development

It’s much easier to do nothing when your team comprises of highly skilled geniuses. By working hard to consider the needs of all team members you can help them approach highly skilled genius status thereby reducing the need for you to do things. Over time you may even find that they appreciate taking on greater responsibility, further reducing instances where you will appear to be doing something.

Finally, having assembled and developed a great team, don’t blow it all by checking up on them every forty five minutes, just stand back and do nothing for a bit.

I gave up Caffeine

Last Spring I attended QCon London, during the course of the conference I attended Linda Rising’s talk entitled Agility: Possibilities at a Personal Level

It was suggested that in the same way agile teams rigorously apply a regime of inspect and adapt, the individual should do the same at a personal level. As an example, she asked why tea and coffee are so prevalent in the workplace and linked this back to the working practices of the industrial revolution. Specifically, that while caffeine was very useful in making sure unskilled workers arrived at the factory on time and alert, she questioned what penalties we incur force fitting these practices onto a skilled craft like software development?

Just to be clear, Linda Rising stood up in front of a room of computer people and said ‘Stop drinking coffee’ and at no point did anyone try and throw her out of the window. So it was a pretty good talk, so good in fact that immediately afterwards I decided to give up caffeine to see how I would react.

Firstly a bit of background. My company supplies free coffee beans and making coffee is a complex and communal ritual, verging on the sacrosanct. The etiquette has been built up over years, and even offers rival schools to subscribe to. We have Wiki pages detailing coffee brewing best practice and it’s no joke to say that my social standing and ability to make good coffee are linked. In short it’s a bigger part of my day than just a caffeine high.

I wasn’t a big coffee drinker by any stretch of the imagination, the coffee was definitely strong but I’d only have two or three cups a day. Nonetheless going fruit tea total came as a bit of a shock.

The first two weeks were pretty miserable, I found that I got very tired towards the end of the day, and where previously I might have stayed late if it meant finishing something off, I was pretty useless after 6pm. I also missed the ceremony and social interaction, sure I still went to the kitchen and brewed my cup of lemon and ginger (or whatever) but the whole process was deeply unsatisfying. I went through all manner of cravings which was strange since at weekends where coffee really isn’t part of my routine I hardly noticed the difference.

After having got through the hard part I started to notice some benefits, while the tiredness was still there, it was much more manageable, and crucially the knowledge that I couldn’t just stay late meant that I was more focussed during the day. I became more disciplined at trying to perform the tasks that required the most energy earlier leaving repetitive, less creative activities for the last few hours. Despite working less hours overall I think that I achieved a similar amount.

A second benefit that sounds silly in hindsight was that I started sleeping much better. As a student I wasn’t a big coffee drinker but I’d always been one to lay awake at night pondering some puzzle or other. So when this behaviour continued into my working life I didn’t consider it strange. Giving up caffeine meant that I had a fighting chance of falling asleep not long after I switched the lights out, which in turn meant I had less need for an early morning coffee.

As an Englishman, it was unlikely that I was ever going to last without tea in the long term. For a start my Nan would have struggled to comes to terms with the change. So after a few months I felt that I’d served my time and now allow myself tea and the odd coffee in the morning, interestingly though I can no longer run with my old coffee pals as the strength of their brew gives me headaches.

So the key points to take away are:-

  • Stopping completely after only a few cups a day led to notable and sustained withdrawal symptoms.
  • Not having the safety net of being able to stay late at work meant that I worked more effectively during the day.
  • Cutting out caffeine in the afternoon made a real improvement to my sleep patterns.

Kanban in practice

My team has been using a Kanban board to manage our day to day work flow for the past few months having switched from time boxed iterations, what follows is a description of how we use the board in practice. This post concerns itself only with day to day development and does not cover high level planning.

First some context. The team is made up of four developers, a technical lead and myself as project manager. We are responsible for the ongoing development, deployment, support and maintenance of six hosted products that are expected to run continuously. We would expect to release new functionality to the live environment a ‘few’ times a week.

The Board

Our is board is positioned in clear view of the team and we conduct stand ups in front of it.

Example Kanban Board

The board contains 5 sections

  • Not Started – Stories we have committed to completing and that can be started at any time.
  • In progress – Stories where development and verification are underway
  • Ready to deploy – Stories that are ready to be deployed into the live environment
  • Done – Stories deployed live
  • Blocked – A place to track stories that have become blocked by parties outside of the team as project manager it is generally my responsibility to unblock these stories.

We limit work in progress in two places, currently we assign roughly a week’s work for ‘Not Started’ and a further week’s work for the combined total of ‘In Progress’ and ‘Ready to Deploy’.

We do not consider these limits to be ‘hard’ and will from time to time exceed them, however in doing so we will have had a conversation to ensure that we are happy to be exceeding the limit temporarily.

Stories

Stories are represented by post it notes, in addition to a brief description of the story each card is presented in a standard form like so:-

Post It Example

Size

Many lean practitioners view estimation to be a form of waste and therefore something to eliminate. Estimating is still valuable to us as it provides a means to build consensus over what the story entails, this is especially useful for less experienced members of the team who may not have considered all aspects of the story fully.

Project

While each story is a deployable artefact they generally form part of a larger set. By grouping projects in this way the story description can be simpler since the project itself provides context.

Dates

We record the date a story entered the ‘Not Started’ state, the date we began work on it and the date the story was deployed into the live environment. Doing so allows me to answer questions like ‘if we were to start a three point story right now, how long would it take to get it live?’, this in turn allows me to justify spending a fortnight on improving our deployment process as the positive impact is there for all to see.

Description

We try to keep the description as brief as possible, by ensuring that lead time is minimised to a few days it is practical to communicate decisions verbally. For complicated stories we can provide more specific details via a wiki or a tool like agilefant.

Pink Notes

In addition to planned project work we will receive a certain level of support and maintenance requests, that we represent by pink post its. These tasks are large enough to add up, but each one will typically take no more than half a day. Anyone within the team can create a pink note and we ensure that our board has enough slack to accommodate these new tasks.

New pink notes are discussed at stand up, after which there are three outcomes:-

  • The task has been prioritised and someone is already working on it (this is rare and occurrences should be minimised)
  • The task is added to our standard work flow and does not receive special prioritisation – we work on the assumption that our lead time is sufficiently low to respond quickly.
  • The task is removed and ignored. We explicitly do not use bug tracking software reasoning that tasks we choose to ignore when they are fresh in our minds are unlikely to be returned to.

The volume of pink notes is monitored so that more realistic estimates can be made for project completion as well as tracking gradual increases and decreases over time.

Maintenance

Smaller day to day maintenance tasks like reviewing error logs and looking into monitoring alerts/trends are handled by a designated maintainer. One off tasks that will take more than a few hours to complete can be added to the board as a pink note. The role of maintainer switches on a weekly basis amongst the developers and the aim is that the non maintaining developers can focus on project work and/or pink notes.

Performance Tracking

We attempt to track team performance over time so that we provide estimates for project work with greater confidence. It also means that we can track the impact of changes to the team e.g. what is the impact of moving a developer on loan to another team?

To do this we use two different methods. The first draws on Aslak Hellesøy’s work, and measures standard lean metrics such as lead time and WIP. The second provides some supplementary metrics such as finer granularity over what types of work were delivered on a weekly basis and finer granularity of lead time between phases. Aslak’s sheet generates a number of graphics, one being a cumulative flow chart which neatly visualises our recent Christmas change freeze.

A second sheet of my own making produces a chart for velocity, this is an agile rather than a lean metric, but I find it useful to track where we are spending our effort. The yellow ‘Total Velocity’ line saw tooths at about ~ 14 points, suggesting that we work to a fortnightly rather than weekly cadence. It also shows that during late Nov early Dec we spent more time on unplanned pink notes than we did on project work. Much of the pink note load will have come from outside of the team so it’s important that we have a way to make the impact visible.

The Switch to Kanban

Recently my team have been trialling the use of a Kanban board to manage our work flow and in doing so have dropped time boxing. So the far the switch has been very positive, while it may not be an appropriate transisition for all teams I thought that it might be useful to share our experiences.

In brief, Kanban is an agile development methodology that does not prescribe time boxing (i.e. fortnightly iterations), rather it focusses on limiting work in progress. For each stage of a story’s life span e.g. ‘in progress’ or  ‘ready to deploy’ the team has a maximum amount of work that can exist. As stories are completed the team pull new stories from the project backlog. For more information I would recommend Henrik Knibergs excellent comparison of Scrum and Kanban

Why Kanban?

We originally adopted time boxing about 6 months ago, the key benefits we were looking for included:-

  • A sense of rhythm and points to reflect on our working practices.
  • Better visibility over tasks that were dragging on.
  • A highly visible feedback loop to help improve our estimations.

For the most part the adoption of time boxing was very successful, we’ve certainly improved our abilities to estimate, and generally we have better visibility over our performance. The thing that didn’t work so well was working to the self imposed iteration deadline since it often felt false and sometimes was counter productive.

Any team working to an iteration will occasionally finish early or fail to complete all tasks, it’s not ideal but manageable so long as the team continually refines it’s working practices to minimise occurrences. In my team’s specific case this is especially hard since so much of our work is dependent on our interactions and integrations with companies much larger than ourselves. Experience tells us that it is practically impossible to predict the time spent on an integration and even harder to guess at what point the time will be spent. As a result, despite working in a predictable manner on the majority of our iteration goals it was common that we would under or over shoot.

This is hardly the end of the world though it started to chip away at the sanctity of the iteration and continually raised questions of why the system was pushing us in ways that felt wrong to us i.e. if we finish early then we can take another story from the back log so long as it fits into the time remaining, however ideally, for a given project, we’d want to the story with the greatest risk attached which may or may not fit in the time allotted. It felt that time boxing was nudging us away from this idea.

Furthermore, my company is such that our developers typically have a very good level of product domain knowledge, as such the need for a formal Scrum style Project Owner is reduced. This means that while we would demo our work to interested parties on completion of features, come the end of an iteration it was rare to demo our progress.

It was clear that while providing some benefits we were fighting against time boxing and we needed to find an approach that better described our work flow. What we were aiming for was not so much to complete a set of tasks in a pre-allotted time period, more that we should be able to regularly produce deployable features in a predictable manner. By limiting the work in progress rather than limiting the work per time Kanban presented a viable alternative we felt better reflected how we actually work, while preserving the discipline necessary to deliver working software multiple times a week.

Initial findings

Since mid iteration our practices were already similar to that of Kanban we found that migration to the system caused very little disruption and we did not notice a productivity hit while we got used to the new system.

The migration has brought about subtle but positive changes to how we work, largely due to the fact that by explicitly limiting the amount of current work it forces conversations to be had where we are close to violating the limits.

In addition to making it easier for us to refine our process, the main benefits so far include:-

  • Greater flexibility in our work flow, we can respond in days rather than the three weeks a two week iteration promises. This is key, as delivering supplier driven changes quickly provides significant competitive advantage.
  • On a related note, we no longer feel that we are fighting our process. It is no longer the case that we regularly take a course of action that violates our process (albeit for perfectly sensible reasons).
  • The board provides a highly visible place to display tasks, communicate priorities and highlight bottle necks. These benefits are not Kanban specific however Kanban’s greater emphasis on use of the board (through tracking work in progress limits) means that any benefits are magnified.
  • It saves us an iteration meeting – the role of the meeting is covered in stand ups or in project specific design meetings.
  • Our stand ups have become much more focussed, this is partly because we do them in front of the board. Again this is not Kanban specific but in Kanban running them anywhere else makes no sense at all. Id’ say that we’ve halved the length of time we devote to stands ups, where further discussion is necessary it is taken outside of the stand up.
  • Our retrospectives are no longer coupled to the period of our iteration, currently we still run retrospectives fortnightly, though since our stand ups are now more effective for day to day tweaks we may extend this period.

In terms of cons, thus far we do not feel that we have lost anything by adopting Kanban, the only potential problem that we have encountered is that greater discipline is required in ensuring that all tasks are completed in a timely manner.

The future

So far we are very happy with Kanban and will continue to use it.

In order to measure performance over time Kanban teams track two metrics.

  • Throughput
  • Lead time – the time it takes for a feature to be delivered after the team has committed to its delivery.

At present we do not have enough data for meaningful results but long term we will maintain something much like this as suggested here.

Further reading

In addition to Scrum vs Kanban comparison referenced above you may find the following useful.

Finally not everyone is as sold as I am on Kanban for some anti Kanban debate see:-

I found the comments that follow the posts to be of particular interest.

Agile Testing Meetup

The other week I attended the Agile Testing Meetup group at the new Skills Matter offices. The purpose of the group is to encourage debate and share ideas relating to testing software in an agile environment.

The session started with a short presentation on Fitnesse and finished with a general discussion.

Fitnesse is an interesting tool, but probably not of much practical use in an environment where acceptance tests are largely conceived by developers which, rightly or wrongly, is true in my case.

The general discussion focussed on problems faced by testers in an agile environment and yielded the following list:-

  • Differences between environments (e.g. Dev, Staging, Testing)
  • Testing is always low priority.
  • Time to Automate tests is longer that manual testing.
  • Unclear requirements can make testing difficult.
  • Development overruns into Testing.
  • No time to “skill up” (train) testers.
  • Hard to get a shared understanding at the right level.
  • Dependency of developers.
  • Environments and switching between them.
  • Testers not involved in specification until too late.
  • Organisations definition of Quality.
  • Agile Onshore + Offshore + Distributed teams communication.
  • Employing Contractor Testers result in a lack of continuity.
  • Trying to practice agile in a non-agile environment.

What I thought was interesting was that most of the points seemed to have little to do with agile and much more to do with testing in general.  The point is that agile methods won’t solve your problems they just make it easier to spot and solve them.