The Human Side of Agile

If I have a criticism of the agile movement it is that not enough time is given over to the role of management, and furthermore not enough care is taken over the distinction between project and line management.

In a study by VersionOne that looked to asses the reasons for failed agile adoptions, almost all could be traced back to reluctance or failure to engage on the part of management and little wonder when the role of leadership is left so ambiguous.

Gil Broza recently wrote a book entitled the Human Side of Agile that aims to address the human aspects of agile implementations, in particular offering practical guidance to how an agile team leader might incorporate these ideas into their role.

Prior to publication, Gil asked that I contribute an anecdote and I was only too happy to oblige – the chapter was on the subject of making the most of your immediate environment. I drew anecdote from my old office where space was in short supply.

It may not always be possible to create the perfect working environment, however it is important to make the most of what is available. My team were looking to map their work flow using a white board and sticky notes. Unfortunately we were situated in the middle of an open plan office without access to walls, nor did we have the necessary space for a for a free standing white board. In the end we bought a roll of white board sheeting and applied it to a nearby structural pillar. Work items flowed from top to bottom and space was tight, but it served our purpose and is still in use years later.

Elsewhere in the book Gil also references How to do Nothing.

Great stuff Gil.

The Complex, the Complicated and what it means for NASA

When people say ‘Simplifying Complexity’ I hear ‘I don’t know what these words mean’. This isn’t entirely fair, as we shall see, but it gives you an idea of my preconceptions before attending a training session of that title.

The facilitator started off by inviting each member of the group to talk about the influence of complexity in their professional lives. As we went around the room it became clear that for most of the group complexity was a synonym for hard or complicated. Many would agree, including the Oxford English dictionary. However, here was a group dominated by senior members of a very large IT professional services company. These are people for whom complexity science is highly relevant to their professional lives and people for whom there ought to be value in defining Complexity with a big C and in particular differentiating the Complex from the Complicated.

Cynefin

The Cynefin Framework characterises systems (and problem spaces) into five classes – Simple, Complicated, Complex, Chaotic and Disordered. Once the class of a system is identified the framework provides guidance on how to work most effectively with the system. On a personal level I find it very useful in explaining to me not only why an incremental and iterative approach generally works well in software development, it also suggests to me under what circumstances such as approach does not hold. You won’t see NASA using Scrum for instance, more on that later.

Cynefin can be expressed visually like so:-

cynefin_as_of_1st_june_2014-1

Credit Wikipedia

The key distinction I see is that of Complex and the Complicated.

Complicated

An example of a complicated task is building a submarine. It’s definitely not trivial, but if the tasks are broken down sufficiently the problem is tractable and predictable – essentially it becomes a series of Simple tasks. By contrast a Complex task retains its complexity even after being broken down.

Complex

An example of a Complex task could be to ask a room of people to arrange themselves such that the closest person to them is exactly half the distance from them as that of the 2nd closest. It would be extremely difficult for an individual to orchestrate this task, instead each individual actor must make a small change, observe the consequence and then make a further change based on the feedback.

What does this mean for Software?

Taking a manufacturing production line as an example, once it reaches steady state the system can be described as Complicated and is predictable. This is important because for many years Software Development looked to the world of manufacturing for guidance. In doing so, implicitly defining software development as a Complicated task. The thinking being that with sufficient up front analysis the problem could be solved without a line of code being being written. It is for this reason that the Waterfall model rose to such prominence and was so readily adopted.

More recently manufacturing has been shown to be an inadequate metaphor and that in fact software development is more akin to product design, thereby inhabiting the Complex quadrant. This means that the correct approach, according to Cynefin, is to probe, sense and respond. In effect the iterative and incremental approach practiced by those inspired by Agile and Lean thinking.

So what does this mean for NASA?

If what I say is correct, what does this mean for NASA? After all they have some of the smartest brains on the planet available to think about this sort of thing and yet their processes appear to assume a Complicated rather than a Complex environment.

NASA is an extreme organisation and it is only natural that what works for them will deviate from the common case. Despite the challenging nature of their work I would argue that their domain is Complicated rather than Complex.

Complex systems are characterised by there being many unknowns and an inability to determine the nature of those unknowns at the beginning of the process. In NASA’s case they are genuinely in a position where, for software at least, the problem space is well defined.

For instance, the software runs upon hardware based on Intel’s 386 architecture, hardly cutting edge, but whose behaviour (warts and all) is well understood and has not changed in years. This means that the inputs to the system are well understood and the behaviour deterministic. NASA has managed to turn what would ordinarily be a Complex system (that of software development) into a Complicated system and in doing so their software teams work to vanishingly small defect rates, albeit at enormous cost and at the expense of delivery time. I found this article to be a fascinating insight into their working practices.

In conclusion, Complexity science provides a means to characterise systems, the Cynefin framework provides definitions to differentiate the Complex from the Complicated. Software development is almost always Complex though it was originally assumed to be Complicated. This is why agile and lean approaches have been shown to much more effective than traditional methods that assume a Complicated system such as Waterfall.

Too Much Trust

Trust trust trust trust trust trust trust trust trust trust

Excerpt from the management book I wish someone would write

A central theme in agile software development is that of trust. The agile (small a) movement speaks of openness, collaboration and collective responsibility – none of which are possible without trust. As a manager my team cannot be effective if they do not trust each other nor can I bring about anything but the most superficial change if they don’t trust me.

I’m not the only one who feels this way, turns out I’m in good company 1 2 3

So I like trust and consider it to be a ‘good thing’ but the point of this post is not to talk about how great it would be if there was more trust in the world. In fact I want to talk about situations where increasing trust can actually be destructive.

The total level of trust is undoubtedly important, but equally important is the distribution of that trust. The greater the differential between the relationship containing the most trust and that containing the least the less chance that the overall group can act as effective team.

A good high level example might be an engineering org and a sales org. It doesn’t matter how much internal org trust exists – if org to org trust is low the company will not perform as well. In fact the lack of inter org trust will felt all the more keenly in contrast to the strong internal trust that exists.

Applying this idea to a single engineering team, if a team has high trust for one another and a new member joins then it will take time for that new member to earn the group’s trust and be accepted as part of the team. This healthy and only natural. However if the team is split down the middle with two groups of high internal trust who do not trust one another then strengthening internal group trust will only entrench the distrust of the other group. In this case increasing trust can actually be harmful.

What I’m saying is that the effectiveness of a group to act as a team can be characterised by the weakest trust links in the group. If the differential between relationships is high then increasing trust in already strong relationships can actually hinder rather than help the team.

From a practical perspective, the manager’s job is always to create an environment where trust can grow, but it is important to focus on the low trust relationships since they are the ones that characterise the effectiveness of the team.

Hell is for Heroes

Last December I attended the London XPDay, the session I enjoyed most was run by Chris Matts on the subject of Heroes in the context of software development teams.

What makes a Hero?

Chris put forward the idea of the ‘Hero’, an unofficial role assumed by an experienced developer critical to the project. There are positive aspects to the role, this is person turned to in the moment of crisis when something must be fixed ‘right now’, they are the person with the deepest knowledge of the system and an indispensable contributor to the project. As with many things, however, this strength can also be a weakness. The feeling of being indispensable is very powerful and freely sharing knowledge and collaborating with other less experienced/capable team mates only undermines this feeling. If the Hero is no longer the only person who can fix the problem, surely that makes them less important?

In extreme cases the presence of the Hero becomes toxic, reluctance to collaborate is unpleasant, but active obstruction and obfuscation is something else. At this point the team/project has some serious problems. On the one hand it is doomed to failure without the Hero, on the other-hand, the ability of the group to act as a team evaporates and progress on the project is brought to all but a standstill.

What to do when a Hero goes bad?

We spoke for about an hour on the subject and while there were one or two examples that partially dealt with the problem, ranging from ‘move them to another project’ through to ‘fire them’ (!), no-one in attendance was able to provide a truly positive example of recovering from such a scenario.

I am fortunate in never having worked with anyone quite as extreme as the examples presented in the session, but where I have seen glimpses of this behaviour my sense is that overwhelmingly, the behaviour is the product of environment rather than necessarily any failing on the part of the individual.

The participants in the session, seemed to be made up of managers/coaches rather than out and out developers, which may explain why much of the discussion seemed to presuppose that the fault lay solely with the Hero.

Common environmental factors that I have observed include:-

Perceived ambiguity over who has technical responsibility for a system
Poor performance feedback and/or poorly communicated career development
Lack of trust/respect amongst team mates
Seemingly overwhelming operational issues
Compensation schemes pitting team mates against one another

It is the manager not the Hero who has most influence over these points. So I think that before answering the question ‘What to do when your Hero goes bad?’, a better question, as a manager is ‘What have I done wrong to allow my Hero to go bad?’.

Focus on Teams

Unhelpfully, as with all management problems, the best way to solve the problem is not to have it in the first place. Placing greater emphasis on the performance of the team rather than on the individual can help here. Any action that benefits the whole team is recognised and celebrated so the Hero need not lose prestige by supporting those around him. In fact the Hero’s standing increases since he is now multiplying his own capability through increasing the skills of the team. As a side effect, since the team is now more capable the Hero has more time to spend on the truly difficult problems, which in time, he will pass onto the rest of the team.

Switching focus away from individuals and towards the team is a non trivial exercise, but if the agile movement has brought us anything it is methods to engender collaboration, trust and team level thinking.

Why I don’t use bug tracking software – slides

In addition to speaking on Kanban at XP Day 2010 I also gave a short lightning talk based on my earlier fragile posts on bug tracking (1,2). Initially I was apprehensive about standing up in front a room of agilistas and telling them I’d dispensed with digital bug tracking, but since the previous session had been about throwing everything out (really really everything, just deploy to live), I felt positively conservative.

[slideshare id=6265230&doc=bugtracking-101220175702-phpapp02]

From Scrum to Kanban – slides

Last month I attended the excellent London XPDay. This year, XP Day was largely made up of open space discussions but there was an ‘Experience Report’ track where I gave a presentation on switching my team from Scrum to Kanban. The talk draws heavily on previous fragile posts (1,2), but also reflects what I have learned during the interim.

[slideshare id=6265160&doc=switchtokanban2-101220175109-phpapp01]

Motivation (almost) for free

Individuals and interactions over processes and tools

So reads the first line of the Agile Manifesto. Agile (and Lean) frameworks differentiate themselves from traditional software project management in the value that they place in people.

I’m always confused when I people attempt to separate out line and project management in the context of software. To me they are intrinsically linked, and cannot meaningfully be considered in isolation. In fact I’d go so far as to say that one of the principle drivers of the agile movement is not so much what it says about project management, significant as that is, but more what it says about motivation and simply letting smart people get on and do their jobs.

As an illustration, I’d like to use Martin Haworth’s ‘Ten Tips About People Management’ as a representative article of a more general approach to people management. For each point I’ve highlighted how it is commonly addressed in the context of an agile team.

1. Talk to Your People Often
By building a great relationship with your people you will bring trust, honesty and information. This gives you a head start in Performance Management of your people.

Daily stand ups provide a frequent and regular period to interact with the team. It doesn’t provide one on one time but it does mean that raising impediments and sharing of ideas is common place such that it then feels much more natural for one on one conversations to occur.

2. Build Feedback In
On the job two-way feedback processes gets rid of the nasty surprises that gives Performance Management such a bad name. By building it in as a natural activity, you take the edge away.

Agile is all about rapid feedback. Both at a technical level in the form of TDD and CI and a personal level through daily stand ups and a commonly agreed definition of done.

3. Be Honest
By being frank and honest, which the preparation work in building a great relationship has afforded you, both parties treat each other with respect and see each other as working for everyone’s benefit.

With lead times measured in days, trust and openness are essential for any agile team, where honesty does not exist the whole process collapses.

4. Notice Great Performance
When you see good stuff, shout about it! Let people know. Celebrate successes and filter this into formal processes.

At a team level daily stand ups and visualisation of work flow provides this for free. Furthermore, since features are delivered in a state ready for production, regular product demos provide the customer with a hands on measure of progress.

5. Have a System
Performance Management is a process and needs some formality – especially for good personnel practice and record. This need not be complicated, but it needs to be organised and have timescales.

Agile frameworks have little to say directly on the subject of performance management, though there is an assumption that team members are continually looking to improve their skills and performance. Agile working models introduce the idea of cadence where there is a period dedicated to retrospection and continuous improvement.

6. Keep it Simple
But do keep it simple. If you have a relationship with your people that is strong anyway, you already know what they are about. Formal discussions can be friendly and simple, with formality kept to a minimum.

With an emphasis on verbal communication, it’s easy to have serious conversations in a relaxed but constructive manner.

7. Be Very Positive
Celebrate great performance! Focus on what’s going well. It’s about successes and building on strengths, not spending ages on their weaknesses – that serves no-one. Go with the positives!

Again, constant feedback through regular delivery of working software, reinforces and encourages good practice.

8. Achieve Their Needs
Remember that we all have needs that we want fulfilling. By working with your people to create outcomes that will do this, you will strengthen your relationships and channel effort in a constructive direction.

Since teams are typically cross functional, team members are exposed to a range of challenges, and have a clearer idea of where they feel they are strongest. While an agile framework does not specifically look to fulfil the longer term needs of an individual, it at least attempts not pigeon hole them into a specific roles.

9. Tackle Discipline
Whilst it often happens, Performance Management is not about managing indiscipline. That has to be managed in a different way. By setting clear standards in your business that everyone understands and signs up to, discipline becomes much, much easier.

In the same way that team success and good performance are highly visible to the team, individual poor performance is also obvious. Expectations of good practice are generally arrived at through team consensus and so could not be clearer. If the expectations are inappropriate or unrealistic then the team has the power to amend as necessary.

10. Learn from Mistakes
As part of regular on-the-job and informal review, mistakes will come to light; things will go wrong. By using the ‘What went well? And ‘What could you do differently?’ format, the unsatisfactory performance becomes controllable and a positive step.

Where teams are correct to take take credit for their success, it is equally important that take responsibility for failure. Examples include retrospectives or Lean style 5 Whys root cause analysis.

Software is an industry not always known for the strength of its people management, by pushing human issues to the fore agile and lean frameworks ensure that not only is the management of of the project considered, but also the management of the people. In a practical sense, while applying agile principles won’t necessarily make me a good manager, they make me less likely to be a bad one.

Agile makes me SMART

Performance review 101, objectives should be SMART. Theoretically, SMART objectives make an awful lot of sense to me, but when asked to set objectives for a developer to cover the next 6 months it really doesn’t feel practical.

Normally, where I have a management problem, the agile community is a great source of inspiration and advice. It turns out that while it has little to say about performance review (why should it?), I’ve been using SMART objectives all along in the form of stories.

Let’s have a look at what SMART means and how it applies to stories.

Specific

By breaking down epics into manageable features it’s possible to isolate a small chunk of behaviour, such that all involved have a clear of idea of the scope of the problem.

Measurable

Acceptance tests provide a clear means to determine when the story is done. In many cases the tests can be expressed in terms of ‘Given a set of preconditions, when a specific event occurs, expect the following behaviour’, this serves to reduce ambiguity and provide black and white tests to determine success.

Attainable

Stories are estimated by the developers charged with their implementation. If the developers cannot see a way to approach the problem then it will not be possible to provide an estimate and the story cannot be accepted. At this point the story will either need to be broken down, or reconsidered before being re-estimated in its new form.

Relevant

The story must in some way relate to the bigger picture, to do this stories refer to features recognisable by the customer as opposed to tasks recognisable only to the development team.

Time Bound

Working in time boxed iterations, or alternatively a work in progress limited flow, means that clearly defined expectations for delivery exist. Where these expectations are not met is is clear to all and it is time to discuss what went wrong.

I have a long held belief that one of the main reasons that the agile movement has been so effective is that it builds good management practices in from the ground up. SMART objectives in the form of stories are just one example.

In Defence of ‘Scrum but…’

Scrum, as a framework, is practiced in many different organisations and can be considered a mainstream approach to software development. To me agile is way of scaling common sense and while I no longer practice Scrum I consider it an excellent implementation and would happily return to it if my circumstances were better suited to time boxing. However as with all great ideas that are adopted widely, there are many instances where people are simply doing it wrong.

Warren Buffett has a neat way of describing this:

First come the innovators, who see opportunities that others don’t. Then come the imitators, who copy what the innovators have done. And then come the idiots, whose avarice undoes the very innovations they are trying to use to get rich.

In the case of software it’s reasonable to replace the goal of getting rich with that of successfully delivering software that is valued by the customer.

As such there are a wealth of posts discouraging the adaption and variation of Scrum, in fact the notion of Scrumbut is so well understood it even has a definition.

scrumbut [skruhmbut] noun.
1. A person engaged in only partially Agile project management or development methodologies
2. One who engages in either semi-agile or quasi-waterfall development methodologies.
3. One who adopts only SOME tenents of the SCRUM methodology.
4. In general, one who uses the word “but” when answering the question “Do you do SCRUM?”

Project management will necessarily vary from company to company. As such the ubiquity of a term like Scrum has less value within a specific company, since the shared understanding of the company’s approach ought to be evident to all. When I talk to people outside of my company it’s useful to reference commonly understood terms as a starting point. For time boxed approaches Scrum is an obvious point of reference, but the specific system that I wish to describe is highly likely to deviate from Scrum in some way and I do not necessarily see this as a bad thing.

The challenge of software development is to produce something that someone will find valuable. There are plenty of different ways to achieve this and all are highly dependent on the context of the project. Generally an iterative and incremental approach that bakes quality in from the start is preferable and a number of ready made implementations exist such as Scrum, Kanban or Evo. In some cases it may be appropriate to lift something like Scrum wholesale, but in many other cases, there are legitimate reasons for adapting the process. For example a team might be writing an implementation of a system where the behaviour is well understood. In fact in order for the system to be compliant and therefore usable it must adhere to a large pre-written spec provided by an external party. Clearly this is at odds with Scrum but there is no need for the team to drop time boxing or stand ups as a result.

So adapting a process is not in itself something to be frowned upon. Clearly bad project management is bad, and if the manager blames their own failings on their inability to implement a methodology then larger problems exist. The key point is that good managers should not worry that they are running Scrumbut, or indeed if they are ‘doing it right’ as the only metric for correctness is that the team maximises the value they create.

Kanban in practice

My team has been using a Kanban board to manage our day to day work flow for the past few months having switched from time boxed iterations, what follows is a description of how we use the board in practice. This post concerns itself only with day to day development and does not cover high level planning.

First some context. The team is made up of four developers, a technical lead and myself as project manager. We are responsible for the ongoing development, deployment, support and maintenance of six hosted products that are expected to run continuously. We would expect to release new functionality to the live environment a ‘few’ times a week.

The Board

Our is board is positioned in clear view of the team and we conduct stand ups in front of it.

The board contains 5 sections

Not Started – Stories we have committed to completing and that can be started at any time.
In progress – Stories where development and verification are underway
Ready to deploy – Stories that are ready to be deployed into the live environment
Done – Stories deployed live
Blocked – A place to track stories that have become blocked by parties outside of the team as project manager it is generally my responsibility to unblock these stories.

We limit work in progress in two places, currently we assign roughly a week’s work for ‘Not Started’ and a further week’s work for the combined total of ‘In Progress’ and ‘Ready to Deploy’.

We do not consider these limits to be ‘hard’ and will from time to time exceed them, however in doing so we will have had a conversation to ensure that we are happy to be exceeding the limit temporarily.

Stories

Stories are represented by post it notes, in addition to a brief description of the story each card is presented in a standard form like so:-

Size

Many lean practitioners view estimation to be a form of waste and therefore something to eliminate. Estimating is still valuable to us as it provides a means to build consensus over what the story entails, this is especially useful for less experienced members of the team who may not have considered all aspects of the story fully.

Project

While each story is a deployable artefact they generally form part of a larger set. By grouping projects in this way the story description can be simpler since the project itself provides context.

Dates

We record the date a story entered the ‘Not Started’ state, the date we began work on it and the date the story was deployed into the live environment. Doing so allows me to answer questions like ‘if we were to start a three point story right now, how long would it take to get it live?’, this in turn allows me to justify spending a fortnight on improving our deployment process as the positive impact is there for all to see.

Description

We try to keep the description as brief as possible, by ensuring that lead time is minimised to a few days it is practical to communicate decisions verbally. For complicated stories we can provide more specific details via a wiki or a tool like agilefant.

Pink Notes

In addition to planned project work we will receive a certain level of support and maintenance requests, that we represent by pink post its. These tasks are large enough to add up, but each one will typically take no more than half a day. Anyone within the team can create a pink note and we ensure that our board has enough slack to accommodate these new tasks.

New pink notes are discussed at stand up, after which there are three outcomes:-

The task has been prioritised and someone is already working on it (this is rare and occurrences should be minimised)
The task is added to our standard work flow and does not receive special prioritisation – we work on the assumption that our lead time is sufficiently low to respond quickly.
The task is removed and ignored. We explicitly do not use bug tracking software reasoning that tasks we choose to ignore when they are fresh in our minds are unlikely to be returned to.

The volume of pink notes is monitored so that more realistic estimates can be made for project completion as well as tracking gradual increases and decreases over time.

Maintenance

Smaller day to day maintenance tasks like reviewing error logs and looking into monitoring alerts/trends are handled by a designated maintainer. One off tasks that will take more than a few hours to complete can be added to the board as a pink note. The role of maintainer switches on a weekly basis amongst the developers and the aim is that the non maintaining developers can focus on project work and/or pink notes.

Performance Tracking

We attempt to track team performance over time so that we provide estimates for project work with greater confidence. It also means that we can track the impact of changes to the team e.g. what is the impact of moving a developer on loan to another team?

To do this we use two different methods. The first draws on Aslak Hellesøy’s work, and measures standard lean metrics such as lead time and WIP. The second provides some supplementary metrics such as finer granularity over what types of work were delivered on a weekly basis and finer granularity of lead time between phases. Aslak’s sheet generates a number of graphics, one being a cumulative flow chart which neatly visualises our recent Christmas change freeze.

A second sheet of my own making produces a chart for velocity, this is an agile rather than a lean metric, but I find it useful to track where we are spending our effort. The yellow ‘Total Velocity’ line saw tooths at about ~ 14 points, suggesting that we work to a fortnightly rather than weekly cadence. It also shows that during late Nov early Dec we spent more time on unplanned pink notes than we did on project work. Much of the pink note load will have come from outside of the team so it’s important that we have a way to make the impact visible.