How to appraise a developer (part 2)

This post forms part of a series, if you’ve not done so already you might want to check out part 1.

So last time I talked about why on earth you might want a performance appraisal process, this time around I want to go over the scheme I came up with. At this point, I’d like to stress that an annual review is useless without continual follow up via day-to-day interactions as well as specific time set aside for 1-1s. Furthermore, without good management you will always fail, whatever the process.

The key aims of such a scheme are to

• Provide junior and mid range staff something to aim at and work to.
• Provide senior staff who do not wish to enter management an alternate career path without the fear of adversely affecting their seniority or salary.
• Be flexible enough to account for nuances in the role of individuals

Warning

This post contains phrases like ‘Core Knowledge Area’ and ‘Capability Level’, I have managed to avoid ‘Leverage Synergies’ but you get the idea. You have been warned.

Areas of investigation

Before starting it seemed sensible to see if there was anything already out there that I could make use of. By far and a way the best resource I found was the Construx Professional Development Ladder, which in turn is heavily based on SWEBOK.

I also looked into Dreyfus modelling, which is a way of characterising improvement in a particular skill and used by coaches of all disciplines, from what I can tell the SWEBOK (and Construx) capability levels are based on the Dreyfus approach.

Overview of the scheme

Overall performance is split into a number of Knowledge Areas, this means that strong or poor performance in one area does not affect performance in another unrelated area. Once defined, Knowledge Areas are assessed against standardised Capability Levels and the resulting spread of skills is reduced to an overall Ladder Level.

This means that people with wildly different roles can be expressed in terms of one another, this is absolutely crucial in tackling the question of progression for senior developers who do not have formal management responsibilities.

The appraisee has full visibility over the analysis which forms the basis for further professional development and only the final aggregated level is made public.

Knowledge Areas

Before anything can be done it’s necessary to find a way to break down the role into specific skills, this means that feedback can be provided in a very targeted manner. Good appraisers do this naturally but since the feedback needs to be quantifiable the extra layer of formality is necessary.

I’ve identified 15 knowledge areas, these are largely based on the SWEBOK areas with a few additions/alterations to reflect my specific working environment. Were I thinking about including a wider range of roles (e.g. Sys Admin or Support) or to consider such a scheme for another company, then this table would need to be adapted.

The full list lives here, but a couple of examples include :-

Knowledge Area	Description
Coding	The creation of software according to a specified design. The primary activity is creating code and configuration data to implement functionality using the selected languages, technologies, and environments.
Design	The bridge between requirements and construction, design defines the structure and dynamic state of the system at many levels of abstraction and through many views.
Technology	The use of tools, technology, methodologies, and techniques for software engineering.
Operations	System installation, deployment, migration and investigation. Working safely with live systems.

Core Knowledge Areas

It is not the case that all developers should expect to score highly in all areas, in fact the point of the scheme is to recognise variance in roles. As such some Knowledge Areas will be unavailable to some appraisees, Recruitment is a good example. However it is important to recognise that some Knowledge Areas areas are central to the performance of the individual regardless of what else they are able to contribute and that they are expected to score well in these areas. For example it is hard to envision a Software Developer for whom Coding is not a core skill however it may not be necessarily be a core skill for a Project Manager.

For each individual the appraiser must select a a group of Core Knowledge Areas to reflect what is most important about the role. Roughly ten Core Knowledge Areas from the total fifteen feels about right.

Capability Levels

Once the Core Knowledge Areas have been identified, it is time to grade each level. This part is fraught with problems and I’d be kidding myself if I thought this step could be completely fair. However I found the following to be a serviceable starting point.

Capability Level	Description
Introductory	The employee performs or is capable of performing basic work in an area, generally under supervision. The employee is taking effective steps to develop his or her knowledge and skills. In general the employee looks for support and guidance from the immediate team
Practitioner	The employee performs effective, independent work in an area, serves as a role model for less expert employees, and occasionally coaches others.
Leadership	The employee performs exemplary work in an area. The employee regularly coach employees and provides project as well as tech team level leadership. The employee is recognized within the company as a major resource in the knowledge area

In the future it may be necessary to define a level of achievement above Leadership, in which case it could be defined as “The employee is universally acknowledged by those who have achieved Leadership in a particular knowledge area to be a company expert in the area – essentially providing leadership for those already at Leadership level”.

Mapping actual performance to Capability Levels is made much easier through example and I’ve included a selection broken down by Knowledge Area here. In the general sense I have found the following to be a good guide.

Introductory	Rarely coaches others.
	Looks to the team for support and guidance.
Practitioner	Occasionally coaches others, usually those at Introductory level
	Occasionally introduces new ideas and practices to the team.
Leadership	Regularly coaches others, their own skills and expertise are amplified through the help that they provide.
	Regularly introduces new ideas into the team with examples of where these ideas have been adopted.

Ladder Levels

Once the appraisee’s Knowledge Area’s have been defined and assessed an overall ladder level can be assigned. This is important because this is the only part of the scheme that it made public to the company as a whole. In my current context three levels feels most natural. Other teams may benefit from a higher resolution.

Level	Requirements	Description
i	Default grad level	Entry level position, makes a positive contribution to the team but requires support and guidance from those around them.
ii	3 areas at Practitioner and Introductory in all other Core Knowledge Areas	A key member for their team, they are able to work with little guidance and act as a trusted peer to other grade iis as well as providing support to those less experienced. Team members on this grade will have a few years commercial experience.
iii	3 areas at Leadership, 8 areas at Leadership and Practitioner combined and finally Introductory in all other Core Knowledge Areas	A leader at development team level and a significant contributor to the tech team as a whole.

Summing up

So in summary, look at the appraisee’s role, pick the most important attributes and mark them as ‘Core’. Then grade them against all Knowledge Areas, this forms the basis for further professional development. Turn the handle and convert the assessment into a ladder level and hey presto people with disparate job roles can be recognised for their seniority regardless of whether they are pure devs or pure managers, or most probably some combination of the two.

As with anything, I’m always keen to reduce process to the barest minimum possible without things  descending into chaos and I’m aware that at first glance this scheme might feel quite heavy. However without formally breaking down a role, I can’t see a fair and transparent way to appraise technical staff.

Next time round I’ll post some worked examples of how to apply this in practice.