Performance Reviews

One of the things that makes my commute to work enjoyable (next to the BBC Friday night comedy hour) is the LeanBlog podcast. In it Mark Graban talks to different people about Lean. The people he talks to have varying backgrounds, and the general quality of the podcast is very high.

One of the podcasts I listened to a while back was with Samuel Culbert, who was talking about performance reviews. I was reminded of that episode tonight while talking with some ex-colleagues about the way performance reviews were done in our old company. We weren’t very enthousiastic.

Now you might think that that is not too strange. People are very rarely enthusiastic about their performance reviews. The way most appraisal systems are set-up is that only a small percentage of people if allowed to receive a very high score. The idea is that there are a very few high-performers that should get an extra pay raise or bonus (or both), but that their occurrence should be very rare. The result is that for most people, performance review meetings contain lots of unpleasant surprises.

Why? Well, one reason is that managers are only allowed a quotum of high performers. This certainly was the case in the company I was talking about with my ex-colleagues, and going against that got me into some trouble at the time. So the team that worked fantastically together, and managed to bring a very difficult project to a successful conclusion against the odds saving the company millions. That team only gets one person to have the highest level pay-raise. The others, who worked just as hard, and were just as indispensable for the results, get a disappointment. Not that they don’t get a raise, but for those people the manager needs to emphasis the things that they did wrong. To be able to defend them not getting the top score, despite their success.

Lucky for the manager/team-lead there is always something wrong to be found! Lucky for the company, they hire completely random, so their bell-curve of performance always works out… Actually, Culbert mentions that these distributions usually mean that about 70% of the people should be marked as average. That is going to be difficult, isn’t it?

But the appraisal is necessary, right? To ensure people stay motivated. To ensure they’re focused on improving. To set explicit targets, both for their work (because you can’t expect them to understand what’s important for their day-to-day work), and for their personal development (because… sorry can’t seem to think of a reason).

Mr. Culbert makes some good points in the podcast, calling performance reviews “corporate theatre,” as well as a “sham,” a “facade,” “immoral” and “intimidating”. Unsurprisingly, since he’s written a book called “Get rid of the performance review“.

His latest book is apparently called “Beyond Bullshit“, which sounds like another one I should pick-up.

My own views on this practice is aligned with Culberts: at best, a performance review doesn’t add any value; mostly, it is harmful for people and company.

If the feedback you as a manager give during a review is in any way new to the person you’re reviewing, then you’ve failed as a manager to do your job (which should have been helping that person to do their work as best as possible, not collecting information on how they do that work for later use). If not, why have the meeting.

If the review is about verifying whether targets have been met, or to set new targets, you really should read Dan Pink’s Drive about all the ways the use of targets is hurting motivation, and thus performance, of your people.

Kanban for a Recruitment Agency

At Qualogy, where I work, we have a team that acts as a recruitment agency for freelancers towards our customers, so that we can help our customers fill open positions even when we don’t have a suitable internal candidate. This team’s main goal is to find suitable candidates as quickly as possible, offer them to the customer, and help both to come to a mutually acceptable agreement.

The team of three that takes care of this work was looking for ways to improve the service to their customers. That meant being quicker to offer possible candidates to customers, but also to provide faster feedback towards (rejected) candidates.

As a way to achieve this, we decided to try a basic Kanban system, with the process from receiving the request from a customer to either an accepted candidate, or all candidates rejected and notified.

We started out by charting the existing process and identifying the parts that were in our control, and those that are not. There were obvious places where we would simply be waiting for a response from a customer (‘We do/don’t want to talk to this candidate’, and ‘We do/don’t want to take on this candidate’). This made the initial sketch of the process look as follows:

Process sketch

Process sketch

We played around a bit to fit this partially conditional flow on a Kanban board. Initially we had the idea of a linear board with each state in our process mapped to a column, where columns could be skipped if they were not applicable. This left us a bit unsatisfied, as part of the agreed process was not really explicit. By looking at  the whole of the process as consisting of two parts, first finding candidates, and secondly offering them to customers, we got closer to a working solution. Then by making the difference in flow for accepted (at least for an interview) and rejected candidates explicit on the board, with the token from the first part of the process (finding candidates) cut in half between candidates offered, and those rejected, we ended-up with the following structure:

Initial Kanban

Initial Kanban

As part of the use of Kanban, we also wanted to keep track of how long it takes to go through the stages of the process. As is recommended practice, we added the time for relevant states to each token:

Cards with dates

Cards with dates

Making the process explicit, by setting-up the board and clearly agreeing on what the work in the different stages should consist of, is the first basic step when starting with Kanban. It gives a lot of clarity for the team, and makes it easier (possible) to start discussing improvements in process with a common understanding.

One thing that anyone familiar with Kanban will find puzzling is that we didn’t specify any WIP limits on the board. We did discuss doing that, of course, but we decided to wait. One approach to setting the WIP limits would have been to start with the number of people in the team, and use that as a starting point. It is very normal in this type of work for one person to be handling multiple cases, though, and no one was sure how many. Another approach is to simply look at a normal week, and see the maximum number of items there are in any given column during that week. Though we started with this approach, demand was too low to be able to come up with any useful figures, so for now we are still WIP-less.

After some initial apprehension on the introduction of the board, the reactions to working with the Kanban were very positive. The perception was that there was more clarity as to what was in progress, and what needed to be picked up. The daily stand-up in front of the board took some getting used to, but also helped with the division of work amongst the team members. The clear state and frequent communication was seen as a very good way to be able to fill-in for colleagues when they were away.

After one month of this process, we held a retrospective to see whether everything was going well, and what we could improve. The retrospective turned up some interesting points.

On the positive side, better communication, both within the team as towards customers and candidates, was seen as a very positive point. The visual process had also much improved keeping all the administrative tasks up-to-date, which had previously often been collecting into large backlogs.

On the improvements side, there was worry about tasks that do not fit into the standard process, as those are not visible, and thus parts of the work that people do is not visible. A further point was that after an interview at a customer, sometimes it would take a long time before the customer reacted, and thus a long time before we could provide any feedback towards the candidate. This would create extra work, as the candidates would start calling regularly to request status updates (a very good example of failure demand).

As a result of the retrospective, the team decided to annotate tokens that were in the ‘Interview’ column with dots, which would be added at each stand-up meeting. And after a set amount of dots (which equals to a set amount of time since the interview), contact would be made pro-actively with the customer, and after that with the candidate, to keep everyone up-to-date.

Another agreement from the retrospective was that any status change would always be immediately updated on the board. This would make taking over any tasks by another team member much easier. The team already thought this had improved much in the new process, and wanted to further improve in that regard.

Finally, it was agreed to put up a separate board for non-applicant work, so that that could also be tracked. This would be a generic ‘to-do, in progress, done’ style board for any other type of work. The reason there was suddenly a different type of work for the team was a need from another department in the company, due to illness there. The work was completely unrelated to the process discussed above, and the best we could come up with was using a separate board (or separate ‘lane’ on the current board, but that didn’t work out with the widescreen board we were using). This doesn’t help much in showing if specific items are delayed by the ‘extra’ work, but at least it does show what work is in progress at any time, indicating a ‘total’ delay.

The next step for this team will be in finding out what could be useful WIP-limits (Work-In-Progress limits, which are agreements on how many items can be in the same state on the Kanban board, which should expose any bottlenecks quickly). So far the demand has been low enough that there has been no real opportunity to find out where the bottlenecks are. It’s expected that as the market becomes more active, the first bottlenecks will be automatically revealed.

Scrum vs. Kanban: A Game of Life?

I’ve been following some of the discussions on the differences between Scrum and Kanban. And learning more about Kanban, of course. One point that is emphasized a lot is that Kanban requires fewer up-front changes than Scrum does. The term “Big Change Up-Front” has even been coined, by Alan Shalloway.

There’s certainly truth in that. Scrum doesn’t have many rules, but it is very strict in assigning a very limited set of roles and responsibilities. Kanban can be used with existing roles, as long as you make make sure you make the existing roles and policies explicit. Asking which one of those option is better is really beside the point. It simply depends on the context. In my situation, I usually get called in by companies who have already decided to ‘go Agile’, and as such are already part way through some of those changes. Of course, the changes are not always successful, but it doesn’t provide me with a change to start slowly.

Interesting discussion, of course, and for me it brought to mind Conway’s Game of Life. For those unfamiliar with it, this is a cellular automaton game, where a set of rules iteratively executed over a set of cells (with a state of on or off), where all kind of interesting stable and continuously changing patterns can occur based on the initial pattern put on the board.

Scrum could be compared with a fairly big and complex ‘breeder’ pattern, which needs to be placed on the board as a complete set. It’s quite an apt comparison in that the infinite growth belonging to such a pattern doesn’t happen if you get part of a breeder pattern wrong. And since you can see the Game of Life as a universal Turing machine, infinite growth means a continuous generation of information, which can be as continuous learning.

Kanban can start with an existing stable pattern (an oscillator), and can tweak that to move, step by step, towards breeder status. At least, that’s how I see it.

The analogy will probably break down after this initial thought, and might be made better if we move from comparing to patterns to comparing to rule-sets, but my brain started to hurt when I went along that route…

Scrum vs. Kanban? Not really…

Peter Stevens, over at 

Scrum Breakfast has an interview up with Mary Poppendieck on Lean, Scrum, Kanban and Leadership. The part of the interview that caught my attention was a question on the relationship between Scrum, Kanban, and Lean in general.

I like Mary’s response a lot, where she basically states that Scrum and Kanban each have their own strengths, and each is suited for their own specific set of circumstances.

Scrum is basically a method of accomplishing work through cadenced iterations. Kanban is a method of accomplishing work through limiting work-in-process and managing flow. I have found that some work especially creative work is more effectively managed with iterations, while other work especially naturally sequential work is more naturally managed with Kanban. — Mary Poppendieck

She also stresses that, whether you choose to use Scrum or Kanban, the point is that you keep improving on your way of working, so:

Lean would have every company view these techniques as starting points that are constantly improved, so after a few years, Scrum and Kanban should evolve and change to something quite different than their starting point. — Mary Poppendieck

This suits the way that I view these things very well. Use the tools most suited for the situation, and see where it leads you.

Of course, the best way to choose is to try each, and measure results. Which brings us to another question: what and how do we measure? At the moment, I’m leaning towards flow (time for work to flow through the system), and Henrik Kniberg’s Happiness index. Getting that last one adopted anywhere is going to be an interesting challenge, though…

Learning is key

An old article I just came across, posits that learning is the thing of value in software development:

When we present this hypothetical situation to students – many of them with 20+ years experience in building software – they typically respond with anywhere between 20% to 70% of the original time.  That is, rebuilding a system that originally takes one year to build takes only 2.5 to 8.5 months to build.  That’s a huge difference!  It’s hard to identify another single factor that could affect software development that much!

The article goes on to discuss in detail how learning is enabled by agile processes. The advantages of quick feedback cycles are not just ‘fail early’, but also ‘learn early’ (and continuously).

Agile Feedback Loops

Agile Feedback Loops

Lean and Kanban Europe 2010 – Part 2 – Mary Poppendieck

Mary Poppendieck: What’s this thing called “Pull”

Mary Poppendieck had the second talk of the first day of the conference. She talked about “The power of pull”.

She started the presentation with a story on her introduction to pull at the video tape factory where she worked when video tape was still current. I didn’t make too many notes here, since this story is very well documented in her books (which I can thoroughly recommend: Implementing Lean Software Development and Leading Lean Software Development).

The emphasis she put on participation in this story was relevant, though. She described very nicely how the main reason that thenew ‘Pull’ system that she and her colleagues introduced was so succesful was that it was designed by the people who actually implement and use the system.

Next, Mary went on to something I hadn’t heard before. Referring to Andy Grove’s Only The Paranoid Survive, she explained the concept of the ‘Strategic Inflection Point”. She claimed that the move from push to pull will create a strategic inflection point.

Strategic Inflection Point, copied from Mary's sheets

When describing the strategic inflection point for software development in general, characterising the left side of the chart as “1.0 – Contract Focus, ‘Waterfall’ (-2005)”, the middle “2.0 – Development Focus – ‘Agile’ (2005-2010)”, while the right, upward climbing, line is “3.0 – Customer Focus (2010-)”.

The elements of this customer focus are given as:

  • Team vision and initiatives
  • Validated learning
  • Customer discovery
  • Initiating changed

One thing that came back a number of times during the conference was references to Kent Beck, who has been giving a lot of though on the ‘Building the right thing, not just building it right” side of things. Mary referred to a keynote he gave at the ‘Startup Lessons Learned” conference. And since that got me curious, here’s a link to the video, sheets and some earlier discussion. Good stuff!

She could link that to Ethnography and Ideation, ideas that are part of their ‘Leading’ book, where really understanding the job at hand (i.e. “the thing you’re going to be automating in this project”) is so very important. Understanding that the purpose of your software is to make a job easier, faster, better, etc. And understanding the job well enough that you can appreciate the value gained doing the job in a new way.

Next came a bit that links closely to what seems to be becoming a recurring speech from me, about how quality and discipline in development are crucial to make an Agile (and Lean :-) implementation work.

Pull systems tightly couple the value stream.

Meaning, of course, that by taking much of the delays and queues out of a value stream, there is less room for the extensive rework and fixing cycles that you may be used to from a waterfall environment. And of course, if you’re used to it, you’ll probably also depend on it. The only way to make Agile and Lean really work is by concentrating on quality and discipline in the technical side of the work.

She then explained John Boyd’s OODA (Observe->Orient->Decide->Act) loop, which includes the conclusion that the faster you can go through this loop, the higher your chances of winning. Which in Boyd’s case meant not being shot out of the sky. So better listen to the man’s thoughts on this!

And you prepare to be able to do that loop quickly by learning to be fast with good quality.

Dan Pink‘s ‘Drive‘ got mentioned. Another recurring theme during the conference, btw. Mastery, Autonomy and Purpose, and in particular this video:

Dan Pink on motivation (try if the above link doesn’t work)

Continuing on that theme, Mary ended with an overview from Steve Weber’s book on ‘The Success of Open Source‘. The way that open source project work be definition needs the contributors to be motivated to contribute. The ‘tragedy of the commons‘ would seem to indicate that this can’t happen, but apparently it is possible. The reason it is possible is that people will willingly share their expertise (and work) if they are intrinsically motivated to do so.

So if open source projects routinely manage to do this, maybe we should look at the processes involved to find out how to motivate the people working in our companies.

Product development needs to sell the product roadmap to the team

Some of the ideas I found interesting from that are “Treat workers as volunteers” (so management is a marketing job!), The above quote of selling the roadmap to the team, and the idea that in a modern market, variety is the thing that will safeguard a company’s survival, not efficiency.

Next time, John Seddon’s closing keynote from day one, which really blew me away both in contents and presentation.

Lean and Kanban Europe 2010 – Part 1

I attended the Lean and Kanban Europe conference last week, and I thought I’d do a little write-up to share my impressions.

As an Agile Coach and Scrum Master, I was going to this conference to find what parts of Lean and Kanban (in that order) could be of use for me when helping clients improve their way of working. I was in particularly interested in ways of extending the values and principles of Agile into the non-IT parts of an organisation.
I was not disappointed! The central theme of the conference was very much about enabling organisations to provide better value and service to their customers. As is appropriate for a Lean conference, the ‘how’ of doing that was all about Systems Thinking, Flow, and Eliminating Waste. And about pushing responsibility and decision making down to the people doing the work.Lean and Kanban Europe 2010 Logo

The general atmosphere at the conference was very open, speakers were mingling with everyone else during the breaks, and always welcomed questions and discussions. The focus of most presentations was on explaining why certain Lean principles work, and what common problems they solve. There was also enough focus on practical implemtation examples, and during the Open Space part of the conference, there was also room for some games that illustrated things very clearly.

Agile vs. Lean?

Before I left for the conference, I ran into a thread on the yahoo scrumdevelopment list that discussed whether Lean and Kanban were Agile, or not. The discussion was quite intense, but as far as I can see there is so much more overlap between the Lean and Agile approaches that to most outsiders the two would be pretty much the same. The general ideas at the conference also seem to support this idea. Lean is seen as a way to achieve ‘organisational agility’, and can be used in combination with many existing Agile approaches. There does seem to be some strife between some members of the two communities, though, some of which did come out during David Anderson’s presentation. Apparently he wasn’t made very welcome with a talk about Kanban at an Agile conference a few years ago, and some of that discussion still gives some friction. Mostly though, the general feeling was that Agile methods don’t always succeed in triggering required change in the broader organisation.

Agile also doesn’t always give much of a handle on deciding *what* to build. It promotes customer interaction, but especially in an enterprise scenario, that doesn’t mean you’re actually building the right thing. This theme returned in a number of talks, ans very notably in the keynote by John Seddon, who impressively presented a usecase from the Portsmouth housing authorities the demonstrated systems thinking very convincingly.


I’ll give my main impressions on the talks I attended below and in follow-up posts, starting with the opening keynote by Alan Shalloway.
Maarten from AGILEMinds told me that the slides, and probably videos, of most of the talks would be put up on the conference website. That’s very nice, and also means that I won’t need to give a blow-by-blow description of the contents of each talk. I’ll just be giving the highlights and my impressions for each one.

Opening Keynote by Alan Shalloway on September 23, 09:15-10:45: What is Next in the Agile World: Why Lean/Kanban Matters

Alan Shalloway (, @alshalloway) opened the conference with a keynote talk that discussed the way that Lean thinking can be used to expand the tenants of Agile towards the broader organisation. He started out with a quote from Ken Schwaber where he states that only about a quarter of the teams that *say* they use Scrum are actually really doing something that can be called that (thanks to an update from Alan, I now have a reference for this:

Below are my notes on the talk, which I thought very well performed. He certainly managed to capture my interest with lots of examples that were very recognisable. It was a good introduction to Lean and Kanban, and as such was very accessible for those of us that have not had much experience with implementing Kanban.

lkbe2010 Keynote Alan Shalloway

Alan Shalloway giving the keynote (Photo courtesy of @karelboekhout)

He then went on to explore some of the reasons that these initiatives don’t reach their full potential. In general he says it is because Scrum, though seen as having a fairly simple process/framework, is still too complex or difficult to do. And perhaps too disruptive if done right. He sees a number of corporate Agile anti-patterns:

  • Business
    • Poor Prioritisation
    • Too many projects!
  • Teams
    • Poorly formed teams
    • Teams that don’t know how to do Agile
    • Ineffective release processes
    • Technical Debt
  • Management
    • Management not helping teams
    • Resistence to change

A wonderful quote on that last item:

People know what to do, but they don’t do what they know.

Which is something that I have seen many times. We know exactly how to make sure we do everything right, but seem to put that knowledge aside the moment the pressure is on. And often even when there’s no pressure at all.

He then went on to look into more detail at some of the issues that organisations have organising their technical work. I certainly hope that the slides for the first of those will become available, because I very much liked the clarity they gave on all the problems you can/will get when you arrange your teams by component/specialisations instead of by functional areas. I won’t try to reproduce them, but the message is clear, and familiar: splitting work by component will create hand-overs and delays, and should be avoided.

He also gave a nice overview of how most project organisations created a very high load on their most critical people. People with specific knowledge, experience and/or skills need to be involved in so many different projects that they get overloaded and become constraints on those projects. And thus on the organisation. Mostly this is done out of a misguided attempt to ‘optimise utilisation’, but having everyone allocated to a project 100% of their time doesn’t mean that they’ll be creating the most value for an organisation.

Busy doesn’t mean productive!

In the next bit Shalloway explained what he called ‘induced work’. He explained how, when you ask a group of programmers what they spend most of their time on, they will often say: ‘fixing bugs’. And that he then tells them they’re wrong! They’re not spending their time *fixing* bugs, they’re spending most of their time *finding* bugs. And that’s usually true, in my experience. Once you know what’s wrong, you mostly know how to fix it. And the fresher in your memory the original work is, the easier it is to find problems in it. All this is of course another way to reiterate the familiar truism that the earlier you find a problem, the cheaper it is. Fresh bugs are easier to squash! This means that any delay in the process increases work, and thus makes the process more expensive.

The message is that it’s not just about ‘eliminating waste’, it’s about *preventing* waste.

In informal queries, people estimate the amount of ‘induced work’ in their companies somewhere between 30 and 70 percent. That’s a lot.

Then Shalloway went on to some of the management issues trying to do (especially large scale) scrum. The problem that even with a well designed multi-team approach, the tribal nature of people still brings unwanted competition and separation between the goald of the teams. He also commented on the common problem of management declaring ‘Thou Shalt Be Agile’, and expecting that to be sufficient…

After this description of the limitations and challenges of Agile implementations, he went on to discuss how Lean, and Kanban, provide tools to deal with these kind of issues.

The first is the fact that Kanban allows teams to start from where they are now. It doesn’t require the kind of (sometimes disruptive) changes that a Scrum of XP implementation does. Of course, many people would call that disruption a positive thing, but there are plenty of organisation which are simply not able to deal with that level of abrupt change. A Kanban implementation start by simple modelling the current process, and value stream, giving insight in where any problems lie.

Sometimes chaos is creative, but most of the time it’s just madness.

He sketched a progression of production styles, going from the original ‘Craft’, to ‘Interchangeable Parts (1800s)’, to ‘Interchangeable People – Assembly Line (1900s)’, to the new ‘Engaged, Thinking People – Lean (2000s)’.

Lean management depends on the ability to ask the right questions. It’s not a matter of changing the culture of a company. Culture can’t be changed directly. You have to change the management system.

Culture is the result of the management system

In that same way, you can’t decide to ‘change productivity’, influence ‘trust’, etc. You can influence the system, and build people and those other properties can/will emerge.

The basis of Lean is the continuous process improvement. You take small steps, often, in the right direction.

Focus on a process improvement to achieve a specified improvement, not to achieve a specified $ return.

All this results in the following definition of Lean Management:

Lean Management means setting up an environment in which people can learn to do their work better.

There followed some advice on things to keep in mind when you start with implementing Kanban in your organisation, for which my notes are less extensive (wait for the video! :-), but do include the following quotes:

A Kanban board is a representation of your model, not a process to be followed.

So if you can’t draw up your Kanban board, you don’t know what you are doing!

At the end there was a nice statement on how you still have to make sure that what you are building is solid (“Load bearing walls”). He referred to an article on his company’s website (, on “What a programmer should know before I’ll let him touch my code.”. I haven’t been able to locate the correct link yet, but I’ll update this post when I do.

That’s it for now. Next time I’ll post my notes on Mary Poppendieck’s ‘What is this thing called pull’ presentation.