Agile On The Beach Talk

Ciarán and I had a wonderful time at the Agile on the Beach conference this last week. We did the first full version of our talk: “The ‘Just Do It’ approach to change management”.  I did an earlier version of the talk at the DARE conference in Antwerp earlier this year, but this longer version has gone through quite a few changes in the mean time.

agile-on-the-beach

The conference was set-up very well, and it was great to talk to so many people working on Agile in the UK.

The slides for the talk are up on slideshare:

We got some really nice responses, including:
The next chance to catch us is at the Lean and Kanban Netherlands conferene (“Modern Management Methods: Making Better Decisions”) conference in Maarssen on 7-8 October. We’ll have a new iteration of the talk, of course. Always on the move:-)LKNL-im-a-speaker-badge
UPDATE: The video of the talk was just released, and can be found on the conference website. Our talk can also be viewed directly on YouTube:
 Next year, Agile on the Beach will be on 4-5 September, and you can register your interest.

The ‘Just Do It’ Approach To Change Management

Last Friday I gave a talk at the Dare 2013 conference in Antwerp. The talk was about the experiences I and my colleague Ciarán ÓNeíll have had in a recent project, in which we found that sometimes a very directive, Just Do It approach will actually be the best way to get people in an agile mindset.

Update: The full video of this talk as given on ‘Agile on the Beach’ is available on youtube.

This was surprising to us, to say the least, and so we’ve tried to find some theory supporting our experiences. And though theory is not the focus of this story, it helps if we set the scene by referencing two bits of theory that we think fits our experience.

Just Do It

A long time ago, in a country far away, there was this psychologist called William James, who wrote:

“If you want a quality, act as if you already have it.” – William James (1842-1910)

We often say that if you want to change your behaviour, you need to change your mind, be disciplined, etc. But this principle tells us that it works the other way around as well: if you change your behaviour this can change your thinking. Or mindset, perhaps?

For more about the ‘As If’ Principle, see the book by Richard Wiseman

Another piece of theory that is related is complexity thinking as embodied by the Cynefin framework. Cynefin talks about taking different actions when managing situations that are in different domains: simple, complicated, complex or chaos.

Cynefin Framework

The project

And in chaos, our story begins.

This particular project was a development project for a large insurance company. The project had already been active for over half a year when we joined. It was a bad case of waterfall, with unclear requirements, lots of silo’s, lots of finger pointing and no progress.

The customer got tired of this, and got in a high-powered project manager who was given far reaching mandate to get the project going. (ie. no guarantees, just get *something* done) This guy decided that he’d heard good things about this ‘Agile’ thing, and that it might be appropriate here as a risk-management tool. Which was where we came in.

And this wasn’t the usual agile transition, with its mix of proponents and reluctants, where you coach and teach, but also have to sell the process to large extend.

Here, everyone was external (to the customer), no-one wanted Agile, or had much experience with it, but the customer was demanding it! And taking full responsibility for delivery, switching the project to a time-and-material basis for the external parties.

A whole new ballgame.

Initial actions

We started out by getting everyone involved local. Up to then, people from four different vendors been in different locations, in different countries even. Roughly 60 people in all, we all worked from the office in Amsterdam. Most of these people had never met or even spoken!

We started with implementing a fairly standard Scrum process.

Step one was requiring multi-functional teams, mixing the vendors. This was tolerated. Mostly, I think, because people thought they could ignore it. Then we explained the other requirements. One week sprints, small stories (<2 / 3 days), grooming, planning, demo, retro. These things were all, in turn, declared completely impossible and certainly in our circumstances unworkable. But the customer demanded it, so they tried. And at the end of the first week, we had our first (weak) demo.

So, we started with basic Scrum. The difference was in the way this was sold to the teams. Or wasn’t.

That is not to say that we didn’t explain the reasons behind the way of working, or had discussions about its merit. It’s just that in the end, there was no option of not doing it.

And… It worked!

The big surprise to us was how well this worked. People adjusted quickly, got to work, and started delivering working software almost immediately. Every new practice we introduced, starting with testing within the sprint, met with some resistance, and within 4 to 6 weeks was considered normal.

After a while we noticed that our retrospectives changed from simply complaining about the process to open discussion about impediments and valuable input for improvements generated by our teams.

And that’s what we do all this for, right? The continuous improvement mindset? Scrum, after all, is supposed to surface the real problems.

Well. It sure did.

Automated testing

One of those problems was one which you will be familiar with. If you’ve been delivering software weekly for a while, testing manually won’t keep up. And so we got more and more quality issues.

We had been expecting this, and we had our answer ready. And since we’d had great success so far in our top-down approach, we didn’t hesitate much, and we started asking for automated testing.

Adoption

Resistance here was very high. Much more so than for other changes. Impossible! But we’d heard all those arguments before, and why would this situation be any different? We set down the rules: every story is tested, tests are automated, all this happens within the sprint.

the-princess-bride-inconceivable

And sure enough, after a couple of sprints, we started seeing automated tests in the sprint, and a hit in velocity recovered to almost the level we had had before.

See. It’s Simple! Just F-ing Do It!

Limitations

Then after another 3-4 sprints, it all fell apart.

Tests were failing frequently, were only built against the UI, had lots of technical shortcomings. And tests were built within the team, but still in isolation: a ‘test automation’ person built them, and even those were decidedly unconvinced they were doing the right thing.

In the end, it took us another 6 months to dig our way out of this hole. This took much coaching, getting extra expertise in, pairing, teaching. Only then did we arrive at the stop-the-line mindset about our tests that we needed.

Even with all of that going on, though we were actually delivering working software.

And we were doing that, much quicker than expected. After the initial delays in the project, the customer hadn’t expected to start using the system until… well, about now, I think. But instead we had a (very) minimal, but viable product in time for calculating the 2012 year-end figures. And while we were at it, since we could roll-out new environments at a whim (well… almost:-) due to our efforts in the area of Continuous Delivery, we could also do a re-calculation of the 2011 figures.

These new calculations enabled the company to free a lot of money, so business wise there’s no doubt this was the right thing to do.

But it also meant that, suddenly, we were in production, and we weren’t really prepared to deliver support for that. Well, we really weren’t prepared!

Kanban

And that brings us to one of the most invasive changes we did during the project. After about 5 months, we moved away from Scrum and switched to Kanban.

Just Do It

At that time I was the scrum master of one of the teams, the one doing all the operations work. And our changes in priority were coming very fast, with many requests for support of production. In our retros, the team were stating that they were at the same time feeling that nothing was getting done (our velocity was 0), and they felt stressed (overtime was happening). Not a good combination. This went on for a few sprints, and then we declared Kanban.

That’s not the way one usually introduces Kanban. Which is carefully, evolutionary, keeping everyone involved, not changing the process but just visualising it. You guys know how that’s supposed to be done right?

This was more along the lines: “Hey, if you can’t keep priorities stable for a week, we can’t plan. So we won’t.”

Of course, we did a little more than that. We carefully looked at the type of issues we had, and the people available to work on them. We based some initial WIP limits on that, as well as a number of classes of service. And we put in some very basic explicit policies. No interruptions, except in case of expedite items. If we start something, we finish it. No breaking of WIP limits. And no days longer than 8 hours.

Adoption

That brought a lot of rest to the team. And immediately showed better production. It also made the work being done much more transparent for the PO.

It worked well enough, that another team that was also experiencing issues with the planning horizon also opted to ‘go Kanban’. Later the rest of the teams followed, including the PO team.

Limitations

That is not to say there was no resistance to this change. The Product Owners in particular felt uncomfortable with it for quite some time. The teams also raised issues. All that generated many of those nice incremental, evolutionary changes. And still does. The mindset of changing your process to improve things has really taken root.

The most remarkable thing, though, about all that initial resistance was the direction. It was all about moving back to the familiar safety of… Scrum!

Wrap-up

I’d like to tell you more but this post is getting long enough already. I don’t have time to talk about our adventures with going from many POs to one, introducing Specification by Example, moving to feature teams, or our kanban ready board.

I do feel I need to leave you with some comforting words, though. Because parts of this story go against the normal grain of Agile values.

Directive leadership, instead of Servant Leadership? Top-Down change, instead of bottom-up support? Certainly more of a dose of Theory X than I can normally stomach!

And to see all of that work, and work quite well, is a little disconcerting. Yes, Cynefin says that decisive action is appropriate in some domains, but not quite in the same way.

And overcoming the familiar ‘That won’t work in our situation’ resistance by making people try it is certainly satisfying, but we’ve also seen that fail quite disastrously where deep skills are required. That needs guidance: Still no silver bullets.

Enlightened Despotism is a perhaps dangerous tool. But what if it is the tool that instills the habits of Agile thinking? The tool that forcibly shakes people out of their old habits? That makes the despot obsolete?

Practice can lead to mindset. The trick is in where to guide closely, and when to let go.

Not Estimating At Scale

Estimation is a sensitive subject in Agile. Should we estimate? Do we avoid estimation in days or other time-based units? If we use relative estimation like story points, do we standardize across teams? What do we use estimation for? Are we explicit enough in emphasizing the distinction between estimations and commitments? How do we prevent abuse?

I’m not going to provide an answer to these questions.

If you want to get a good treatment on estimation with regards to agile, I suggest you read Ron Jeffries’ excellent articles in the Pragmatic Programmer’s magazine: Estimation is Evil: Overcoming the Estimation Obsession, and Estimation: The Best We Can Do

I’m just going to describe a fairly simple and effective way to not (well, kind of not) estimate with multiple teams. This is based on a recent experiment doing this exercise in a running project.

Not Estimating

In this project we had not been estimating at all, up until this point, which was 9 months into the project. And when I say we did not estimate, then of course I’m lying. But we did not do estimation of individual user stories (or larger items, Epics, Features, whatever you want to call them).

We started this project introducing Scrum, and going into one-week sprints. None of the participants were used to working in an iterative way. And the requirements for the project were completely unclear. So even if we wanted to do estimations, there was very little to estimate! We decided to forgo estimation, and simply asked the teams to split the user stories into small enough slices that a story could be finished in two to three days.

Of course, that means that there was estimation involved. Otherwise they couldn’t know whether the story was small enough. So I wasn’t lying about the lying. But the effort was limited, and only done for stories that were being groomed for the next sprint(s).

Delivery after nine months?

Fast forwarding nine months into this project, we were in a somewhat different state. For one, we were no longer doing Scrum, but have moved to a Kanban approach. We have a two-step Kanban configuration. Stories are prepared through a READY board, of which the process steps’ Explicit Policies reflect the Definition of Ready we had as a Scrum team. One of the policies is the ‘takes less than 2 – 3 days to get Done’ rule. One of three development teams is involved in the grooming process of an individual story and usually (but not necessarily) picks up that story later by moving it to their Build Board.

At nine months, people traditionally get interested in the concept of delivery. Of course, our project was already delivered. Production figures had been produced. But the project was supposed to ramp down around the 12 month mark. That meant that there was interest in finding out what part of the features still on the wishlist could be delivered by that time. So that required some estimations.

What to estimate

At this point, there are a bunch of high level areas of interest that we are interested in. These haven’t been looked at or communicated with the teams yet, and haven’t been prioritized. In fact, one of the main reasons we need the estimations is to help with prioritization. We do not want to spend a lot of time on estimating these things. We should focus on actually delivering software, not talking about things we might never work on.

We also don’t want to give the impression that the estimations that we will come up with are very accurate. A good estimation makes the uncertainty of the estimation explicit. We decide that we’ll be estimating the incoming feature requests in ‘T-Shirt sizes’, each categorized by a certain bandwidth of the number of stories expected to be necessary to implement the feature:

T-Shirt Sizes

  • Small: 1 – 5 Stories
  • Medium: 6-10 Stories
  • Large: 11-15 Stories
  • Parked: Not clear enough to size, or simply too large: requires splitting and clarifying

To make sure we wouldn’t spend a lot of time in detailed discussion, we decided to use Silent Grouping as the method of estimation. To make it work across our teams, we added a little diverge and merge to the mix.

Estimation process

We arranged for a number of sessions, each 90 minutes long. Each session would be dedicated to one main functional area. The Product Owner (or in this case the functional area PO helper) would introduce the functional area, explain the overall goals, and run through the features that had been identified in that area. This introduction would be half an hour, and would allow the teams to ask any questions that came up on the subject discussed.

Then we split into four different corners of our room, each attracting 8-10 people, and performed the silent grouping exercise there. This simply meant that everyone that showed up got a feature token (in this case simply the title of the feature printed on a piece of paper) and was invited to put it onto a board in one of four columns (the categories described above). Then people were allowed to change the position of any paper if they didn’t agree with its placement. And all of this happened in complete (well, we tried…) silence.

After a few minutes, things stopped moving, and we then went for the ‘merge’: on a central board, we called out each feature title, and based on the placement on the four separate boards determined the final position for the story. We did a few iterations of this, but our final set of rules seemed to work quite well:

  • Stories that have the same estimation on all boards, obviously go into that category on the main board
  • Stories that have two different, but adjacent, estimations go into the larger category
  • Stories that have three or more different estimations go into ‘parked’

We found that we regularly had some discussion on whether something was parked because of uncertainty, or size. But when we tried those out as separate categories, most turned out to be uncertain and the added value was limited.

Sizing Board Picture

Results

We had four 90 minutes sessions to estimate what turned out (of course:-) to be quite a bit more than 3 months of work. We found that quite a large part of the features ended up in ‘parked’, simply because they were not clear enough for the development teams to give even this kind of ball-park estimate. To get these features clearer, three amigo grooming sessions were set-up. This brought the number of parked features down considerably, and got us to a fair idea of the total amount of work. Since those sessions did not include the entire team, this did break our intent to have everyone involved in estimation, but we haven’t found a better way of doing this yet.

A second, and maybe even more important effect was that the whole team was instantly up to date on what we had planned for the next time period. A number of additional features and technical improvement were brought up during and after this session as people realized they might be needed, or would fit into the direction the project was going.

And the estimates gave the more distant business owners the required feedback to consider the priorities and decide what they could do without. For now.