Scaling: Local vs Full Vertical scaling

It’s funny, isn’t it? Everybody is still talking about ‘scaling agile’. A whole industry has been created on the premise that large companies need process structures to help them manage pushing very large projects through huge sets of development teams.

Luckily, the DevOps movement (and the continuous delivery movement, I’m not sure they’re really separate) happened early into that process, so in most of these large scale processes there’s at the very least lip service to the idea that quality needs to be high, and delivery needs to be automated, even if they don’t aim for continuous.

Unfortunately, most of the work on the other side of the workflow was not included. Even though a number of initiatives have started in the last five years to bring product, marketing and UX people more into the fold to really include the customer into the process, this is still a very rare occurrence.

This isn’t all that surprising, I suppose. The longer-term planning and significant coordination overhead is familiar and comforting. And the large organisations that start using LeSS, SAFe, etc., will actually be better off than they were before. They’ll deliver more predictably, they’ll get into production quicker and they will feel more in control. Not bad, actually.

But even though their products will be in the hands of customers somewhat faster, the feedback loops from their customers will still be too slow. And the people that should be the link to the customers, the people in Marketing, Sales and Product Management, are way too far from the action. Too far to get fast feedback from the customer directly, but also too far from development to start using all the possibilities of technology to get more information on that customer and their preferences.

We scale our development organizations but ignore the capacity of its customers to make use of it

We scale our development organizations but ignore the capacity of its customers to make use of it

The solution is, predictably, to bring those roles into the fold and create a combined team where marketeers, product manager, software developers, ux specialist, testers and operations people work together to deliver on business goals. We have already seen the game changing effects that occur when software developers, testers and operations truly combine their knowledge and efforts. This next step will have more impact, simply because the knowledge that is being combined in the team is much broader, and more directly linked to the customer.

There are new issues to resolve when we do this. Coordination on a product level needs to be done in a different way. And coordination on the level of the different areas of expertise also needs to be done in a different way. Much more thought needs to be put into matters of vision and strategy, and how those have to be communicated to the teams that are formed. Into how this translates into rewarding people. But we’ve solved those kinds of issues before, and we can tackle them again.

Stop trying to scale agile. Scale your organisation.

The three failures of Continuous Delivery

Everyone seems to want to get on the Continuous Delivery train. Rightfully so, I think. For most, though, it’s not an easy ride. From my work with client and conversations with other coaches there’s a few common barriers to adoption.

In the end, the goal should be to be able to react faster to the market. And, to be honest, to finally be in actual control. But in business terms, it’s about cycle times. That’s what allows you to not just react quickly to market circumstances, but to actively probe markets and test product ideas.

So, as I mentioned, there’s a few common problems companies run into. First, just the basic technical steps to create a fully automated pipeline. Then, getting the tests sorted to a level that gives enough confidence to deploy to production whenever they’ve run. Only when those technical matters have been sorted do we get to the more interesting issues of allowing the business to make use of the possibilities offered by the newfound agility. Those have their own challenges.

Let’s have a look at the the ways these particular subject give teams trouble. In the hope that being forewarned some will be able to avoid them. I’ll go into more detail on how to avoid them in subsequent posts.

Get a pipeline

Now, if you’ve paid any attention to the literature, you know that at the core, CD is all about important things like process and a culture of quality. Which is all true, but that probably won’t help you very much. Most development organisations have spent years wrapping themselves in workarounds and buffers all painstakingly created to prevent detection of their real problems. So taking a relatively small, technical, step in setting up a delivery pipeline at least seems somewhat feasible and will by its nature start showing where some of the real problems lie.

A Delivery Pipeline

From what I’ve seen, just trying to set up that pipeline is trouble enough. That’s why I’ve put it as the first barrier to adoption of CD. It may seem easy, but there turn out to be many basic technical challenges. Most teams go through those same pains, and it’s not really surprising. There’s quite a bit of (often new) knowledge and skills involved. And teams usually have to deal with all kinds of legacy code and infrastructure, which doesn’t make it any easier.

Mostly, what companies find here is that they are missing is skills. And there are a lot of skills involved! A real DevOps approach should include operations knowledge in a team, but even then most of the skills needed to create a modern, fully automated infrastructure are something that takes most organisations a long time to develop.
It’s not that these things are beyond those teams, it’s just that they’ve not had to deal with them before. Sure, it is easy enough to package your application in a docker container and run it locally, but people are discovering it is quite a different thing to build it out further than that.

Testing

Testing is the achilles heel of many development teams. Most agile teams work hard to get and keep their code under test. Many fail. The advantage that Continuous Delivery has is that it sets explicit expectations on quality. There’s really no room to skimp on testing if every push you do should end up on production.
Like was the case for Continuous Integration, testing is what makes a Delivery Pipeline useful. It’s great if you have fully automated deployment, but if you have no way to determine if the code you’re building can be trusted, you’ll still not be in production any sooner.
There’s different ways teams fail with testing. Insufficient unit testing. Too limited protocol and service testing. A reliance on slow and brittle end-to-end testing. Skipping manual / exploratory testing, that may no longer be a gateway before going into production but is still very much necessary.

Business

Organisations that manage to get past the first two hurdles have at their disposal a tool that can bring them unimagined business advantages. But even having come this far, existing silo’s, processes and political positioning prevent organisations from profiting from their newly found technical capabilities.

Symptoms of this can be found in the ignoring, or even complete lack, of market data in deciding on new products and functionality. In continuing a practice of long term planning, without built in checks to see if the intended goals are being achieved. In basing priorities on political influence instead of business goals. And even in a reluctance to release new features to users even once they’re available behind a feature toggle in production.

These issues can be the most difficult to address and need to be picked up at the highest management levels. They are attacked with changes in goal setting, reward systems, and organisational structure.

Interlocking pieces

As with any process, these different elements cannot exist for long without the others to support them. Testing withers if it cannot be run quickly and frequently enough. A delivery pipeline has little value if you have no way to know if you can trust the code that it’s building. And a highly evolved technical team that is not clearly and directly involved with business goals and customers will easily find more fulfilling work elsewhere.

That’s why my advice is to start in this order, picking up the next challenge as soon as there’s clear progress on the previous. You start building technical skills and then use that base as a flywheel to get a change in the rest of the company going.

Agile On The Beach Talk

Ciarán and I had a wonderful time at the Agile on the Beach conference this last week. We did the first full version of our talk: “The ‘Just Do It’ approach to change management”.  I did an earlier version of the talk at the DARE conference in Antwerp earlier this year, but this longer version has gone through quite a few changes in the mean time.

agile-on-the-beach

The conference was set-up very well, and it was great to talk to so many people working on Agile in the UK.

The slides for the talk are up on slideshare:

We got some really nice responses, including:
The next chance to catch us is at the Lean and Kanban Netherlands conferene (“Modern Management Methods: Making Better Decisions”) conference in Maarssen on 7-8 October. We’ll have a new iteration of the talk, of course. Always on the move:-)LKNL-im-a-speaker-badge
UPDATE: The video of the talk was just released, and can be found on the conference website. Our talk can also be viewed directly on YouTube:
 Next year, Agile on the Beach will be on 4-5 September, and you can register your interest.

Scaling Agile with Set-Based Design

I wrote a while back about set-based design, and just recently about a way to frame scaling Agile as a mostly technical consideration. In this post I want to continue with those themes, combining them in a model for scaled agile for production and research.

Scale

In the previous post, we found that we can view scale as a function of the possibilities for functional decomposition, facilitated by a strong focus on communication through code (customer tests, developer tests, simple design, etc.)

This will result in a situation where we have different teams working on different feature-areas of a product. In many cases there will be multiple teams working within one feature area, which can again be facilitated through application of well known design principles, and shared code ownership.

None of this is very new, and can be put squarely in the corner of the Feature Team way of working. It’s distinguished mainly by a strong focus on communication at the technical level, and using all the tools we have available for that this can scale quite well.

set_based_design_image_1

Innovation

The whole thing starts getting interesting when we combine this sort of set-up with the ideas from set-based thinking to allow multiple teams to provide separate implementations of a given feature that we’d like to have. One could be working on a minimum viable version of the feature, ensuring we have a version that we can get in production as quickly as possible. Another team could be working on another version, that provides many more advantages but also has more risk due to unknown technologies, necessary outside contact, etc.

set_based_design_image_2

This parallel view on distributing risk and innovation has many advantages over a more serial approach. It allows for an optimal use of a large development organization, with high priority items not just picked up first, but with multiple paths being worked on simultaneously to limit risk and optimize value delivered.

Again, though, this is only possible if the technical design of the system allows it. To effectively work like this we need loosely coupled systems, and agreed upon APIs. We need feature toggles. We need easy, automated deployment to test the different options separately.

Pushing Innovation Down

But even with all this, we still have an obvious bottleneck in communication between the business and the development teams. We are also limiting the potential contributors to innovation by the top-down structure of product owner filling a product backlog.

Even most agile projects have a fairly linear look on features and priorities. Working from a story map is a good first step in getting away from that. But to really start reaping the benefits of your organisation’s capacity for innovation, one has to take a step back and let go of some control.

The way to do that is by making very clear what the goals for the organisation are, and for larger organisations what the goals for the product/project are. Make those goals measurable, and find a way to measure frequently. Then we can get to the situation below, where teams define their own features, work on them, and verify themselves whether those features indeed support the stated goals. (see also ‘Actionable Metrics at Organisational Scale‘, and ‘On Effect Mapping and Pirate Metrics‘)

set_based_design_image_3This requires, on top of all the technical supporting practices already mentioned, that the knowledge of the business and the contact with the user/customer is embedded within the team. For larger audiences, validation of the hypothesis (that this particular, minimum viable, feature indeed serves the stated goals), will need to be A/B tested. That requires a yet more advanced infrastructural setup.

All this ties together into the type of network organisations that we’ve discussed before. And this requires a lot of technical and business discipline. No one ever said it was going to be easy.

The ‘Just Do It’ Approach To Change Management

Last Friday I gave a talk at the Dare 2013 conference in Antwerp. The talk was about the experiences I and my colleague Ciarán ÓNeíll have had in a recent project, in which we found that sometimes a very directive, Just Do It approach will actually be the best way to get people in an agile mindset.

Update: The full video of this talk as given on ‘Agile on the Beach’ is available on youtube.

This was surprising to us, to say the least, and so we’ve tried to find some theory supporting our experiences. And though theory is not the focus of this story, it helps if we set the scene by referencing two bits of theory that we think fits our experience.

Just Do It

A long time ago, in a country far away, there was this psychologist called William James, who wrote:

“If you want a quality, act as if you already have it.” – William James (1842-1910)

We often say that if you want to change your behaviour, you need to change your mind, be disciplined, etc. But this principle tells us that it works the other way around as well: if you change your behaviour this can change your thinking. Or mindset, perhaps?

For more about the ‘As If’ Principle, see the book by Richard Wiseman

Another piece of theory that is related is complexity thinking as embodied by the Cynefin framework. Cynefin talks about taking different actions when managing situations that are in different domains: simple, complicated, complex or chaos.

Cynefin Framework

The project

And in chaos, our story begins.

This particular project was a development project for a large insurance company. The project had already been active for over half a year when we joined. It was a bad case of waterfall, with unclear requirements, lots of silo’s, lots of finger pointing and no progress.

The customer got tired of this, and got in a high-powered project manager who was given far reaching mandate to get the project going. (ie. no guarantees, just get *something* done) This guy decided that he’d heard good things about this ‘Agile’ thing, and that it might be appropriate here as a risk-management tool. Which was where we came in.

And this wasn’t the usual agile transition, with its mix of proponents and reluctants, where you coach and teach, but also have to sell the process to large extend.

Here, everyone was external (to the customer), no-one wanted Agile, or had much experience with it, but the customer was demanding it! And taking full responsibility for delivery, switching the project to a time-and-material basis for the external parties.

A whole new ballgame.

Initial actions

We started out by getting everyone involved local. Up to then, people from four different vendors been in different locations, in different countries even. Roughly 60 people in all, we all worked from the office in Amsterdam. Most of these people had never met or even spoken!

We started with implementing a fairly standard Scrum process.

Step one was requiring multi-functional teams, mixing the vendors. This was tolerated. Mostly, I think, because people thought they could ignore it. Then we explained the other requirements. One week sprints, small stories (<2 / 3 days), grooming, planning, demo, retro. These things were all, in turn, declared completely impossible and certainly in our circumstances unworkable. But the customer demanded it, so they tried. And at the end of the first week, we had our first (weak) demo.

So, we started with basic Scrum. The difference was in the way this was sold to the teams. Or wasn’t.

That is not to say that we didn’t explain the reasons behind the way of working, or had discussions about its merit. It’s just that in the end, there was no option of not doing it.

And… It worked!

The big surprise to us was how well this worked. People adjusted quickly, got to work, and started delivering working software almost immediately. Every new practice we introduced, starting with testing within the sprint, met with some resistance, and within 4 to 6 weeks was considered normal.

After a while we noticed that our retrospectives changed from simply complaining about the process to open discussion about impediments and valuable input for improvements generated by our teams.

And that’s what we do all this for, right? The continuous improvement mindset? Scrum, after all, is supposed to surface the real problems.

Well. It sure did.

Automated testing

One of those problems was one which you will be familiar with. If you’ve been delivering software weekly for a while, testing manually won’t keep up. And so we got more and more quality issues.

We had been expecting this, and we had our answer ready. And since we’d had great success so far in our top-down approach, we didn’t hesitate much, and we started asking for automated testing.

Adoption

Resistance here was very high. Much more so than for other changes. Impossible! But we’d heard all those arguments before, and why would this situation be any different? We set down the rules: every story is tested, tests are automated, all this happens within the sprint.

the-princess-bride-inconceivable

And sure enough, after a couple of sprints, we started seeing automated tests in the sprint, and a hit in velocity recovered to almost the level we had had before.

See. It’s Simple! Just F-ing Do It!

Limitations

Then after another 3-4 sprints, it all fell apart.

Tests were failing frequently, were only built against the UI, had lots of technical shortcomings. And tests were built within the team, but still in isolation: a ‘test automation’ person built them, and even those were decidedly unconvinced they were doing the right thing.

In the end, it took us another 6 months to dig our way out of this hole. This took much coaching, getting extra expertise in, pairing, teaching. Only then did we arrive at the stop-the-line mindset about our tests that we needed.

Even with all of that going on, though we were actually delivering working software.

And we were doing that, much quicker than expected. After the initial delays in the project, the customer hadn’t expected to start using the system until… well, about now, I think. But instead we had a (very) minimal, but viable product in time for calculating the 2012 year-end figures. And while we were at it, since we could roll-out new environments at a whim (well… almost:-) due to our efforts in the area of Continuous Delivery, we could also do a re-calculation of the 2011 figures.

These new calculations enabled the company to free a lot of money, so business wise there’s no doubt this was the right thing to do.

But it also meant that, suddenly, we were in production, and we weren’t really prepared to deliver support for that. Well, we really weren’t prepared!

Kanban

And that brings us to one of the most invasive changes we did during the project. After about 5 months, we moved away from Scrum and switched to Kanban.

Just Do It

At that time I was the scrum master of one of the teams, the one doing all the operations work. And our changes in priority were coming very fast, with many requests for support of production. In our retros, the team were stating that they were at the same time feeling that nothing was getting done (our velocity was 0), and they felt stressed (overtime was happening). Not a good combination. This went on for a few sprints, and then we declared Kanban.

That’s not the way one usually introduces Kanban. Which is carefully, evolutionary, keeping everyone involved, not changing the process but just visualising it. You guys know how that’s supposed to be done right?

This was more along the lines: “Hey, if you can’t keep priorities stable for a week, we can’t plan. So we won’t.”

Of course, we did a little more than that. We carefully looked at the type of issues we had, and the people available to work on them. We based some initial WIP limits on that, as well as a number of classes of service. And we put in some very basic explicit policies. No interruptions, except in case of expedite items. If we start something, we finish it. No breaking of WIP limits. And no days longer than 8 hours.

Adoption

That brought a lot of rest to the team. And immediately showed better production. It also made the work being done much more transparent for the PO.

It worked well enough, that another team that was also experiencing issues with the planning horizon also opted to ‘go Kanban’. Later the rest of the teams followed, including the PO team.

Limitations

That is not to say there was no resistance to this change. The Product Owners in particular felt uncomfortable with it for quite some time. The teams also raised issues. All that generated many of those nice incremental, evolutionary changes. And still does. The mindset of changing your process to improve things has really taken root.

The most remarkable thing, though, about all that initial resistance was the direction. It was all about moving back to the familiar safety of… Scrum!

Wrap-up

I’d like to tell you more but this post is getting long enough already. I don’t have time to talk about our adventures with going from many POs to one, introducing Specification by Example, moving to feature teams, or our kanban ready board.

I do feel I need to leave you with some comforting words, though. Because parts of this story go against the normal grain of Agile values.

Directive leadership, instead of Servant Leadership? Top-Down change, instead of bottom-up support? Certainly more of a dose of Theory X than I can normally stomach!

And to see all of that work, and work quite well, is a little disconcerting. Yes, Cynefin says that decisive action is appropriate in some domains, but not quite in the same way.

And overcoming the familiar ‘That won’t work in our situation’ resistance by making people try it is certainly satisfying, but we’ve also seen that fail quite disastrously where deep skills are required. That needs guidance: Still no silver bullets.

Enlightened Despotism is a perhaps dangerous tool. But what if it is the tool that instills the habits of Agile thinking? The tool that forcibly shakes people out of their old habits? That makes the despot obsolete?

Practice can lead to mindset. The trick is in where to guide closely, and when to let go.

Conway’s Organizational Structure Heuristic

organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations. — Melvin Conway

We often run into examples of Conway’s Law in organizations where silo-ed departments prompt architectural choices that are not supportive of good software design. The multi-functional nature of Agile teams is one way to prevent this from happening. But why not turn that around? If we know that organizational structure influences our software design, shouldn’t we let the rules of good software design inform our organizational structure?

Good software design leads to High Cohesion and Loose Coupling. What happens when we apply those principles to organizational design, and in particular to software teams? High Cohesion is about grouping together the parts that work on the same functionality, or need frequent communication. Multi-functional teams do just that, of course, so there we have an easy way of achieving effective organizational design.

Loose Coupling can be a little more involved to map. One way to look at that is that when communication between different teams is necessary, it should be along well-defined rules. That could be rules described in programming APIs when talking about development. Or rules as in well defined pull situations in a Lean process. Or simply the definition of specific tasks for which an organization has central staff available, such as pay-roll, HR, etc.

In general, though, the principles make it very simple: make sure all relevant decisions in day-to-day work can be made in the team context, with only in exceptional situations a need to find information or authorization in the rest of the organization.

On Discipline: Fooling yourself is an important skill!

Discipline is an interesting subject. One that I find myself regularly talking about. Or discussing about.derren brown mind control stunt

In the last year I lost about 20kg of body weight through a combination of diet change and exercise. This apparently give some people the impression that I am very disciplined. I’m not. I do know, however, how to make change easier to absorb. And how to inspect and adapt.

Fooling yourself is an important skill

The best ways to make sure you are able to keep discipline is to make being disciplined easy. And luckily, human beings are exceedingly good at fooling ourselves (click on the picture of Derren Brown, here, to watch a nice youtube clip demonstrating this).

For losing weight, this included:

  • Making sure that what I could eat was as least as nice to eat as what I had to stop eating (more meat, less potato chips)
  • Teaming up with my cousin to make not going to the gym not an option
  • Daily measurements of multiple metrics to get an understanding of progress, and variation
  • A very substantial reduction of insulin use, and improvement in blood sugar values with the corresponding increase in energy levels
  • Setting sensible targets that I adjusted as soon as I reached them.
  • Regular experiments to see what helped (drinking a lot of water), and what didn’t (too much exercise)

It’s just common sense. If you want change, make sure that what you’re changing to is enjoyable, and that your progress is clearly apparent. And a little peer pressure can also be useful.

Many of the practices promoted for agile processes fall in the same mould.

  • Short iterations give a clear sense of purpose for the short term, and a sense of accomplishment when completing them.
  • Visual progress, such as a scrum/task board give a direct view of progress, and the impression of things getting done (plus a little peer pressure…) when everyone on the team gets up and moves post-its a couple of times a day
  • Acceptance Test Driven development gives clear and achievable goals
  • Pair programming helps keeping focus throughout the day
  • Test Driven Development does the same, and again keeps progress visible
  • Retrospectives (and the resulting improvement experiments) give the sense of continuous improvement needed stimulate a positive feeling of support of the surrounding organisation

It might be slightly irreverent to throw all of these things together and talk about ‘fooling ourselves’. Is working in concordance with human needs ‘fooling yourself’? I don’t know, but I rather like the sound-byte:-)

On Effect Mapping and Pirate Metrics

During the Specification by Example training I talked about recently, Gojko Adzic introduced me to Effect Mapping. He’s writing a more extensive booklet on the subject, of which he’s released a beta here. I think this is an excellent tool for exploring goals, opportunities and possible features. It can be used as a tool to generate a backlog of features, as a way to explore possible business hyptheses, and perhaps even as a light-weight way to do strategic management of a company.

But let’s start with a short description (see Gojko’s site or beta booklet for the longer one) of what effect mapping is.

Effect Mapping basics

The basic structure of an effect map is that of a structured Mind-Map. A mind-map is a somewhat hierarchical way to note down ideas related to a central theme.

A Mind Map

A Mind Map

The effect map is a mind-map with a specific structure. The different levels of the mind-map are based around the answers to four questions:

  • Why? (The Goal)
  • Who? (Who can have a role in reaching that goal; Or preventing it)
  • How? (In what way can they help, business activities)
  • What? (What are the concrete software features to make; Or non-software actions to take)

Additional levels can be specific User Stories, tasks, or actions, but that depends more on how you want to organise your backlog. The important thing is that this provides an uninterrupted flow from high-level goal or vision to concrete work.

In this way, effect maps can provide one of the important missing steps in the Agile software development world: how to determine what features provide value supporting specific business goals.

The goal used in the centre of the effect map is supposed to be a measurable goal: we need to know unambiguously when this goal has been reached! Gojko gives a nice overview of the way this can be done, using a lighter-weight version of the way Tom Gilb prescribes for making goals measurable. This involves the scale (thing we’re measuring), meter (way we’ll measure it), benchmark (current state), constraint (minimum acceptable value, break even point), target (what we want it to be).

Effect mapping is not just about what ends up on the diagram, it’s also the process of generating the map. This is, of course, a collaborative approach. Getting the involved people together, and creating and discussing the goal, and the ways to get there. Important is that both business people with decision power as well as subject matter experts and technical people that can know possible solution directions should be present. And yes, they do have time for this. Using diverge and merge, as discussed in my previous post, can be very useful again. There’s more, but it’s not relevant to the rest of my post, about iterating, prioritising, etc. So just go and read the booklet, already.

Our Effect Map

In his booklet, Gojko also links this process to the Lean Startup process of customer development. I think this is a great combination, but I would like to see some tweaks in the type of measurements we use for goals in that case.

Actionable Metrics for Pirates

In his book The Lean Startup, Eric Ries talks extensively about the importance of Actionable Metrics, as opposed to Vanity MetricsAn actionable metric is one that ties specific and repeatable actions to observed results. A Vanity Metric is usually a more generic metric (such as total number of hits on a website, or total number of customers), that is not tied to specific (let alone repeatable) actions. There can be many reasons why those change, and isolating the reason is one part of making metrics actionable.

Dave McClure user the term ‘Pirate Metrics‘ to talk about the most important metrics he sees for organisations:Pirate Flag

A – Acquisition – User is directed to your site;

A – Activation – User signs up or is otherwise engaged;

R – Retention – User keeps coming back, i.e., is engaged over time;

R – Referral – User invites others;

R – Revenue – User pays or is otherwise monetized;

These are also the familiar ‘funnel’ metrics, and the above link to Ash Maurya’s site has much more background on them. When using these metrics, it is recommended to do Cohort testing, so that you can see the different results for different groups of users (again, see Ash Maurya’s site). Doing this such that the source of the (new) users is trackable allows you to identify the best ways of increasing the number of users, without deluding yourself into extrapolating from great growth figures if they’re the result of a single marketing action.

This is probably the only spot where Gojko’s booklet could use some tuning. The Gilbian metrics he uses for his example are functional metrics (SMART, and all that), but not the type of actionable metrics that would fit into the lean startup mold. In this example, we’re talking about reaching a certain number of player (one million), in a certain space of time (6 months), while keeping costs and retention rates to certain limits. Gojko does of course explain how to do this iteratively, taking a partial goal (less player than in the end-goal) and checking at a defined milestone whether the goal was reached. And because we’ve only done one ‘branch’ of the effect map, we do have a specific action to link to the result.

If we were to re-cast this more along the lines of our pirate metrics, we could rephrase the goal as increasing the Acquisition and Activation rates. To be clear: this is a whole other goal! This goal is about a change that will ensure structural growth in the longer term. The goal on 1M extra users could (perhaps) be reached by increasing marketing spending (note that only  operational costs are taken into account in the original example). In that case, the goal would be reached, but given that there is a typical Retention rate of (for example) 6 months, then there would be an equivocal exodus of users after that time. If the company had reacted based on total number of users, this could lead to incorrect actions such as hiring extra people, etc.

If the CEO comes down with the message ‘We want 1 million users!’, and it turns out he wants that in about 6 months time, we can then say, “Ok, that’s about 5500 new users per day, or
a growth rate of 1.6%” (per day, again). Then we can start creating and testing hypotheses in much smaller steps than those 6 or three months. What’s more, by using these metrics (and goals) it should become possible to use the same metrics further out into the effect map.

And if this is something at a larger scale, teams can take some of the higher level hypotheses / business activities and use their own domain knowledge to devise more experiments to find a way to reach those goals. Since churn is included in the figure, this also allows experiments based on increasing retention rates. And since we’re doing cohort testing on pirate metrics, we can know day-to-day and week-to-week whether what we’re doing has the expected results. This extends the set-based design paradigm used in effect mapping to a broader organisation.

Strategy and organisational structure

So by using effect mapping, we can make the relation between high-level goals, stakeholders and intermediate level goals visible, using a collaborative process involving different parts and levels of the organisation. By using a consistent set of (value / end-result focused) metrics that can be applied in both the short and longer term, throughout the organisation, we can enable all levels of the organisation to apply their knowledge and skills to reach those goals. And thus allows for more self-organisation (and experimentation) at all levels…

I recently came across the system of Hoshin Kanri (via Bob Marshall). This has some remarkable similarities to what I’ve been talking about. It’s also a method to ensure policy/goals can be distributed throughout the organisation, with involvement of multiple levels and stakeholders at each step. I’ve not studied it extensively, but to me it does feel like a strictly hierarchical system, and one that is mostly used in large companies with a fairly slow (yearly and half-yearly) cycle time. It is used by Toyota, and is part of Total Quality Control, which is supposed to be “designed to use the collective thinking power of all employees to make their organization the best in its field.” It’s nice to see that everything has already been thought of, and we’re just repeating the progress of the past:-)

The difference with effect mapping is its light-weight focus, and the ease with which effect mapping can more easily be used in less hierarchical organisations. In fact, I think such a system might well be a prerequisite to the type of organisations we’re talking about in light of the Stoos network: “learning networks of individuals creating value”. My recent proposal for a session at the Stoos Stampede was precisely about finding out how we can link an organisation’s vision/mission to goals specific enough that teams can work towards them independently, but open enough that they would not suffocate and entrap those teams. I think this might be one solution to that problem.

Stoos Stampede

Management Innovation, ca. 1972

SilosYesterday, after my brother’s 47th birthday, I was talking with my father. My father is 79, and he has had an interesting professional life. He started out as a catholic priest but, as you could guess from the fact of my existence, at some point figured out that this was not a sustainable career path for him.

While talking, my father touched upon the subject of work, and the importance of working for the right reasons. He discussed people that had found a work/life balance that worked for them, by not focusing on financial rewards, but on the contents of the work, and the need to spend time on their families. Of course he sees that many people, including himself at one point, have manoeuvred themselves in positions where making those choices has become very difficult, if not impossible. Certainly many people now are in positions where two full-time salaries are needed to be able to keep paying the mortgage. But focusing on intrinsic motivation for work is very important for your enjoyment of work, and life.

We also talked about management a bit. In 1972, also the year of my own vintage, my father became director of the social services department (‘Sociale Zaken’) of the city of Haarlem. This organisation of around 250 people was then organised in strict silos, where social workers, legal people, administrative work and other departments worked separately, and with little understanding of each other’s specific challenges. Clients (people who were waiting to hear whether they would be getting social support, usually) regularly had to wait until each of those departments had had their say, often adding up to a 4 month wait before their application was either approved or denied. Meanwhile social workers were reporting their conclusions to those client back in language that  was very hard to understand. All of this resulted in very unhappy clients. And thus for the people that had the direct contact with those client not always being treated in the kindest ways.

My father apparently discussed the use of understandable language extensively with the social workers, at one point demonstrating the problem of jargon by slipping back into some latin phrases

Quidquid recipitur ad modum recipientis recipitur – “Whatever is received is received according to the mode of the receiver.”

I have, having failed spectacularly at the one year of Latin I had in school, no idea if this was the actual phrase he used. But it does seem apt.

He also instigated a reorganisation of the service. The reorganisation moved from the siloed system to one where different geographical parts of the city were going to be services by multi-functional teams, consisting of social workers, legal experts, administrative workers and directly client-facing people. The goal was to reduce time wasted with hand-overs, breed understanding for people with another area of expertise, and create more understanding for the clients predicaments.

This move was not welcomed by the employees. So much so that unions were involved, strikes threatened, and silo walls reinforced. In the end, it happened anyway. The arguments were solid, and even with unions involved, there was a rather clear power structure in place. And eventually, when the dust had settled down, the results were very positive indeed. Client satisfaction was up and response times were down (I think it went from roughly 4 months to two weeks). But just as important, internal strife was removed to a large extend, employees were picking up skills (and work!) outside of their area of expertise, and employee satisfaction was considerably better.

Now for me, this is not surprising. It’s not surprising because I know and have lived the results of similar ways of working in Agile teams.

Or maybe I’ve been doing that because, in the end, some values are instilled at an early age, and I had a great example.

Turning it up to 11

Turning It Up To 11It’s odd how I’ve been unable to be very consistent in my subject-matter for this blog. I tend to hop around, going from very technical subject to very organisational ones. Some might see this as lacking focus. Maybe that’s true. I’ve never been able to separate execution from organisation and vision very well. To me they seem intrinsically linked. It’s comforting to me that even such luminaries as Kent Beck also seem to see things in this light.

If I look at my bifurcated (tri-? n-?) interests, I see a striking resemblance in the states of technological, managerial and commercial maturity in the world. In all of these areas, the state of affairs is abysmal. In all three areas, we seem to have recognised that this is the case. In all three, though, most people performing those roles are so used to the current state that only rarely do they see that a different approach could bring improvement. Could turn their work ‘up to 11’. There are some differences, though.

Technology

On the technology side, we’ve pretty much identified what works, and what doesn’t. Basically, XP got things right. Others before that also hit the right spot, but we know a mature team sticking to XP practices will not mess things up beyond salvation. If we compare that approach to what one finds in the run-of-the-mill waterfall situation, the differences are so great that there is truly no comparison. There are other questions still at least partially open, but most of those are concerned with scale, organisation, and finding out what should be built. And thus belong in the other categories. The main challenge is one of education. And, granted, a bit of proselytising.

Commercial

More commercial questions are less clear-cut, at least for me. In my work I’ve very rarely seen commercial, product development and marketing decisions taken with anything resembling a structured approach of any kind of rigour. A business case, if one is available at al is often only superficial, and almost never comes with any defined metrics and decision moments. The Lean Start-up movement is the only place I’ve seen that is trying to improve that. Taking this approach out of the start-up and into all the product development and marketing departments in the world is going to take a while, but it will happen. If only because companies capable of doing that will completely out-perform the ones that don’t.

I don’t think the case here is as clear cut as on the technology side, but we have a start. The principles of the Lean Start-up are based on the same ideas as Agile development: know what you want the result to be (validated learning) and iterate using short feedback loops. What to do, exactly, in those feedback loops is known for some types of learning, in some situations, but we’re still working on expanding our knowledge and skills in this area.

Management

As the solutions for the commercial and technical sides of things are rooted in experimentation and short feedback cycles, one might assume that the same would be true for the management side of things. And it’s true that those techniques have value in management in many situations. Many of the ideas on management are based on feedback cycles, Lean/Deming’s PDCA is one, for instance, but Cynefin‘s way of dealing with systems in the complex area is another. But we do seem to have many different ideas about how management should be done, how organisations should be structured and what gives people the best environment to work in.

One place where some of these ideas have gotten together is the Stoos Network. It’s interesting because of the different backgrounds of the people involved: Agile, Beyond Budgeting, Radical Management, Industry Leaders. Their initial gettogether this year resulted in a shared vision, with again an emphasis on learning.

“Organizations can become learning networks of individuals creating value, and the role of leaders should include the stewardship of the living rather than the management of the machine.” — Stoos Communique

This clearly expresses some of the shared values of the Stoos people, but still leaves quite a lot to the imagination. The people and ideas involved are interesting enough that I’ve volunteered to help organise one of the follow-up meetings,  the ‘Stoos Stampede’, which takes place in Amsterdam, 6 and 7 July.

Next to Stoos, as I said before, there are many ideas on how to change management. Lean has had an impact, but though the Toyota Way certainly does talk about people and how to support them in an organisation, this is not the prime focus of most Lean implementations. CALM has started talking about combining Complexity, Agile and Lean ideas, but so far has also not posted any results.  We’re still a bit lost at sea, here.

So what would we need from a new management philosophy?

  • We’d need to know how to structure an organisation. Stoos clearly think the current semi-hierarchical default is not workable for the future, or at the very least severely suboptimal. But what do ‘learning networks’ look like? And how do we grow them?
  • We’d need to know how to provide the organisation with a purpose. A Mission, a Vision, a Goal. Whatever you want to call it. Most organisations do have some sort of mission statement, but it is usually so far removed from the everyday practice of everyone working within the organisation that it might as well be absent.
  • We’d need to know how to connect that purpose to the rest of the organisation. How do we link the work of everyone in the organisation to its stated purpose? If the mission is specific this should be possible. But if we connect the work too tightly, it could be stifling.
  • We’d need to know how to connect the organisation with its customers, its suppliers, its partners. This would be different out of necessity, as the structure of the organisation itself is different. It would also be different out of philosophy, as those relations take on different meaning is the goals of the organisation outside of the monetary rise in importance.
  • We’d need to know how to align such organisations with the demands of the outside marketplace and governance. If the organisation is more oriented towards longer term viability and purposeful behaviour, this might have a good long term effect on profitability, but will certainly in the short term have a different financial behaviour. And budgeting and bookkeeping are areas that need very specific attention with an eye on the external rules these subjects need to comply with.

But apart from what new management would do to the idea of an organisation, there are also questions related more to the question of how to get there from here. Why would current managers want to change their organisations? Why would they want to change so drastically? There are plenty of reasons, but would they be convincing to the current CxO? What would they need to learn to be able to execute on such a vision? Will everyone enjoy working in these kinds of more empowering organisations, or will some people prefer something more hierarchical?

All of these things I want to know. Some of them we’ll discuss during the Stoos Stampede (propose a subject to discuss!), but personally I think we’re still at the very earliest stages of this particular change. In the mean time, we do have a few good examples, and some patterns that seem to work, and I’m going to try and get a few more organisations turned up to 11.