Scaling: Local vs Full Vertical scaling

It’s funny, isn’t it? Everybody is still talking about ‘scaling agile’. A whole industry has been created on the premise that large companies need process structures to help them manage pushing very large projects through huge sets of development teams.

Luckily, the DevOps movement (and the continuous delivery movement, I’m not sure they’re really separate) happened early into that process, so in most of these large scale processes there’s at the very least lip service to the idea that quality needs to be high, and delivery needs to be automated, even if they don’t aim for continuous.

Unfortunately, most of the work on the other side of the workflow was not included. Even though a number of initiatives have started in the last five years to bring product, marketing and UX people more into the fold to really include the customer into the process, this is still a very rare occurrence.

This isn’t all that surprising, I suppose. The longer-term planning and significant coordination overhead is familiar and comforting. And the large organisations that start using LeSS, SAFe, etc., will actually be better off than they were before. They’ll deliver more predictably, they’ll get into production quicker and they will feel more in control. Not bad, actually.

But even though their products will be in the hands of customers somewhat faster, the feedback loops from their customers will still be too slow. And the people that should be the link to the customers, the people in Marketing, Sales and Product Management, are way too far from the action. Too far to get fast feedback from the customer directly, but also too far from development to start using all the possibilities of technology to get more information on that customer and their preferences.

We scale our development organizations but ignore the capacity of its customers to make use of it

We scale our development organizations but ignore the capacity of its customers to make use of it

The solution is, predictably, to bring those roles into the fold and create a combined team where marketeers, product manager, software developers, ux specialist, testers and operations people work together to deliver on business goals. We have already seen the game changing effects that occur when software developers, testers and operations truly combine their knowledge and efforts. This next step will have more impact, simply because the knowledge that is being combined in the team is much broader, and more directly linked to the customer.

There are new issues to resolve when we do this. Coordination on a product level needs to be done in a different way. And coordination on the level of the different areas of expertise also needs to be done in a different way. Much more thought needs to be put into matters of vision and strategy, and how those have to be communicated to the teams that are formed. Into how this translates into rewarding people. But we’ve solved those kinds of issues before, and we can tackle them again.

Stop trying to scale agile. Scale your organisation.

Don’t Refactor. Rebuild. Kinda.

I recently had the chance to speak at the wonderful Lean Agile Scotland conference. The conference had a very wide range of subjects being discussed on an amazingly high level: complexity theory, lean thinking, agile methods, and even technical practices!

I followed a great presentation by Steve Smith on how the popularity of feature branching strategies make Continuous Integration difficult to impossible. I couldn’t have asked for a better lead in for my own talk.

Which is about giving up and starting over. Kinda.

Learning environments

Why? Because, when you really get down to it, refactoring an old piece of junk, sorry, legacy code, is bloody difficult!

Sure, if you give me a few experienced XP guys, or ‘software craftsmen’, and let us at it, we’ll get it done. But I don’t usually have that luxury. And most organisations don’t.

When you have a team that is new to the agile development practices, like TDD, refactoring, clean code, etc. then learning that stuff in the context of a big ball of mud is really hard.

You see, when people start to learn about something like TDD, they do some exercises, read a book, maybe even get a training. They’ll see this kind of code:

Example code from Kent Beck's book: "Test Drive Developmen: By Example"

Example code from Kent Beck’s book: “Test Drive Development: By Example”

Then they get back to work, and are on their own again, and they’re confronted with something like this:

Code Sample from my post "Code Cleaning: A refactoring example in 50 easy steps"

Code Sample from my post “Code Cleaning: A refactoring example in 50 easy steps”

And then, when they say that TDD doesn’t work, or that agile won’t work in their ‘real world’ situation we say they didn’t try hard enough. In these circumstances it is very hard to succeed. 

So how can we deal with situations like this? As I mentioned above, an influx of experienced developers that know how to get a legacy system under control is wonderful, but not very likely. Developers that haven’t done that sort of thing before really will need time to gain the necessary skills, and that needs to be done in a more controlled, or controllable, environment. Like a new codebase, started from scratch.

Easy now, I understand your reluctance! Throwing away everything you’ve built and starting over is pretty much the reverse of the advice we normally give.

Let me explain using an example.

Contine reading

Extending the Goal in Scrum

In his post “The Goal in Scrum“, Ron Jeffries makes the case for having a proper, higher-level-than-stories, Sprint Goal. As he says:

This is better, because it allows the wisdom and knowledge of the team to be fully exercised, and because it keeps focus on “what” is needed more than on just how it is to be done.

The point is well made, and true. Many Scrum teams would be much better off when adopting this practice. If you haven’t read the article yet, please do so now. It’s short and to the point, I’ll wait right here.

I think there are further steps beyond the point that Ron describes, that a good Agile organisation should aspire to. And that help get closer to the XP idea of an on-site customer.

For an example, let’s take the same team that Ron is talking about, working on some web-shop like domain. I’ll take a point in time a little further out than Ron did. They already learned his lesson, after all. And having done that they have a nice web shop running, with a working checkout flow, and even a wish-list.

The shop has a reasonable number of visitors, and sells enough to keep everyone employed. But though new functionality is built regularly, growth in terms of revenue is very uneven and not clearly linked to the efforts of the development team. This worries the CEO. He even considers whether changes in the team (bigger/smaller?) are necessary. The PO advises a more considered approach. He goes to the team and tells them about the issue:

“It seems our work sometimes helps us make money, but other times has no effect at all!”

The team has a nice, long, retro discussion about this. They remind the PO that they sometimes have raised questions on the practical use of some of the things they were building. He reminds them that those same things sometimes turned out to work well. And sometimes not. They realise they are missing an important feedback cycle.

Step one: Sprint Goal as a Business Test

The team is a very competent XP team, and knows that the best way to develop is to pull your assumptions forward. Test first. And change direction if the results tell you it’s not working. They agree with the PO to take a similar approach to the Sprint Goal: Describe the Goal as a test. Not a Unit Test. Not an Acceptance Test. Maybe a Business Test? One of the members talks about hypotheses but is voted down because the international team knows they’ll fail pronouncing that.

So for the next sprint, the PO and the rest of the team discuss what the Goal should be. The PO tells them that it seems many people put items in their shopping cart, and even go to checkout, but then stop and never go to payment. They agree that the goal should be to find out how to improve the conversion in that part of their sales funnel.

Step two: Information Radiator for Business Goal

The first story they agree on is to create a dashboard for the team to see this particular funnel. Easily done with their existing analytics software, but the team hasn’t been looking at that until now.

Step three: Generate ideas that could influence Business Goal

Overly simplified dashboard

Overly simplified dashboard

Then they think of all the reasons why they think someone would stop at that point. Could it be that the total amount frightens them? Should that be in the short view of the shopping cart on the main page? Is it the account creation that stops people going forward? Or the selection of the payment method? One of the team thinks the absence of PayPal as an option could be the problem. They decide they don’t know. And decide to find out.

Step four: Verify ideas

The other stories they create are small changes. And as part of those stories they encode decisions. Decisions that will result in more stories. Or will result in quickly deleting the just built functionality.

One example is the amount: they make a change in the shopping cart view on the main page that shows the approximate amount. The amount will be calculated client side, not taking into account tax and such which would require much more work. And they build it so that about 20% of their users get this new version while the rest get the old one. And compare the results. They agree up-front that only when this has more than 1% effect in the conversion they will build a more capable version of the feature in the next sprint.

The team member that likes PayPal gets a go too: let’s just put a ‘Pay with PayPal’ button on there, and see how often its pressed. Again shown to only a small subset of users. And again, only if it results into an increase of 1% or higher, they will build the PayPal integration.

Step five: Build feature

Based on the results of their experiments, which were very easy and quick to build, they create further stories for the backlog. Depending on how much time they had to spent, some of those stories could even be added to the current sprint. They’re proven to be supportive of the Goal. But if that doesn’t happen, it could also be fine to plan them in the next sprint, or later. At least the business value of those stories are very well defined.

Report

The PO is exited, but also worried a little. They’ll be building partial solutions. He is used to reporting on completed features to management. He works with the Scrum Master and one of the developers on the type of data they’ll be deciding on and creating a good report on them. Then he goes to discuss those reports with management. Management likes the figures, but would like to add a few forecasts on how these figures influence the revenue figures. That is quite easy to do, using the conversion figures along with average order size, and pretty soon they have a report that everyone is happy with.

The PO now reports on which parts of the sales funnel they’ve worked on, what ideas they testes, which worked and which didn’t, and how they are influencing revenue. Because they employ small experiments, they don’t spend much on the ideas that don’t work. And the report makes very clear that the increases in revenue that occur are significantly less than the stable costs of the team, even if the difference isn’t constant.

TL:DR

Defining your Sprint Goal in measurable business terms (such as Pirate Metrics for a web shop) gives more transparency and closer integration between development teams and their stakeholders.

From Here to Continuous Delivery

Situation Normal

There’s a clear pattern for software development. A pattern of lost opportunity.

In most, if not all, places where I’m called in the base question deals with the inability to deliver. Management sees that the plans they have are simply not going to be realised.

Business opportunities are lost waiting. Waiting for the next available spot in the product roadmap. Waiting for the development team to finish ‘stabilizing’ the system. Waiting for lengthy ‘refactoring’ phase to complete. Waiting for new servers to be delivered (in only six weeks!). Waiting for a PMO organisation to complete project initiation and relative priority. Waiting for development to complete coding. Waiting for a testing phase to complete. Waiting for management to analyse long lists of known issues and risks so they can decide whether a release is possible.

Anyway, there’s waiting involved.

As with any interesting problem, this one doesn’t have a single identifiable cause. The marketing department will blame the development team for being too slow. The development team will blame the marketing team for not knowing what they want. The development manager will blame his team for being slow and writing buggy code and all his stakeholders for not being realistic. Upper management will blame the marketing and development managers for being slow and not delivering.

They are, all of them, right.

And you can’t point a finger at one root cause. Yes, development messed up and wrote crappy, unmaintainable code. Yes, the business focused too much on short term gains and put too much pressure on development to deliver early. Yes, management should have focused on mission and strategy so the rest of the company could have managed scope. Yes, all were too hasty hiring new people when the pressure was on, and too much incompetence entered the company.

I’m willing to bet you’ve heard most of those complaints, made a few of them, and been the subject of others.

How do we get out of this vicious circle?

Step one: Fix execution

Stop. You’re trying to do too many things at once. The first thing that needs to be done is to get your technical house in order. As long as you can’t deliver a working, tested, system at the drop of a hat, you’ll always be too slow.

So now, immediately, start changing your technical practices to support better quality. Deploy automatically, Test automatically, test everything, and test in all manner of ways. Change your architecture to support quick change and better practices. And do all that while still delivering value.

That sounds difficult. It is. But it’s possible. I’ve done it. Others have.

You will slow down a little, initially. But you can use architectural changes, such as a Strangler Pattern or Branch by Abstraction, to quickly start over without throwing all your existing systems away.

And focus on quality. This is hard. Management needs to be extremely explicit in this. Technical teams are used to a focus on progress and speed, and will automatically revert to those and subvert the quality of your system in any case of perceived pressure.

Focus on quality

Add all the elements that give you more control over your systems. That means fully automated deployment, include the automated testing that you need to be confident enough to have every push of a developer going to production. Then make sure that actually happens.

Introduce feature toggles, so that your decision to supply a new feature to end-users becomes exactly that: a decision. And not related to your release cycle.

Measure everything.

Measure the results of new functionality on your business. Whether it’s in use of features for an internal system, or all your Pirate Metrics for your product website.

Step two: Fix alignment

Now that you’re able to deliver, it’s time to start making use of the opportunities that gives you.

You already know, now, how to measure the effects new features have on the use of your product. For some, that is already a direct link to the money being made by the product. For the funnel on a product website or web-shop, we can calculate the revenue increase (or decrease) from a change. Other types of applications need a little more work and imagination, but we can certainly get to some measure of value.

If you can’t measure the effects of your work, you can be sure you are not doing the right things.

But that is all still a bottom-up approach. Effective for short term goals, but potentially dangerous if the metrics we use aren’t aligned with longer term business goals.

im_example

If you have your mission and vision defined for you company, and there’s a strategy that you expect will bring you there, you should now spend some time to hammering out a small set of actionable metrics that we can use to prioritize our opportunities on a day-to-day basis. The post linked to above shows a way to determine that, based on Gojko Adzic’s excellent Impact Mapping.

You pick a very few metrics. In fact, you should aim for that One Metric That Matters. This is your compass, steering the whole of the company. Be careful that you don’t have other metrics hidden that undermine this, for instance in a target/bonus system.

dashboard-metrics

This OMTM should be permanently visible for everyone. It should be continuously, and automatically measured and updated. And it should be directly coupled to the various day-to-day activities of everyone in your company.

On the level of product development, all your priorities should be determined by the impact on your OMTM.

For most companies, this level of focus and clarity of purpose is far off. It requires clear vision and leadership. And will transform your organisation.

When are you getting started?

Scaling Agile with Set-Based Design

I wrote a while back about set-based design, and just recently about a way to frame scaling Agile as a mostly technical consideration. In this post I want to continue with those themes, combining them in a model for scaled agile for production and research.

Scale

In the previous post, we found that we can view scale as a function of the possibilities for functional decomposition, facilitated by a strong focus on communication through code (customer tests, developer tests, simple design, etc.)

This will result in a situation where we have different teams working on different feature-areas of a product. In many cases there will be multiple teams working within one feature area, which can again be facilitated through application of well known design principles, and shared code ownership.

None of this is very new, and can be put squarely in the corner of the Feature Team way of working. It’s distinguished mainly by a strong focus on communication at the technical level, and using all the tools we have available for that this can scale quite well.

set_based_design_image_1

Innovation

The whole thing starts getting interesting when we combine this sort of set-up with the ideas from set-based thinking to allow multiple teams to provide separate implementations of a given feature that we’d like to have. One could be working on a minimum viable version of the feature, ensuring we have a version that we can get in production as quickly as possible. Another team could be working on another version, that provides many more advantages but also has more risk due to unknown technologies, necessary outside contact, etc.

set_based_design_image_2

This parallel view on distributing risk and innovation has many advantages over a more serial approach. It allows for an optimal use of a large development organization, with high priority items not just picked up first, but with multiple paths being worked on simultaneously to limit risk and optimize value delivered.

Again, though, this is only possible if the technical design of the system allows it. To effectively work like this we need loosely coupled systems, and agreed upon APIs. We need feature toggles. We need easy, automated deployment to test the different options separately.

Pushing Innovation Down

But even with all this, we still have an obvious bottleneck in communication between the business and the development teams. We are also limiting the potential contributors to innovation by the top-down structure of product owner filling a product backlog.

Even most agile projects have a fairly linear look on features and priorities. Working from a story map is a good first step in getting away from that. But to really start reaping the benefits of your organisation’s capacity for innovation, one has to take a step back and let go of some control.

The way to do that is by making very clear what the goals for the organisation are, and for larger organisations what the goals for the product/project are. Make those goals measurable, and find a way to measure frequently. Then we can get to the situation below, where teams define their own features, work on them, and verify themselves whether those features indeed support the stated goals. (see also ‘Actionable Metrics at Organisational Scale‘, and ‘On Effect Mapping and Pirate Metrics‘)

set_based_design_image_3This requires, on top of all the technical supporting practices already mentioned, that the knowledge of the business and the contact with the user/customer is embedded within the team. For larger audiences, validation of the hypothesis (that this particular, minimum viable, feature indeed serves the stated goals), will need to be A/B tested. That requires a yet more advanced infrastructural setup.

All this ties together into the type of network organisations that we’ve discussed before. And this requires a lot of technical and business discipline. No one ever said it was going to be easy.

Actionable Metrics at Organizational Scale

I recently chaired a session on ‘Going from company vision to Actionable Metrics‘ at the Stoos Stampede conference in Amsterdam. In that session I tried to show some ideas on making the link from an overall company vision, through different approaches to achieve that vision, to concrete actionable metrics allowing teams within a company to autonomously pursue steps towards making that vision a reality. I’m not sure I succeeded in all of that in the session, so I’m trying again in this post…

Autonomy

A goal of a lean enterprise is to ensure that the people doing the work have all the information, knowledge and skills necessary to make decisions in their day-to-day work. For a lean knowledge organisation that means that people don’t just need to know their own work field well, they need to be able to relate the decisions they make every day to the longer-term goals and vision of the organisation.

Much has been said about supporting high levels of motivation and customer focus within companies. Especially in larger companies this is quite hard to sustain, which is not surprising with works such as Dan Pink’s Drive emphasising the importance of autonomy for the knowledge worker. Ensuring the right information, and a quick feedback loop for knowledge workers is key to motivated, high performing people.

Networks

Such autonomy can’t easily be achieved in a classically structured hierarchical organisation. The siloes inherent in that type of structure are natural barriers. Barriers to the autonomy of action where the distribution of necessary skills and knowledge over separate departments is an impediment to producing work and serving the customer. Barriers as well to the autonomy of reaction where the feedback loop on whether an action was in any way effective in reaching the goals of the organisation or not is too long, or absent.

An organizational structure much more compatible with that goal of autonomy, is that of a network organisation. The basic concept of a network organisation is that of independently working cross-functional teams that gather each other’s support where necessary but generally are able to make their own decisions. Enabling them to make their own decision is the subject of this post. These are the type of organizations that the Stoos Network is considering as the preferred replacement for today’s dysfunctions.

Actionable Metrics

The Lean Startup concept of Actionable Metrics (in order to create Validated Learning) is a great way to give a team the necessary autonomy to work independently towards the right goals. In a startup those metrics can be very directly linked with the goals of the company. In  larger organisations there is need for a clear link between the overall company vision and Actionable Metrics that are usable on the team level.

An actionable metric is one that ties specific and repeatable actions to observed results. — Ash Maurya, http://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/

In this post I’ll be using an Effect Map as the method to link the vision to specific metrics, but other methods exist of course. During the session, Catherine Louis mentioned GQM as a method designed determine which metrics to use. This paper gives some more background on GQM. The GQM method seems mostly concerned with determining the right metric for any given goal or problem, and can as such be very useful within the type of context I’m talking about. Another approach at determining the metrics you need is the A3 method.

The nice thing about Effect Maps is that they are very inclusive, and involve different roles and functions in their creation. This fits well with the multi-functional teams in our target organisation. They’re also easy to scale, using a diverge and merge facilitation process, so you can easily work on this with larger groups with full participation.

We’re on a mission from…

The first point of order is determining why we’re here. Not in a metaphysical way. I don’t really have the patience for that. In a ‘What are we trying to do as a company?’ way.  A company vision and mission statements should provide us with a good starting point here. A vision statement could be “A literate future, ” with a mission statement of “More readers, more books.”

This is of course very generic, and a subsequently generated Effect Map could go all over the place:

Effect Map Example

Effect Map Example

One thing we always need to add to the ‘Why?’ part of the Effect Map is a concrete, measurable, goal. This could be, in this case, encourage people to read more books, going from a current estimate of 100 books in a ‘lifetime’ (30 years, apparently, in the poll we got that figure from), to 1000.

Our company could encourage people to read more books in many different ways. The Effect Map shows various directions: working through publishers, changing business models, working with public libraries, promote reading in schools, making books cheaper, working with writers directly instead of through publishers, and some ways of helping people find the right books through technology.

Since we have a technology company, those last seem more relevant to start with. A larger company would probably start exploring some of the other possibilities as well, and perhaps be able to integrate those with the technology work. That could mean incorporating different sales models into the e-reader software. Or creating a 2nd hand e-book market in there. Or something. Plenty of opportunities!

Getting to actionable metrics

How do we go from such a generic goal (people read 10x more books in their lifetime!) to some actionable metrics that can be used by the multi-functional teams that our network organization comprises of? These teams need to be able to use those metrics in their day-to-day decision making. They need to be able to devise experiments, prioritise work, and navigate towards products and solutions without the type of top-down supervision that characterizes the more traditional organization.

First of all you need a baseline. Say we have a product through which people can read books: e-reader software (I told you we were a tech company). From that software we could gather statistics on the number of books people read. To do this well, we’d probably need to track this relative to how long customers have been using our software so we don’t get skewed figures from early enthousiasm (for instance). The term to look for is cohort testing. In our example, it turns out people are buying, on average, one book every three months. To get to the goal of 10x more books, we should then improve this to 3 books a month! This is already a shorter term, and thus more helpful, goal.

Pirates

To get to more useful figures, we need to turn to Dave McClure’s Pirate Metrics. Pirate Metrics are all about the funnel of attracting customer interest, keeping them, and selling to them: Acquisition, Activation, Retention, Referral and Revenue, or AARRR. Just by looking at customers through this lens gives a useful perspective. Our goal is phrased as getting people to read (on average) 10x more books. This could be approached as a matter of increasing retention (more books per customer), but also as on of Acquisition/Activation (getting more customers). That last one only works if we don’t take them away from other sources of books, of course. Can you think of a way to measure that? Certainly combined with increasing retention it would still give a new positive effect.

This would give us two main variables to pay attention to: Retention and Acquisition. We should, as a matter of course, be paying attention to at least the first three (AAR) of these metrics, and most companies will have a natural tendency to also track the last R… But tracking what the result of specific actions are on Retention and Acquisition should have our focus for now.

Pirate Metrics

Pirate Metrics

Splitting Metrics

But wait! In the Effect Map we had come up with two high-level feature ideas that would help us reach our goals: ‘Social Reading’ and ‘Better Book Recommendations’. Should both these ideas work with the same metrics?

Interesting question. On the one hand, I’d expect to be tracking all the pirate metrics in a well established application. But. The whole idea here is that you focus. So while we should keep a global eye on the whole (I’ll get back to that later), the experiments we’re conducting should focus on the change of a particular (set of) variables.

For our examples:

  • Social Reading – This is mostly about having existing readers getting each other interested in other books. That would be Retention. Secondary would be getting new customers in by sharing outside the app, which would be Referral. It’s important to note that distinction, as this has a direct influence on the priority of hypotheses to try.
  • Recommendations – This is also mostly about Retention. Existing readers should be getting more relevant recommendations, and thus but more books. The second level would be Activation. People who visit our shop already, but haven’t bought anything yet, should also be getting better recommendations and thus be prompted to buy.

This is consistent with the way we defined our goals, focusing on existing readers. That means it gives a decided focus to our development work. Phrasing our goals a little differently might increase our attention to new customer acquisition, but we’re not doing that. Consciously diving down into our metrics makes these kind of choices explicit, and that’s A Good Thing.

Absolutely Relative

So how would our teams take these more metrics towards specific hypotheses? First, we’d establish a baseline for retention. That could be

  • When people buy a book, the average time between this purchase and the previous one is 92 days

Then we can start measuring this over time. A nice, always visible chart on a big screen in the development teams’ rooms would be a great idea.

This is a useful metric, as we can measure it day-by-day. It can also be calculated in time based cohorts, as well as feature based cohorts, so we can compare normal changes over time with changes caused directly by our new features.

Hypothetically Speaking…

Ok, now we can get started. “Social Reading” is quite a broad concept. Our imaginative team of developers and product people can brainstorm-up quite a large cloud of ideas that fall within that scope, and they might have a collective gut-feeling on which ones of those would be most effective. They might have used another Effect Mapping exercise to generate ideas, and dot-voted on the most plausible ones. Or not.

The question they should be asking themselves is:

  • What would be the simplest way, costing the least effort, to prove that this idea can indeed prove effective in decreasing the average time between purchases?

If that’s not what they’re asking, they might as well be asking their company if it was feeling lucky, inc.

So for any ideas they generated, they should be thinking about this questions: how can it help  disprove (or prove) that the “Social Reading” idea is plausible?

From the long list (or effect map) of ideas that they generated (sharing quotes, sharing notes, rating books, publishing ‘reading lists’, embedding shared things on blogs, embedding on facebook or twitter, etc.) they pick one item. In this case that item might be a very basic “If a user can easily share he’s reading a book on twitter, this will trigger a shorter time between purchases”.

Now there are some problems with this one. Most important of all is that we don’t limit our audience, so we don’t know if people receiving the tweet will be existing customers. That’s Ok, though. It simply means we’re also testing for referral. Having an ‘internal’ audience might be more effective. But it would probably require a much larger up-front investment to create a communication channel between just our customers, an as such would be a less efficient way to test the hypothesis.

Another problem might be that we’re not helping the customer to share parts of the book, or anything, so the content of the tweet will probably be unspecific. We want more!

Stop!

Hold on. Take it easy. Hold your horses. We were looking for the simplest way to validate our hypothesis. How did we get into a discussion on all the cool features that should be in there? This feature, that feature, estimations (of both effort and expected value), discussions about opinions about hypotheticals…

If you want to know whether some tweets, to an audience that probably includes some existing customers, about a specific book, have some impact on sales then what you should do is write a few tweets. About some books. With an account that’s probably already there, from one of the people in the team. That probably already has other users of the service among its followers.

We all know that this is what should be done, that this is what the whole Lean Startup idea professes: Do The Simplest Thing. But even (or particularly?) in a bigger enterprise we need to put our money where our mouth is. And more importantly, avoid putting too much money where our mouth is and focus on getting that (in)validation of our most important hypothesis.

Giving the reins to the team

Taking those minimal steps is an important part of the overall process. It also seems to be one of the most difficult parts. Like developers needing time and practice to get used to working in the small steps of Test Driven Development. Like the Product Owners needing practice to split their requirements up into small enough chunks to be practical within a short sprint. Doing the absolute minimum work required to invalidate a hypothesis is probably the most difficult skill (or discipline?) to master from the Lean Startup mindset.

You can’t make it work without, though.

Especially in larger organisations where, by simple imperative of the size of the organisation, the involvement in individual projects, products and teams from the people setting the overall direction is much less than in a small startup!

The collaborative construction of Effect Maps ties together our organisation with a common vision and goal. Our carefully crafted and continuously tuned set of actionable metrics give teams clear direction within their level of influence to achieve.

To ensure that the organisational leadership doesn’t need to feel nervous about progress towards their goals, it is crucial that we fail as fast as is possible. And adjust. And try the next idea.

All Together Now

So organizational leadership can comfortably sleep at night in the knowledge that the full intellect and energy of their entire company is being put to work in the pursuit of truth, happiness and organizational goals while continuously self-correcting by the application of validated learning.

What more could they want?

There is one step still missing in this particular example, though. The metrics gathered for the specific experiments provide the very specific data needed for validated learning on the team level. The broader metrics that those are built on are still necessary for the bigger picture.

In our example that means that the targeted cohort testing done in each team is only one slice of the whole. The same (pirate) numbers are being gathered for a much broader cohort over longer periods of time to check whether the organisation as a whole is on the right track. Since that broader cohort would include the entire customer base, it will capture the combined results of all the teams.

Combining Cohorts

Summary

In this article I’ve tried to illustrate, using a simple example, how longer term organizational goals can be made measurable in the short term, and can be used to provide the direction and purpose for teams to work independently and with full autonomy towards a shared organizational purpose.

Can you capture your organization’s vision in goals? What end-result metrics will you introduce? Can you refrain from cost metrics and focus on new value delivery? Go on. Do it.

On Effect Mapping and Pirate Metrics

During the Specification by Example training I talked about recently, Gojko Adzic introduced me to Effect Mapping. He’s writing a more extensive booklet on the subject, of which he’s released a beta here. I think this is an excellent tool for exploring goals, opportunities and possible features. It can be used as a tool to generate a backlog of features, as a way to explore possible business hyptheses, and perhaps even as a light-weight way to do strategic management of a company.

But let’s start with a short description (see Gojko’s site or beta booklet for the longer one) of what effect mapping is.

Effect Mapping basics

The basic structure of an effect map is that of a structured Mind-Map. A mind-map is a somewhat hierarchical way to note down ideas related to a central theme.

A Mind Map

A Mind Map

The effect map is a mind-map with a specific structure. The different levels of the mind-map are based around the answers to four questions:

  • Why? (The Goal)
  • Who? (Who can have a role in reaching that goal; Or preventing it)
  • How? (In what way can they help, business activities)
  • What? (What are the concrete software features to make; Or non-software actions to take)

Additional levels can be specific User Stories, tasks, or actions, but that depends more on how you want to organise your backlog. The important thing is that this provides an uninterrupted flow from high-level goal or vision to concrete work.

In this way, effect maps can provide one of the important missing steps in the Agile software development world: how to determine what features provide value supporting specific business goals.

The goal used in the centre of the effect map is supposed to be a measurable goal: we need to know unambiguously when this goal has been reached! Gojko gives a nice overview of the way this can be done, using a lighter-weight version of the way Tom Gilb prescribes for making goals measurable. This involves the scale (thing we’re measuring), meter (way we’ll measure it), benchmark (current state), constraint (minimum acceptable value, break even point), target (what we want it to be).

Effect mapping is not just about what ends up on the diagram, it’s also the process of generating the map. This is, of course, a collaborative approach. Getting the involved people together, and creating and discussing the goal, and the ways to get there. Important is that both business people with decision power as well as subject matter experts and technical people that can know possible solution directions should be present. And yes, they do have time for this. Using diverge and merge, as discussed in my previous post, can be very useful again. There’s more, but it’s not relevant to the rest of my post, about iterating, prioritising, etc. So just go and read the booklet, already.

Our Effect Map

In his booklet, Gojko also links this process to the Lean Startup process of customer development. I think this is a great combination, but I would like to see some tweaks in the type of measurements we use for goals in that case.

Actionable Metrics for Pirates

In his book The Lean Startup, Eric Ries talks extensively about the importance of Actionable Metrics, as opposed to Vanity MetricsAn actionable metric is one that ties specific and repeatable actions to observed results. A Vanity Metric is usually a more generic metric (such as total number of hits on a website, or total number of customers), that is not tied to specific (let alone repeatable) actions. There can be many reasons why those change, and isolating the reason is one part of making metrics actionable.

Dave McClure user the term ‘Pirate Metrics‘ to talk about the most important metrics he sees for organisations:Pirate Flag

A – Acquisition – User is directed to your site;

A – Activation – User signs up or is otherwise engaged;

R – Retention – User keeps coming back, i.e., is engaged over time;

R – Referral – User invites others;

R – Revenue – User pays or is otherwise monetized;

These are also the familiar ‘funnel’ metrics, and the above link to Ash Maurya’s site has much more background on them. When using these metrics, it is recommended to do Cohort testing, so that you can see the different results for different groups of users (again, see Ash Maurya’s site). Doing this such that the source of the (new) users is trackable allows you to identify the best ways of increasing the number of users, without deluding yourself into extrapolating from great growth figures if they’re the result of a single marketing action.

This is probably the only spot where Gojko’s booklet could use some tuning. The Gilbian metrics he uses for his example are functional metrics (SMART, and all that), but not the type of actionable metrics that would fit into the lean startup mold. In this example, we’re talking about reaching a certain number of player (one million), in a certain space of time (6 months), while keeping costs and retention rates to certain limits. Gojko does of course explain how to do this iteratively, taking a partial goal (less player than in the end-goal) and checking at a defined milestone whether the goal was reached. And because we’ve only done one ‘branch’ of the effect map, we do have a specific action to link to the result.

If we were to re-cast this more along the lines of our pirate metrics, we could rephrase the goal as increasing the Acquisition and Activation rates. To be clear: this is a whole other goal! This goal is about a change that will ensure structural growth in the longer term. The goal on 1M extra users could (perhaps) be reached by increasing marketing spending (note that only  operational costs are taken into account in the original example). In that case, the goal would be reached, but given that there is a typical Retention rate of (for example) 6 months, then there would be an equivocal exodus of users after that time. If the company had reacted based on total number of users, this could lead to incorrect actions such as hiring extra people, etc.

If the CEO comes down with the message ‘We want 1 million users!’, and it turns out he wants that in about 6 months time, we can then say, “Ok, that’s about 5500 new users per day, or
a growth rate of 1.6%” (per day, again). Then we can start creating and testing hypotheses in much smaller steps than those 6 or three months. What’s more, by using these metrics (and goals) it should become possible to use the same metrics further out into the effect map.

And if this is something at a larger scale, teams can take some of the higher level hypotheses / business activities and use their own domain knowledge to devise more experiments to find a way to reach those goals. Since churn is included in the figure, this also allows experiments based on increasing retention rates. And since we’re doing cohort testing on pirate metrics, we can know day-to-day and week-to-week whether what we’re doing has the expected results. This extends the set-based design paradigm used in effect mapping to a broader organisation.

Strategy and organisational structure

So by using effect mapping, we can make the relation between high-level goals, stakeholders and intermediate level goals visible, using a collaborative process involving different parts and levels of the organisation. By using a consistent set of (value / end-result focused) metrics that can be applied in both the short and longer term, throughout the organisation, we can enable all levels of the organisation to apply their knowledge and skills to reach those goals. And thus allows for more self-organisation (and experimentation) at all levels…

I recently came across the system of Hoshin Kanri (via Bob Marshall). This has some remarkable similarities to what I’ve been talking about. It’s also a method to ensure policy/goals can be distributed throughout the organisation, with involvement of multiple levels and stakeholders at each step. I’ve not studied it extensively, but to me it does feel like a strictly hierarchical system, and one that is mostly used in large companies with a fairly slow (yearly and half-yearly) cycle time. It is used by Toyota, and is part of Total Quality Control, which is supposed to be “designed to use the collective thinking power of all employees to make their organization the best in its field.” It’s nice to see that everything has already been thought of, and we’re just repeating the progress of the past:-)

The difference with effect mapping is its light-weight focus, and the ease with which effect mapping can more easily be used in less hierarchical organisations. In fact, I think such a system might well be a prerequisite to the type of organisations we’re talking about in light of the Stoos network: “learning networks of individuals creating value”. My recent proposal for a session at the Stoos Stampede was precisely about finding out how we can link an organisation’s vision/mission to goals specific enough that teams can work towards them independently, but open enough that they would not suffocate and entrap those teams. I think this might be one solution to that problem.

Stoos Stampede

The Strategic Inflection Point as a Special Case Pivot

I’ve noticed that I very regularly get people visiting my blog through a Google search for the term ‘Strategic Inflection Point’. Since that term has some very direct connections to other concepts I’ve been learning about, I thought I’d give some detail on Strategic Inflection Points, and their relation to the Lean Startup ideas of Pivots and Pirate Metrics.

I once reported on a presentation by Mary Poppendieck at the Lean and Kanban conference in Antwerp of 2010, where she mentions Andy Grove‘s book ‘Only the paranoid survive‘. This 1999 book deals with the way Grove ran Intel, and includes the concept of the Strategic Inflection Point, also described by Grove in his 1998 speech at the Academy of Management.

Strategic Inflection Point, copied from Mary's sheets

Grove describes the Inflection Point:

A Strategic Inflection Point is that which causes you to make a fundamental change in business strategy.

For anyone who’s been keeping up with the discussion on the Lean Start-up will see the similarity with Eric Ries’ concept of the Pivot:

“A structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth.” – Eric Ries, The Lean Startup

The language is a bit different, of course, but there are obvious places of overlap. Grove’s Inflection Point has a main emphasis on external factors changing, necessitating change from the side of the company.

A major change due to introduction of new technologies. A major change due to the introduction of a different regulatory environment. The major change can be simply a change in the customers’ values, a change in what customers prefer.

A Pivot can be seen as a reaction to encountering a Strategic Inflection Point. Pivots also happen in search of the correct strategy, product of market fit, which makes them more active and Ries’ structured approach to identifying the need for a pivot seems to be exactly what Grove is looking for.

The biggest difficulty with Strategic Inflection Points is telling one from the many changes that impinge on you in the business. How do you know if a change is just a garden variety change or qualified to be this monumental, catastrophic change category that we call an Inflection Point? I could never come up with a particularly satisfactory answer to that question. And I’m quite sure that if I could, I wouldn’t be talking about them. A set of principles along those lines would be extremely helpful as a competitive weapon because we deal with changes every day.

Ries’ innovation accounting provides the tools necessary to identify that Inflection Point. Pirate Metrics, as coined by Dave McClure,  can provide an early warning that the strategy for a product is not, or no longer, working. And there are some different variants possible for different market situations.

Innovation accounting is the structured approach of doing cohort testing to determine the viability of a product’s engine of growth. This is not specific to a start-up. You can track Acquisition, Activation, Retention, Referral and Revenue for any product, whether in a small start-up trying to find the right product and sales model or an existing company that needs to track the health of existing products and models.

So if you’re worried about running into a Strategic Inflection Point for your business, this is what you do: start keeping (actionable) metrics on how your products are doing, what the growth rates are for their engines of growth, make those numbers clear and visible for everyone in your company, from the C-suite to the janitors, and if you do see a need to pivot, do so with clearly defined hypotheses, and check the results.

 

 

 

 

Turning it up to 11

Turning It Up To 11It’s odd how I’ve been unable to be very consistent in my subject-matter for this blog. I tend to hop around, going from very technical subject to very organisational ones. Some might see this as lacking focus. Maybe that’s true. I’ve never been able to separate execution from organisation and vision very well. To me they seem intrinsically linked. It’s comforting to me that even such luminaries as Kent Beck also seem to see things in this light.

If I look at my bifurcated (tri-? n-?) interests, I see a striking resemblance in the states of technological, managerial and commercial maturity in the world. In all of these areas, the state of affairs is abysmal. In all three areas, we seem to have recognised that this is the case. In all three, though, most people performing those roles are so used to the current state that only rarely do they see that a different approach could bring improvement. Could turn their work ‘up to 11’. There are some differences, though.

Technology

On the technology side, we’ve pretty much identified what works, and what doesn’t. Basically, XP got things right. Others before that also hit the right spot, but we know a mature team sticking to XP practices will not mess things up beyond salvation. If we compare that approach to what one finds in the run-of-the-mill waterfall situation, the differences are so great that there is truly no comparison. There are other questions still at least partially open, but most of those are concerned with scale, organisation, and finding out what should be built. And thus belong in the other categories. The main challenge is one of education. And, granted, a bit of proselytising.

Commercial

More commercial questions are less clear-cut, at least for me. In my work I’ve very rarely seen commercial, product development and marketing decisions taken with anything resembling a structured approach of any kind of rigour. A business case, if one is available at al is often only superficial, and almost never comes with any defined metrics and decision moments. The Lean Start-up movement is the only place I’ve seen that is trying to improve that. Taking this approach out of the start-up and into all the product development and marketing departments in the world is going to take a while, but it will happen. If only because companies capable of doing that will completely out-perform the ones that don’t.

I don’t think the case here is as clear cut as on the technology side, but we have a start. The principles of the Lean Start-up are based on the same ideas as Agile development: know what you want the result to be (validated learning) and iterate using short feedback loops. What to do, exactly, in those feedback loops is known for some types of learning, in some situations, but we’re still working on expanding our knowledge and skills in this area.

Management

As the solutions for the commercial and technical sides of things are rooted in experimentation and short feedback cycles, one might assume that the same would be true for the management side of things. And it’s true that those techniques have value in management in many situations. Many of the ideas on management are based on feedback cycles, Lean/Deming’s PDCA is one, for instance, but Cynefin‘s way of dealing with systems in the complex area is another. But we do seem to have many different ideas about how management should be done, how organisations should be structured and what gives people the best environment to work in.

One place where some of these ideas have gotten together is the Stoos Network. It’s interesting because of the different backgrounds of the people involved: Agile, Beyond Budgeting, Radical Management, Industry Leaders. Their initial gettogether this year resulted in a shared vision, with again an emphasis on learning.

“Organizations can become learning networks of individuals creating value, and the role of leaders should include the stewardship of the living rather than the management of the machine.” — Stoos Communique

This clearly expresses some of the shared values of the Stoos people, but still leaves quite a lot to the imagination. The people and ideas involved are interesting enough that I’ve volunteered to help organise one of the follow-up meetings,  the ‘Stoos Stampede’, which takes place in Amsterdam, 6 and 7 July.

Next to Stoos, as I said before, there are many ideas on how to change management. Lean has had an impact, but though the Toyota Way certainly does talk about people and how to support them in an organisation, this is not the prime focus of most Lean implementations. CALM has started talking about combining Complexity, Agile and Lean ideas, but so far has also not posted any results.  We’re still a bit lost at sea, here.

So what would we need from a new management philosophy?

  • We’d need to know how to structure an organisation. Stoos clearly think the current semi-hierarchical default is not workable for the future, or at the very least severely suboptimal. But what do ‘learning networks’ look like? And how do we grow them?
  • We’d need to know how to provide the organisation with a purpose. A Mission, a Vision, a Goal. Whatever you want to call it. Most organisations do have some sort of mission statement, but it is usually so far removed from the everyday practice of everyone working within the organisation that it might as well be absent.
  • We’d need to know how to connect that purpose to the rest of the organisation. How do we link the work of everyone in the organisation to its stated purpose? If the mission is specific this should be possible. But if we connect the work too tightly, it could be stifling.
  • We’d need to know how to connect the organisation with its customers, its suppliers, its partners. This would be different out of necessity, as the structure of the organisation itself is different. It would also be different out of philosophy, as those relations take on different meaning is the goals of the organisation outside of the monetary rise in importance.
  • We’d need to know how to align such organisations with the demands of the outside marketplace and governance. If the organisation is more oriented towards longer term viability and purposeful behaviour, this might have a good long term effect on profitability, but will certainly in the short term have a different financial behaviour. And budgeting and bookkeeping are areas that need very specific attention with an eye on the external rules these subjects need to comply with.

But apart from what new management would do to the idea of an organisation, there are also questions related more to the question of how to get there from here. Why would current managers want to change their organisations? Why would they want to change so drastically? There are plenty of reasons, but would they be convincing to the current CxO? What would they need to learn to be able to execute on such a vision? Will everyone enjoy working in these kinds of more empowering organisations, or will some people prefer something more hierarchical?

All of these things I want to know. Some of them we’ll discuss during the Stoos Stampede (propose a subject to discuss!), but personally I think we’re still at the very earliest stages of this particular change. In the mean time, we do have a few good examples, and some patterns that seem to work, and I’m going to try and get a few more organisations turned up to 11.