The Blindfolded Ninja Model of Software Development

The ancient and respected team of Science Ninja is amazing. For centuries (or so it seems) they’ve protected the temple of Llabdum. The temple is old, with many places showing signs of previous attacks, or simply crumbling rock and weapons still in the skeletal hands of fallen enemies. Or maybe, you know, just lego bricks left lying about by younger Ninja-to-be.

People always marvel at the antics of the Ninja. As they practice moving between different parts of the temple, they put on blindfolds and go from one place on the defences to the other. Some places, they manage to walk in the exact rhythm needed to avoid the lego bricks on the floor, with running strides of the exact needed length, and that final jump at the end of the corridor. Some of the more experienced Ninja know how to cross the ancient garden, with its many treacherous walls, pits filled with blades, and snap-wires intended to trap unsuspecting intruders. They jump, flip, pirouette, land on their fingertips before doing a perfectly timed series of flip-flops through a line of moving stone wheels, ending up on the southern wall ready to defend the temple.

When a visiting warrior, who was sent with a team of eXPerienced soldiers, sees all this, he asks: isn’t that a little cumbersome? Wouldn’t you be able to move more quickly if you got rid of all that rubble? And why have traps inside your temple?

The Ninja scoffed at him. You obviously do not have our skills and experience, so you are not qualified to judge our defenses. The warrior, having just stepped into a particularly nasty piece of lego, winced, and suggested that he and his soldiers at least clear out the main corridor, so that they, in their untrained ways, could reach the two main walls quickly. And so it was done.

The soldiers picked up pieces of lego from all over the floor of the corridor. They found many, and some had to be very carefully extracted so as not to damage the ancient floor any further. They repaired the floor as well as possible, restoring the mosaic, with the picture becoming clearer every day. With the lego-bricks, they actually fortified a part of the southern wall that had been crumbling, but was now restored to a shining, if oddly multi-colored, unassailable wall.

At first, the Ninja still went through the corridor in their traditional pace, with irregular jumps and steps avoiding the now no longer existing hurdles. The next attack on the temple, though, the soldiers ran through the corridor at full speed, and were at the defenses before any of the Ninja. The shame was unbearable, and the Ninja got together to discuss changes to their customs. They talked with the visiting warrior, and had a look at the inner garden. They sectioned off one part (the pit with blades, I think it was), and started the work of making that part of their codebase accessible.

Estimates and Commitments – The Hard Truth

My esteemed colleague, Ciarán ÓNéill just posted a nice and considered discussion on estimation, velocity and cycle time. I, however, do not plan on being so considered, or considerate.

You see, too many people are bent under the crushing weight of living up to estimates. Even reckoning that they provided these estimates to begin with, the continuing focus on this fragile, incomplete, numerical slice of their work is having a seriously detrimental effect on our industry.

I can’t count how many times I’ve seen this playing out, in different companies. The poor product manager, even if he’s sometimes called Product Owner, has made his business case; calculated the expected ROI; probably even spent lots of time plotting the Rate of Return over time; thinking of ways to track the relevant figures closely so as to be able to adjust the approach over time; and shared these ideas with the broadest set of people within the company to get as much feedback as possible.

He’s calculated expected additional visitors to the website, impact on conversion and retention rates, and put in place the instruments to verify those numbers. He’s had his numbers checked by his peers, and carefully explained them in detail to the development team he’s expecting the work to be done by. He’s done everything he can to make sure his estimates are as accurate as he can make them.

And then he’s done the additional work to make sure he can not only track whether he’s on-track, but also proposed different scenarios on how to react if they’re not. He’s done all that for all the different features, and new products, that he deals with, and used the expected returns to prioritise work in the way that profits the company most.

Now he knows, our intrepid product manager, that he’s dealing with a complex issue! He’s been careful to include confidence margins, bandwidths of possible returns, and comments stressing uncertainties and risk factors. He’s done everything right.

So what happens after a few sprints, when the initial (bare minimum) features have seen the day of light and basked in the glory of unadulterated customer scorn? He’s still in the bandwidth of expected returns, but he can see that he’ll probably end-up on the lower end of the range he expected. At the iteration review he shows the current results to the development team (there’s a few managers in the audience as well), shows how things are progressing, where he sees ways to change the feature to get more customers interested, and how the priorities need to be adjusted to make this happen.

Some of the feature, sometimes a lot of it, needs to be cut. Including some functionality already released! After all, no-one is actually using that. And to make it a success, the team will probably have more work to do than was originally expected. In the end, though, they still have a very good chance of making this a success.

And that’s where the trouble hits home. The poor guy is lambasted for not being able to stick to his estimates. He came up with those estimates himself, he should make them work. After all, the numbers added-up, didn’t they? Everyone else is doing their part, and if he’d just put a little effort into it, he really should be able to stick to the plan, and ensure that customers start using the new feature as was planned. Perhaps adding more advertising for this feature is what is needed, and budget should be found for some additional effort on that part. Or maybe, since he’s the one not delivering, he should do that advertising work himself over the weekend.

It’s cruel. We, as an industry, should stop holding our product managers to standards that are impossible to reach. We should accept the inherent complexity of the problem, and help them tackle it by planning, tracking, and adjusting the plan as necessary. In fact, if we make the steps and adjustments small and fast enough, we could possibly relieve them from a large part of the yoke of budgeting, automating many of the tracking of adoption, and making them fully recognised, adult members of the organisation!

XP is Classic Rock

A while back I had a little fun comparing Agile to Rock’n’Roll. It’s still one of my favourite posts, and after my recent talk on the benefits of TDD, I got the idea that the best follow-up on that is something about the XP practices.

Test Driven Development with Bonnie Riatt

The first artist that came up was Bonnie Riatt. This is mostly because Ron Jeffries has mentioned her a few times on the Scrum Development mailing list, and since that picture above is from his site, I figure I owe it to him. Oh, and it’s pretty good music!

She sings ‘I Will Not Be Broken‘, which is as good a summary of Test First development as one could wish for. And if you take into account lines such as ‘But I know where I’m not going‘, and ‘Pull me round; Push me to the limit’, then it’s perfectly clear we’re going through that TDD process cycle of Red, Green, Refactor in as small a steps as possible. Isn’t it?

Pair Programming with Aerosmith / The Beatles

I already mentioned ‘Come Together‘ in the last post, and to be honest, I can’t think of a better Pair Programming song. It does bring with it some of the oft heard objections to pairing, with ‘Hold you in his arms till you can feel his disease‘ being a succinct summary. These things have to be overcome, but you’ll end up with a classic that is covered by practically everyone. I’m going for the Aerosmith version, as their guitar work shows the advantages of having two great practitioners working together…

A great runner up was ‘Let Me Share The Ride‘, by The Black Crowes. All about how sharing the ride can be done with someone who isn’t a burden…

Refactoring with Eric Clapton

So how about Refactoring? Well, refactoring is all about removing duplication. There are many songs about duplicitive women and men, talking about how they’ve been done wrong, but apart from having a completely different meaning, I’d also have to save those for a special post about management practices. A much more suitable song is the classic ‘Double Trouble‘ blues song, which you can see below in a marvellous version by Eric Clapton together with Steve Winwood. This song fits so well because it reminds the young programmer of the dangers that duplication in code brings. ‘I have no job, laid of and I’m having Double Trouble

Simple Design with The Ramones / The Doors

Simple Design is not simple to do. We all have a strong tendency to try to take into account all kind of possible future scenarios when writing code. So the advice that comes out of the The Doors song ‘Take it as it comes’ is very apt. I’ve selected a cover version by The Ramones here, but the central message is the same: “Take it easy baby, take it as it comes. Don’t move too fast if you want your love to last”. Of course, read ‘code’ for ‘love’  there, but that should be automatic for any kind of real Craftsman…

Collective Code Ownership with The Red Hot Chili Peppers

Moving on from there we go on to the circle that deals with wider team alignment. It would be easy to slip in the ‘Internationale‘ here, but that really doesn’t do this practice justice. Another thought was ‘You Don’t Own Me’ by Dusty Springfield, but it really didn’t fit into the classic rock theme, and is much more about not being allowed to access the… object under discussion.
The answer was, of course, found with the Red Hot Chili Peppers song ‘Give It Away‘! Not only do they  know that sharing the code makes everyone wiser: “Realize I don’t want to be a miser;
Confide with sly you’ll be the wiser”, but they know that this practice is crucial to working Agile:
Lucky me swimmin’ in my ability
Dancin’ down on life with agility

Continuous Integration with Bruce Springsteen

Of course, you can’t have collective ownership without a good Continuous Integration system. This one is easy, ’cause that code is ‘Born to Run’!

Customer Tests with Led Zeppelin

Working closely with your customer is the best way to ensure that you’re building the right thing. And having the customer closely involved with defining the acceptance test is the answer to avoiding the dreaded ‘Communication Breakdown’ that has left so many project is shambles:
Communication breakdown, it’s always the same
Havin’ a nervous breakdown, a-drive me insane

Sustainable Pace with Queen

People who know me know I can’t resist a good Queen song. This one emphasises precisely the opposite of what we want, but a negative test case can be very effective at communicating the desired functionality, can’t it? With ‘The Show Must Go On‘, we are confronted with all the dysfuction we find when teams push too hard to deliver impossible projects. Working in empty offices after everyone else has gone home, trying to find that last bug before it’s ready for production:
Empty spaces – what are we living for
Abandoned places – I guess we know the score
On and on, does anybody know what we are looking for…
The classical heroic programmer, working as an unsung (until now!) hero:
Another hero, another mindless crime
Behind the curtain, in the pantomime
Hold the line, does anybody want to take it anymore
 
And that’s it, for this post. I really wanted to get into the XP Metaphor practice as well, but it ended up with me getting headaches trying to understand the hidden meanings of songs like Stairway to Heaven, and Hotel California. Better not go there…

Agile is Rock ‘n’ Roll

[EDIT: Thanks to Hubert Iwaniuk, there’s now a playlist to accompany your reading of this post!]

[EDIT: There’s now also an XP version of this: XP Is Classic Rock]

I’ve had a number of occasions where people, usually working in a very strict, waterfall, environment, have voiced the opinion that ‘all that agile stuff’ is just an excuse to go ‘back to’ cowboy programming and rock ‘n’ roll development.

My normal response to this is something along the lines of ‘Au contraire! Working agile means you need to be more disciplined, not less!’ Which is true, of course, but not always quite in the same sense that they’re talking about.

Recently, though, it has occurred to me that in some ways, Agile really is Rock ‘n’ Roll! Let’s take a look at some examples:

Estimation and Planning

To quote the well respected experts on backlog prioritisation and de-scoping, Jagger and Richards: “You can’t always get what you want, but if you try, sometimes, you get what you need.”

And indeed this is the basis of Agile (release) planning: We take into account what the customer wants at the start of the project (and place that on our backlog), but make it clear that we do not know for sure yet what will be delivered eventually. We do go back to the customer frequently to determine if he’s happy with what he has so far, and whether he actually needs anything more (or something different). This gets the customer what he needs, or as much of it as possible within the time frame.

Team work

Whether you’re more partial to the original Beatles version, or the Aerosmith cover, the message to ‘Come Together’ is fairly core to our agile values. And from the rest of the lyrics it seems fairly possible that they are talking about old-style hackers and agile coaches:-)

Time Boxing

One tool we’ve certainly embraced in the Agile community is time boxing. And this really goes back to one of the originals of Rock ‘n’ Roll: Rock Around The Clock. As the song says: “We’ll have some fun when the clock strikes one”, and keep going, having a good time and “When it’s eight, nine, ten, eleven too, I’ll be goin’ strong and so will you.”. But we are strict in an almost Cinderella like fashion, and “When the clock strikes twelve, we’ll cool off then”, before we start another time-box.

Ch… Ch… Changes

I still don’t know what I was waiting for
And my time was running wild
A million dead-end streets
Every time I thought I’d got it made
It seemed the taste was not so sweet

If there was ever a song that was completely and obviously inspired by the need for Agile development, this is it. Mr. Bowie is even considerate enough to include the importance of testing: “how the others must see the faker, I’m much too fast to take that test”.

As you can see in the quoted lyrics above, we’re dealing with a man tired of waiting long periods of time for something which isn’t quite what he expected. Again, and again, this happens. A sad story, but all too familiar.

Transparency

We have to remember that the reasons for choosing an Agile way of working is not always one of love at first sight. The choice is often the last one after many disappointing previous liaisons. As Freddie Mercury sings in the last of our Agile Playlist, “I want to break free from your lies, you’re so self-satisfied I don’t need you”, right before he breaks the chains of his waterfall process.

But this same song also contains a warning: even if we deliver all we promise with new ways of working, the temptation to go back to the familiar ways of earlier days remains. And people still may ‘walk out the door’, because they ‘have to be sure’. Unfortunate, but unavoidable.

Take me to the other side

Not all songs are about happy things, though. It would of course be possible to note that Waterfall has some very nice tunes of its own. People would note the micro-management displayed in Every Breath You Take, and Walk This Way. They’ll point to the wishful planning complained about in Won’t Get Fooled Again. And Billy Joel’s Allentown, though written for a different industry, also deserves mention in this list. They might even mention Stairway to Heaven, which symbolically represents the Waterfall process’ stairway like structure, and lyrically describes the results (“When she gets there she knows, if the stores are all closed with a word she can get what she came for”, and “There’s a feeling I get when I look to the west, And my spirit is crying for leaving.”)

Are we sure heaven's this way?

Are we sure heaven’s this way?

For those people, it might be good to remember that there must be 50 ways to leave your waterfall project.

The problem is all inside your head
She said to me
The answer is easy if you
Take it logically
I'd like to help you in your struggle
To be free
There must be fifty ways
To leave your waterfall project

She said it's really not my habit
To intrude
Furthermore, I hope my meaning
Won't be lost or misconstrued
But I'll repeat myself
At the risk of being crude
There must be fifty ways
To leave your waterfall project
Fifty ways to leave your waterfall project

[CHORUS:]
You just interact, Jack
don't make a new plan, Stan
Continuously deploy, Roy
And you test regularly
Create a little trust, Gus
You do need to discuss, much!
Work transparently, Lee
And get yourself free