Structured Agile for Product Teams

I teach a structured approach to Agile, developed by the Agile Academy, and adapted for larger organisations who assemble project teams to tackle all manner of projects.

In smaller organizations, development tends to be product- rather than project-oriented.  Instead of routinely assembling (and later dismantling) project teams for major projects (who then handover to standing business-as-usual teams for support and minor enhancements), there are invariably long-lived product-centered teams who perform a mix of new development, minor enhancements, plus ongoing maintenance and second-level support.

This product-team arrangement may be the wave of the future for larger organizations as well: the case for product-teams is eloquently (and vehemently) made by Evan Bottcher in his most excellent diatribe Projects are Evil and Must be Destroyed - read it!

Now product teams -- in my experience -- face a number of typical challenges, especially balancing different kinds of work, especially:

• major development initiatives
• maintenance (bug fixing) and other support needs
• minor enhancements

In the remainder of this post I'll describe how the four-phase, structured Agile approach outlined by the Agile Academy here (similar to AUP) can be adapted to meet the needs of product teams.  At a high level:

1. Concept phase
Project version: Assess the technical feasibility of a business idea.  Draft a "project charter".
Product version: Revise and update the vision  for the product; assess requests from stakeholders (or customers) for feasibility; choose the theme(s) for the next major release.  Update the "product charter".

2. Initiate phase
Project version: Review and refine the business idea for technical development.  High-level prioritization, estimation, and initial release planning for the project.
Product version: High-level prioritization, estimation, and planning for the next major release.

3. Deliver phase
Project version: Iterative and incremental development and delivery of working software.
Product version: Same, interspersed with maintenance and support.

4. Deploy phase
Project version: Rapid and secure deployment of software to required environments.  Handover to BAU team on final deploy.
Product version: Same, but no final and no handover.

Concept and Initiate still make sense, but happen periodically
The major changes involve re-purposing the Concept and Initiate phases to encourage long(ish)-term planning for product teams, centered around the next major release of the software.  This is an opportunity for the business and technical teams to put their heads together, assess and chart a way forward.

Distinguish enhancement requests from defect reports

With standing teams it is all too easy to go reactive, and treat feature requests similarly to defect reports: part of a long list of "to dos" that must be prioritized and got to.  This feels more efficient, but it may fail to reject suggestions and to sufficiently refine and combine the best ones.

The difference from the project situation -- where the Concept phase allows the team to advise the governance body to kill off projects early and cheaply -- is that in the product case the team must advice not whether but what to go ahead with: Are we funding a cohesive and worthwhile upgrade?  This requires discernment and perspective, and going back to stakeholders with counter-offers to their suggestions that enhance the design, experience and overall integrity of the software.

Set-up iterations?
A set-up iteration, usually branded "Iteration 0" takes place at the start of the Deliver phase.  In the case of Product teams, many of the traditional tasks for a green-fields project will not be necessary.  Nevertheless, periodic Iteration 0s offer the opportunity to improve on existing processes, as well as implement changes foreshadowed by the most recent Initiate phase.

Transitioning to Agile
In transitioning an existing product team to Agile, the first Concept and Initiate phase can be usefully devoted to getting the existing processes, team, and backlog of work "over the hump" and across into an initial Agile configuration.

Often challenges will include a lack of automated unit-testing, existing patterns and dysfunctions in engagement with customers, and the difficulty of unlearning old patterns of behaviour.  Early priorities will include getting the team trained up, working cross-functionally, establishing a stable iteration cycle, including daily stand-ups, iteration planning, showcases, and retrospectives, as well as breaking work into small chunks, estimating it and delivering in small increments.

Doing just enough to get the current backlog of work going without planning a new major release (just yet) is effectively a Hudson Bay start to an Agile future.  But remember: it's only a start!

Mixing bug-fixing and enhancement work
There's no one answer to this one.  Bug-fixes are typically difficult to estimate and often urgent, and this combination can play havoc with everything else.  The aim is minimize task-switching costs while maintaining standards of service.  The thing that will help is to lift the quality of the software and thereby reduce incidents.

Fewer Severity 1 reports mean fewer "all hands on deck" interruptions.  A long-term strategy to reduce technical debt, ideally using test-driven development, pair-programming and old-fashioned smarts is a great start.  Israel Gat suggestions monetizing technical debt to make it visible on the balance sheet.

Non-urgent fixes can wait (a bit).  One approach that I have tried is to schedule a day per week for maintenance -- e.g. bug-fix Thursdays -- but this is a bit inflexible.  Another approach is to estimate and schedule the fixing of reported defects in story points.  The problem with this is that estimating how long it will take to fix non-obvious bugs is uncertain.  Mike Cohn suggests another approach: assigning a few points of velocity each iteration to maintenance (convertible to the equivalent number of hours), and tracking the hours spent for reporting and to help guide future allocations.

Minor enhancements
Product teams should be brutal in only doing the most minor enhancements during minor (also known as bug-fix) releases, since such work sucks development effort away from the major-release work and needs to be merged back in, a source of overhead.  Also, the faster response-time may train stakeholders to ask for "minor enhancements" or insist that a feature request is really a defect.

An exception may be made if you have 100% automated test coverage and are practicing true continuous deployment, in which case you've achieved a higher standard of Agility than most!

Ranking Rummy

Where Planning Poker works brilliantly to estimate the size of a task by drawing on the expertise of a group, I feel the need for a similar approach to obtain a consensus ranking (e.g. for prioritization).

Here is my approach -- which I hereby dub "Ranking Rummy" -- to this problem:

Example/procedure: Consider four stakeholders have identified five "top priority" tasks for a project, but the Project Manager has requested a strict ranking.

1. The tasks are labelled and each participant is given (or produces) five corresponding cards to rank independently and secretly -- no discussion.
2. When the participants are ready, they all reveal their highest priority card.
3. If all are in agreement, we have our consensus highest priority, and can move on to find the next highest.
4. Otherwise there will be two or more factions (groups who played the same card).  Similar to Planning Poker, the facilitator asks (only) one participant from each faction to explain his or her choice.  Then there is re-consideration, and the process is repeated until consensus has been achieved or, say, three rounds have been played.  If consensus is not achieved after three rounds, a tie-breaking mechanism is employed: E.g. majority or seniority.
5. Repeat for the remaining tasks.

As with Planning Poker, the procedure encourages initial independent ranking, and structured yet informative communication about the tasks under consideration without negotiation and/or intimidation.  

Seniority is a useful tie-breaker if your group is following a consultative (but non-democratic) approach.  For example, if prioritizing among Time, Scope and Cost, the project sponsor will typically have the final call, but Ranking Rummy would allow him or her and the stakeholders to get an understanding of each others' views and preferences along the way.

Applications

• Ranking tasks, users stories, features
• Alternative to value sliders [pdf], where a strict ranking is desired.

Please let me know how you go with Ranking Rummy.  Suggestions (and donations) gratefully accepted!

No Plan Survives First Contact With the Enemy

... or friends, for that matter.

This saying, from the context of battles, applies equally to projects where outside influences are significant.  Initial planning swiftly becomes outdated, so it wise to plan at first in only broad brush strokes -- at a high level -- perform detailed planning as we go along.

For those who have limited experience as military command consider instead a game of chess (or checkers).  One might go in with certain intentions (play aggressively, use set plays when possible), but there is no way that you can plan out the whole game in advance!  You might plan a few moves ahead, but as soon as your opponent moves, everything shifts, and it's time to re-plan.

1972: Fischer challenges Spassky

As for friends (or family), suggest a plan and they'll want to get on board ;-) too!  Your plans will shift.  In a project this is the equivalent of an internal customer or product owner suggesting a better idea mid-way through the project -- and you acknowledge that it really is better.  Hmmm ... better re-plan.

A.G.I.L.E.

Here's an exercise that I invented to help me quickly get a group engaged at the start of a course, and also give me some insight into their pre-existing experience and attitude towards Agile.

Duration: About 10 minutes
Grouping: Pairs, whole group

Procedure

1. Whiteboard or butcher's paper: write the letters A, G, I, L, E vertically
2. Ask participants to work with the person next to them to turn A.G.I.L.E. into an appropriate acronym in two minutes.  Tell them not to worry if they can't come up with a letter: we'll being pooling the results soon enough.
3. Go around the room and write down whatever they come up with.
4. Have a quick scan of the results and give them (positive) feedback about their degree of familiarity with and/or open-ness to Agile.

Benefits

1. Quick, fun and creative.
2. Sets the tone for working with others in the group and sharing ideas.
3. Gives the facilitator useful insights into the group's existing knowledge of and their attitude towards Agile.

Troubleshooting

• I prefer to get some associations, e.g. "A is for Adaptive Planning, G is for Group cooperation ...", but sometimes get a coherent sentence: e.g. "Always Generate Insights & Leverage Experience" -- which is a nice way of saying "Inspect and Adapt", but is showing inventiveness rather than mining awareness.
• Suggestion: Write "A is for ..." to steer in the preferred direction, or do it twice and ask for both.

Adaptability

• Faster to use pairs than individuals during the collating phase.  For large groups consider using trios instead of pairs.
• It's better if the topic at hand has a short name.  E.g. Better to use L.E.A.N. than C.O.N.T.I.N.U.O.U.S. I.M.P.R.O.V.E.M.E.N.T.!

Failing to Plan is Planning to Fail

Winston Churchill

It was Winston Churchill, during World War II, who memorably said:

He who fails to plan is planning to fail.

But in our smaller projects how many times should we plan, and when?

I propose 4 possible answers: 0, 1, 2, many times.

0: Seat of the pants
This is perhaps permissible for small tasks, and certain expert work, especially when it's been done by said experts many times before.  Jazz musicians and theatrical improvisors are ace at this.

In software development this is called "cowboy coding", and is not recommended for serious projects.

1: Classic waterfall and variations
The big risk here is that on larger projects that if and when the plan fails, we must now re-plan (and we've already blown our whole planning budget) or follow the original plan to ruin.  If the project is long, complex, requirements are poorly understood or subject to change it is prudent to reserve a portion of the planning budget for later re-planning.

A superior variation to doing all planning at the start of the project is what I call "research then development".  This consists of a research phase at the start with no planning in which prototyping and other risk-ameliorating work is undertaken, before the big plan is laid down.  It allows for key learning during the research phase.  From a governance perspective these two phases may be considered as separately funded projects.  It's only after the research phase that we are in position to estimate the cost of the more expensive development part.

2: Build one to throw away (and other variations)
Buiding a cheap prototype to explore risks and gain understanding of the challenges of the particular project before locking down the full plan is quite a good strategy.  Recommended in the original waterfall paper by Winston Royce and by Fred Brooks in The Mythical Man Month (although Brooks now advocates the next approach).

Many times: The Agile way
If we accept that projects involve learning, then it makes sense to plan repeatedly, first in a broad sense, and then refining as we learn more.  Adaptive planning embraces change, both in inner factors (learning) and outer factors (changing requirements or circumstances).  The refinement of fixed length iterations and repeated planning encourages us to expend our planning budget over the course of the project, and minimizes the amount of early planning under incorrect or outdated assumptions that is not only a waste in itself, but also leads to other misguided (hence wasteful) activities.

Conclusion
What would Winnie do?  The picture suggests that he would plan twice (or possibly follow the V method of development!).

For my part, bad experiences with too much planning up-front (and the consequent waste and lack of resources to re-plan) led me to more successful iterative and incremental approaches, and later to the shorter, fixed length iterations of Agile.

Agile is, in part, about not being surprised "when our best laid plans go astray", and instead of being disappointed or abandoning planning, "planning to re-plan" ... cheaply.

An Agile Mnemonic: WRIST SPIN & PACE

Well, maybe more of a mnemonic.  I tried to cook encapsulate a not too long list of the keys to Agile success, and this is what I came up with:

The wrist spinner's stock delivery

Workshops
Retrospectives
Iterations
Stories
Test-driven development

Sustainable pace
Prioritization
Incremental design
iNformative workspace

and

Pair programming
Adaptive planning
Continuous integration
Estimation

The basis for a good bowling attack, plus my choice of foundation practices for Agile success.  Must be cricket season.

Howzat!?

Scaling with Kanban

Here's a neat idea for scaling Agile from David Joyce:  As an alternative to Scrum of Scrums, consider Kanban at the programme level:

The programme board has cards for each of the sub projects or sub feature sets (MMFs) only. The detail for each of these is broken out on each of the teams Kanban boards, not on the programme level board.

This approach visualises what is going on at a higher level, and enables the various representatives from each of the sub teams to collaborate, understand what is coming their way that could affect them, and facilitate synchronisation.

A daily standup is still held, but the rhythm is around:

1. what is blocking your team, or about to block another team
2. what work is in progress
3. bottlenecks (either current or impending)
4. are priorities clear on what gets pulled next
5. what needs to be expedited

The standups still run from right to left on the board, in other words upstream; from what is about to released, back all the way to analysis.  More...

If we think of a story wall as a dashboard for the team (and other interested parties), the programme level Kanban can be thought of as a (higher-level) dashboard for the overall programme.

What is project success?

Moonshot: Successful or "Challenged"? 

What defines project success?  The end result?  The net benefit?  Delivered according to plan?

The Standish Group surveys the outcomes of large IT projects and reports periodically in its Chaos Reports.  For example, in 2008 Standish found that:

• 32% of the projects surveyed "succeeded" (delivered on time, on budget, with required features and functions)
• 44% were "challenged" (late, over budget, and/or with less than the required features and functions), and
• 24% failed (cancelled prior to completion or delivered and never used)

I'd like to note that by the above definitions, a project can both succeed and fail at the same time, if the planned features and functions are all delivered on time and on-budget, but do not add up to a usable product.  [Presumably "failed successes" are classified as failures so that everything adds up to 100%.]

Jim Highsmith observed at the 2010 Agile Australia conference that such surveys measure failures of planning more than performance.  For example, a challenged product (say late), could still deliver enormous business value, while an on-time, on-budget, and "fully featured" might be a flop in practice.

Here's a graphic example.  By Standish standards, the project to put a man on the moon was "challenged".  The time goal and stated objective was met -- do it before the end of the 1960s -- but in terms of cost:

[A] preliminary cost estimate of $7 billion dollars was generated, but this proved an extremely unrealistic guess of what could not possibly be determined precisely, and James Webb used his administrator's judgement to change the estimate to $20 billion before giving it to Vice President Johnson.  Webb's estimate shocked everyone at the time, but ultimately proved to be reasonably accurate. The final cost of project Apollo was reported to Congress as $25.4 billion in 1973.

Informally, this project was one of the greatest successes of all time!  The difference is that informally we look at the realized benefits, as well as the costs incurred.

In practice judging project success is a slippery business, and undertaking large-scale research a challenging undertaking  Part of the challenge is devising good measuring sticks.

Tetris-shaped people

I am unsure who coined the term T-shaped people, but Tim Brown of Ideo gives the gist:

We look for people who are so inquisitive about the world that they're willing to try to do what you do. We call them "T-shaped people." They have a principal skill that describes the vertical leg of the T -- they're mechanical engineers or industrial designers. But they are so empathetic that they can branch out into other skills, such as anthropology, and do them as well. They are able to explore insights from many different perspectives and recognize patterns of behavior...

T-shaped people tend to have the right stuff for cross-functional, self-organizing teams so prized in Agile organizations: we need individuals with some deep special skills, plus at least some broad general skills, and play well with others.

Tetris people!

Going further, I would like to generalize the from T-shaped people to Tetris-piece-shaped people, or Tetris-shaped people (for short).  Remember Tetris, the addictive game of falling blocks?   If you don't, go play it now, and come back when you're done.

Since individuals come with a mix of skills, we need different shapes that fit well together to assemble a great team -- just like the blocks in Tetris.

Also, remember how in Tetris you can rotate the blocks as they fall?  This corresponds to encouraging people to deploy their skills in different ways to help the team achieve its goals.

Take home messages:

1. Recruit T-shaped -- or, better,  Tetris-piece-shaped -- people to your Agile team(s).
2. People as cogs in a giant machine are out; people as Tetris-pieces are in.
3. All that time playing Tetris was not wasted!

Finally, on a highly entertaining tangent, check out this hypnotic performance of Human Tetris from Project GAME OVER:

Have your cake, and eat it too!

You can try for the best of both worlds:

and hope to get Nimble Cascade, but I bet you'll get Half-Arsed Agile instead!

* * *

Here's the original Agile manifesto, plus Ron Jeffries' take on why it should not be updated.

The Agile Triangle

Jim Highsmith gave a great opening keynote at the 2010 Agile Australia Conference.  One of his most striking visuals was a proposal to replace the traditional "iron triangle" -- cost, scope, schedule -- with an Agile upgrade:

Note that the original triangle has not gone away; rather it has been subsumed under Constraints.

The full slides of the talk are available, as are more extensive reports of the session from Craig Smith and my colleague Shane Hastie.

The Agile Triangle

1. Value: releasable product
2. Quality: reliable, adaptable product 
3. Constraints: cost, schedule, scope

In other words, in Agile we're looking at a bigger picture: making software that organizations want, because

• they deliver real Value now by addressing reals needs, even if these weren't well-captured at the start of the project, and
• are of a Quality that will stand up now, and also and in the future when modifications are (inevitably) required, while 
• smart choices are made to keep the Constraints under a reasonable level of control.