What is project success?

Moonshot: Successful or "Challenged"? 

What defines project success?  The end result?  The net benefit?  Delivered according to plan?

The Standish Group surveys the outcomes of large IT projects and reports periodically in its Chaos Reports.  For example, in 2008 Standish found that:

• 32% of the projects surveyed "succeeded" (delivered on time, on budget, with required features and functions)
• 44% were "challenged" (late, over budget, and/or with less than the required features and functions), and
• 24% failed (cancelled prior to completion or delivered and never used)

I'd like to note that by the above definitions, a project can both succeed and fail at the same time, if the planned features and functions are all delivered on time and on-budget, but do not add up to a usable product.  [Presumably "failed successes" are classified as failures so that everything adds up to 100%.]

Jim Highsmith observed at the 2010 Agile Australia conference that such surveys measure failures of planning more than performance.  For example, a challenged product (say late), could still deliver enormous business value, while an on-time, on-budget, and "fully featured" might be a flop in practice.

Here's a graphic example.  By Standish standards, the project to put a man on the moon was "challenged".  The time goal and stated objective was met -- do it before the end of the 1960s -- but in terms of cost:

[A] preliminary cost estimate of $7 billion dollars was generated, but this proved an extremely unrealistic guess of what could not possibly be determined precisely, and James Webb used his administrator's judgement to change the estimate to $20 billion before giving it to Vice President Johnson.  Webb's estimate shocked everyone at the time, but ultimately proved to be reasonably accurate. The final cost of project Apollo was reported to Congress as $25.4 billion in 1973.

Informally, this project was one of the greatest successes of all time!  The difference is that informally we look at the realized benefits, as well as the costs incurred.

In practice judging project success is a slippery business, and undertaking large-scale research a challenging undertaking  Part of the challenge is devising good measuring sticks.