CHAOS Report on IT Project Outcomes: Difference between revisions
Created page with "=== Historical (1994) === In the original Standish Group survey, only 16.2% of software projects were delivered on-time and on-budget ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). Meanwhile 31.1% were canceled before completion ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). Budget overruns were massive: over half (52.7%) of projects ran 189% over the original cost estimate ([https://www.utdallas.edu/~mr..." |
No edit summary |
||
| Line 1: | Line 1: | ||
=== Historical (1994) === | === Historical (1994) ===__NOTOC__ | ||
In the original Standish Group survey, only 16.2% of software projects were delivered on-time and on-budget ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). | In the original Standish Group survey, only 16.2% of software projects were delivered on-time and on-budget ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). | ||
Meanwhile 31.1% were canceled before completion ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). | Meanwhile 31.1% were canceled before completion ([https://www.utdallas.edu/~mrankin/StandishGroup1994.pdf Standish Group 1994]). | ||
| Line 52: | Line 52: | ||
== Sources == | == Sources == | ||
<!-- | |||
{{Paper | |||
|title=Standish Group CHAOS Report 1994 | |||
|type=report | |||
|author=Byrne, A | |||
|journal=The Australian Library Journal | |||
|volume=57.4 | |||
|published=2008-01-01 | |||
|pages=365-376 | |||
|file=Web 2 0 strategy in libraries and information services.pdf | |||
}} | |||
{{Paper | |||
|author=Magne Jørgensen, Kjetil Moløkken-Østvold | |||
|title=How large are software cost overruns? A review of the 1994 CHAOS report | |||
|journal=Information and Software Technology | |||
|volume=48 | |||
|issue 4 | |||
|published=2006 | |||
|pages=Pages 297-301 | |||
|idtype=issn | |||
|id= 0950-5849 | |||
|url= https://doi.org/10.1016/j.infsof.2005.07.002 | |||
|abstract=The Standish Group reported in their 1994 CHAOS report that the average cost overrun of software projects was as high as 189%. This figure for cost overrun is referred to frequently by scientific researchers, software process improvement consultants, and government advisors. In this paper, we review the validity of the Standish Group's 1994 cost overrun results. Our review is based on a comparison of the 189% cost overrun figure with the cost overrun figures reported in other cost estimation surveys, and an examination of the Standish Group's survey design and analysis methods. We find that the figure reported by the Standish Group is much higher than those reported in similar estimation surveys and that there may be severe problems with the survey design and methods of analysis, e.g. the population sampling method may be strongly biased towards ‘failure projects’. We conclude that the figure of 189% for cost overruns is probably much too high to represent typical software projects in the 1990s and that a continued use of that figure as a reference point for estimation accuracy may lead to poor decision making and hinder progress in estimation practices. | |||
}} | |||
<ref>{{Cite | How large are software cost overruns? A review of the 1994 CHAOS report}}</ref> | |||
<ref>{{Cite | Standish Group CHAOS Report 1994}}</ref> --> | |||
#[[Media:chaos report 1994.pdf|Standish Group CHAOS Report 1994]] | |||
#[[Media:CHAOSManifesto2012.pdf|InfoQ: CHAOS Manifesto 2012 Summary]] | |||
#[[Media:project-success-qrc-standish-group-chaos-report-2020.pdf|CHAOS Report 2020 (Standish Group)]] | |||
#[[Media:state-software-budgeting-handbook.pdf|18F: Why Government Digital Services Fail]] | |||
#[[Media:Belfer Center: IT Failures in Government.pdf|Belfer Center: IT Failures in Government]] | |||
#[[Media:WhydoPublicSectorITProjsFailforINFOS2Mar10.pdf|ResearchGate: Project Failure Rate in IT Sector]] | |||
<references /> | |||
Revision as of 19:59, May 14, 2025
Historical (1994)
In the original Standish Group survey, only 16.2% of software projects were delivered on-time and on-budget (Standish Group 1994). Meanwhile 31.1% were canceled before completion (Standish Group 1994). Budget overruns were massive: over half (52.7%) of projects ran 189% over the original cost estimate (Standish Group 1994). Large organizations fared worst – only 9% of their projects succeeded, with 61.5% challenged and 29.5% canceled (Standish Group 1994). Even completed projects delivered far less than planned: projects in large firms averaged only 42% of their original features (Standish Group 1994). Overall, across 3,682 projects in that sample, only 12% were on-time and on-budget (Standish Group 1994).
“CHAOS Manifesto” (2012)
By 2012, Standish data showed improved results: 37% of projects succeeded (on time, on budget, with full scope), 42% were challenged, and 21% failed (InfoQ summary of CHAOS Manifesto 2012).
Recent (2020)
The latest CHAOS data shows renewed difficulties: only 31% of projects were “successful” (Standish CHAOS 2020 Summary). Fully 50% were challenged and 19% failed (Standish CHAOS 2020 Summary). Small projects performed far better (~90% success), while large projects had <10% success (Standish CHAOS 2020 Summary).
Global Outcomes
Across all regions, approximately 30–34% of projects meet all goals (Standish CHAOS 2020 Summary). Cancellations dropped from ~31% in 1994 to ~19–21% in 2020. In 1994, successful projects in large firms delivered only ~42% of planned features (Standish Group 1994); by 2012, full-scope delivery was expected as part of success (InfoQ 2012 Summary).
U.S.-Specific Figures
Standish surveys include many U.S. firms. In 2020, 31% of U.S. projects were canceled, and 53% were challenged – implying only ~16% succeeded (Standish CHAOS 2020 Summary). Government IT projects fare worse:
Only 13% of major U.S. state and local IT projects succeed (18F: Why Government Digital Services Fail).
Only 13% of large federal IT procurements (>$6M) succeed (Belfer Center: IT Failures in Government).
Europe and Other Regions
Standish doesn’t regularly publish EU- or Asia-specific breakouts. However, regional reviews suggest similar trends:
A European analysis found 30% of projects succeed and 20% fail outright (Project Failure Rate in IT Sector – ResearchGate).
Summary Table
| Region / Year | On Time/On Budget (Success) | Within Budget | Meeting Scope | Canceled (Failure) |
|---|---|---|---|---|
| Global (1994) | 16.2% | 16.2% | ~42% of features | 31.1% |
| Global (2012) | 37% | 37% | 37% | 21% |
| Global (2020) | 31% | 31% | 31% | 19% |
| U.S. Projects (2020) | ~16% | ~16% | ~16% | 31% |
| U.S. Gov't IT (2019) | 13% | 13% | 13% | 87% |
Note: 1994 scope data approximates features delivered. 2020 U.S. figures inferred from reported challenge/failure rates.
Sources
- Standish Group CHAOS Report 1994
- InfoQ: CHAOS Manifesto 2012 Summary
- CHAOS Report 2020 (Standish Group)
- 18F: Why Government Digital Services Fail
- Belfer Center: IT Failures in Government
- ResearchGate: Project Failure Rate in IT Sector