Introduction

Request For Comments (RFC) 1925-4: Some things in life can never be fully appreciated nor understood unless experienced first hand. Some things in networking can never be fully understood by someone who neither builds commercial networking equipment nor runs an operational network.

The following quote was taken from a book review of The Mythical Man-Month¹ that was published on the rather edgy Internet site known as Slashdot:

What's bad about this book is that people don't read it. There's no particularly good reason, they just are too busy reading the latest book on today's fad. The irony, of course, is that today's fad will be laughed at next year, while MMM will still be around a decade from now...²

This is by no means the most recent review of The Mythical Man-Month and that, in itself, is an interesting phenomenon. In an industry that considers something from last year to be hopelessly obsolete, a book that was published over thirty years ago still has much to offer. As the quoted review states though, unfortunately, not enough people have read The Mythical Man-Month. Based on my experience in various software development roles, I have to agree. My career as a software development professional started in 1980 at what was then TRW Defense and Space Systems Group (DSSG)³. TRW DSSG was then at the forefront of large scale software development such that projects spanning several years with hundreds of developers were considered normal. Although I don't have a source other than a hazy memory of a conversation, at the time, TRW DSSG claimed to be the third largest producer of computer software (after the U.S. Government and IBM). Dr. Barry Boehm was the company's Chief Software Engineer and it was during this period that he wrote Software Engineering Economics⁴ and created the Constructive Cost Model (COCOMO). Given this background, I professionally “grew up” in an environment where The Mythical Man-Month was considered gospel. After leaving TRW in 1992, I worked at several different non-defense companies in a variety of software development roles. I initially found release from the high-ceremony, highly regulated development methodologies of the aerospace and defense industries refreshing. Over time, I noticed that only very small projects seemed to be successful without the structure of a well-defined software development methodology. As project size and complexity grew, the same freedom that allowed a small project to be completed quickly became just an excuse for flailing and failure. Attempts to point out that methodologies existed for engineering larger efforts were met with the derisive response, “There isn't time for that.” My experience with “non-methodology” software development culminated in the “death march” described in Chapter 1. This experience ultimately led me to write this book. Behind this simplistic, “there isn't time for that,” attitude, there seemed to be a common set of fallacious assumptions. The scariest part was these assumptions had been debunked in The Mythical Man-Month over thirty years ago. Attempts to point out this out weren't met with any sort of well-reasoned response. Instead, my suggestions were “refuted” by pointing out that The Mythical Man-Month was over thirty years old and, therefore, hopelessly obsolete (see the quoted review, above). Pushing the issue would only elicit an ad hominem attack that usually included comments about “mainframes” and “punch cards” or worse. Pointing out that the principles in The Mythical Man-Month fully explained the most recent development failure was no more successful. These efforts, at best, transferred the ad hominem attacks to whoever was unlucky enough to have been chosen as the designated scapegoat for the last software development disaster. What seemed to be lacking was a chain of causality that directly linked the practices of the previous effort to either its inglorious outcome or the current problems. This book attempts to fill this gap by providing this chain of causality. In order to do this, I will concentrate on the central themes of both the eponymous essay of The Mythical Man-Month and the two essays added for the twentieth anniversary edition, “No Silver Bullet – Essence and Accident” and “No Silver Bullet Refired.” In addition to ideas raised in these essays and the remaining essays of The Mythical Man-Month, this book draws heavily from Peopleware by Tom DeMarco and Timothy Lister⁵ and Death March by Ed Yourdon⁶ for supporting arguments. My first goal is to explain how software development projects are governed by a set of fundamental constraints rooted in both human nature and the nature of collaborative problem solving. Primary among these is that, for projects above a certain level of size and difficulty, the nature of the effort required changes from one of simply implementing a computer program to one of managing the implementation of the project. This transition occurs when the size of the program outgrows what a small homogeneous team can successfully implement. Both the nature of the problem (e.g., the need for highly specialized groups within the development organization in order to solve specific aspects of the problem) and the capabilities of the individual team members have a huge impact in determining when this threshold is crossed. Because of the profound effect individual abilities can have on projects, there is no single, simplistic answer as to whether a particular task can be accomplished by a certain organization within a specified period of time. Although what can be said is that, as the size of effort grows, the more likely it is that organizational and industry productivity norms will win out over individual ability. For larger projects, the cost and achievable schedule for a particular development effort will, at best, be determined strictly by the size and nature of the program to be developed. My second goal is by far the most important to both the organization performing the work and to those who will be using the program that results. I will show how attempts to ignore or circumvent this change in the nature of the effort required have a profound impact on the fundamental quality of the product. Forcing a project into an abbreviated schedule, even with supposedly sufficient developer and technological resources, will rarely result in the specified product being completed within the desired schedule. Forcing projects onto too short of a schedule will have long term quality repercussions that go far beyond just causing burn-out and stress for the development team. As the team attempts the impossible, accepted software-engineering practices are discarded since they generally require more of the one thing that isn’t available: time. This lack of a disciplined development effort initially shows up in a lengthy integration effort but the repercussions continue to haunt the development team and their product. It is this relatively hidden but lasting effect of attempting to do too much in too little time that is the ultimate cost of the folly. Thus, the size and complexity of the problem determines the minimum feasible schedule required for solving the problem. Attempts to force a project to completion in less than this minimum feasible schedule will have serious consequences for the ultimate quality of the product that results. These consequences will go far beyond superficial coding errors and will also limit the ability of the development organization to provide future product revisions. This causal relationship can best be exposed by looking at the practices that become common in such an effort, the resulting software development process and the consequences to the product of the development organization using such a “process.” Thus, this book starts with a study of the “death march” or “constant crisis” (think death march but on a continuing basis) as software development processes and the implications that using either process has for the final product. Finally, this book will look at what an organization can do since the schedule pressures for developing new capabilities are typically not within an organization’s control. Here I will look at both existing attempts to solve the problem and propose an alternative approach. In particular, the applicability of “agile development” methodologies (See the discussion in Chapter 6 for a better characterization of agile development methodologies) will be considered. This will include specific attention to how such methodologies differ from the so-called death march and the types of software development problems these methodologies can and, more importantly, cannot solve. The non-linear nature of effort and schedule required for larger projects will frequently not be understood by the technical staff tasked with carrying out the effort; let alone by their non-technical counterparts in sales, marketing or upper management. No rational software project manager will cut resources and especially schedule duration for a project beyond a certain point. This is exactly what market pressures as expressed through management edict force the development manager to do. It is my hope that by showing how such an effort has the effect of “eating their seed corn” that a more rational alternative will be examined and tried. Such fits of desperation indicate a deep lack of competence in any manager who demands such an endeavor. I’ll somewhat excuse the poor soul who is the recipient of such a demand as long as they now point out the consequences of attempting the impossible. The ideas and concepts presented in this book can be summarized as:

• Big, complex problems are difficult to solve and, thus, take a long time to solve.
• Attempts to force an early solution to such a complex problem will not achieve the desired schedule and will have continuing consequences in the form of deep rooted bugs and instabilities.
• While it is not possible to significantly accelerate the process of solving big, complex problems, some projects are just a collection of small problems that individually aren’t hard to solve. Such projects can usually be developed relatively quickly if attacked with an appropriate methodology.
• Many software projects or products consist of a few, difficult pieces plus a collection of easier pieces.
• The optimal development strategy is to recognize the ease or difficulty of each piece to be developed and then apply the appropriate development methodology given this understanding of the nature of the effort required for each piece. This approach will not make the solution to the hard pieces available any sooner. However, this solution will not be plagued with the bugs and instabilities that result from attempting to develop such a solution under too short of a schedule.

Why these restrictions exist and are not simply a case of bad luck, lack of an appropriate technology, or are not solvable by yet another new development methodology, paradigm or tool set is presented in three sections. The first section consists of Chapters 1 through 4 and looks at why some software projects continue to fail by either not meeting technical requirements or by not being completed within the desired schedule (and frequently both). The second section consists of Chapters 5 through 8 and looks at the various ways in which existing attempts to shorten a project do not and cannot achieve the desired result for complex projects. Chapters 9 and 10 constitute the third section. This section examines what alternatives are available for such projects since the market pressures and other constraints that drive software development schedules aren’t going to go away. Finally, a summary and some conclusions are presented in Chapter 11. To keep the terminology straight, the following nomenclature will be used as I examine different ways to participate in, organize and manage a software development effort. Most of these terms are used in the same way that I have heard them used in a number of different software development organizations. I have also heard a few different usages and this will help prevent any confusion. Practices – The activities and actions of developers used in the course of developing software for the organization. Examples include holding code reviews before committing a code change or new code and cut-and-paste sub-classing as a mechanism for extending a function without making the extension satisfy the original use. Some practices are independent of a methodology (e.g., code reviews) while others are not (e.g., pair programming is highly identified with XP). Process – The collected practices and methodologies that can be said to be used by an organization to develop a software product. Examples here include the “waterfall⁷” development model and, by this definition, the collection of practices commonly found during a death march. A development process may apply several different methodologies to different types of problems or to different aspects of the same problem. Methodology – A defined and engineered collection of software development practices. Examples include the Rational Unified Process⁸ (RUP), eXtreme Programming (XP) and Dynamic System Development Method (DSDM). Software Development Paradigm – A philosophical approach to solving software development problems that results in a collection of related practices. Object oriented development is an example of a software development paradigm. Such a paradigm may be implemented by specific methodologies. Even an organization following the worst practices of a death march may make noises that seem to indicate that some more refined paradigm or even some methodology is in use. Project, product, program – These terms will be used interchangeably to mean the collection of code being developed. As will be discussed later, the same schedule pressures that exist for developing a commercial product can exist for internal development efforts. Customer, user – These terms will also be used interchangeably. Even for internal development efforts, the end user is still the customer. Implementation, expression – The computer code and data that supposedly solves a particular software development problem. The collection of executable programs, data files, etc. that are provided to the customer or utilized by the end user which, when installed on an appropriate platform, purports to solve the real world problem the customer or user wanted solved. Software design – An abstract plan or representation of how the implementation achieves its goal.

---

¹Brooks, F. P., The Mythical Man-Month: Essays on Software Engineering. Reading, MA.: Addison-Wesley 1995 (Anniversary Edition). Reference as TMMM.

²From a book review by Jason Bennett of The Mythical Man-Month as published on Slashdot (http://slashdot.org) on 5 August 1998.

³TRW's defense and space related business sectors went through a number of reorganizations while I was employed there. The company was eventually acquired by Northrop-Grumman and the non-automotive businesses became Northrop-Grumman Mission Systems. The automotive related sectors were then “spun off” by Northrop-Grumman and continue to do business as TRW, Inc.

⁴Bpehm, B. W., Software Engineering Economics. Englewood Cliffs, NJ.: Prentice-Hall, 1981.

⁵DeMarco, T. and Lister, T., Peopleware: Productive Projects and Teams. New York, NY. Dorset House Publishing 1000 (2^nd Edition). Reference as Peopleware.

⁶Yourdon, E., Death March. Upper Sadle River, NJ.: Prentice Hall Professional Reference. 2004 (2^nd Edition). Reference as Death March.

⁷W. W. Royce described the classic waterfall development methodology was in “Managing the development of large software systems” Proceedings of IEEE WESCON, August 1970.

⁸See Jacobson, Booch, and Rumbaugh, “The Unified Software Development Process”, 1999

---

This work is copyrighted by David G. Miller, and is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.