Professional Documents
Culture Documents
viewpoints
The Profession of IT
Evolutionary System
Development
Large systems projects are failing at an alarming rate.
Its time to take evolutionary design methods off the shelf.
M
a ny critical large sys- sep05/1455), NMCI (GAO4, www.nm- in many small increments that aligned
tems are failing. The cistinks.com) all faced dynamic envi- with current perceptions of the using
replacement FAA air ronments that changed faster than their environment.
traffic control system, development processes could. Prede- Moreover, the evolutionary process
the FBI virtual case file, cessors of these systems were success- embraces risk, and the patience to see
and the Navy Marine Corps Internet ful because their environments were what emerges. It works with natures
(NMCI), are a few of the many billion- stable, but the current generations en- principle of fitness in the environment:
dollar systems that could not deliver the
functions needed. In stark contrast, the
Boeing 777 aircraft, the Global Position-
ing System (GPS), and the U.S. Census
database system have been outstanding
successes. Why do some systems fail
and others succeed?
Development time is the critical fac-
tor. This is the time to deliver a system
that meets the requirements set at the countered trouble because their envi- components that work well survive,
beginning of the development process. ronments had become too dynamic. and those that do not are abandoned.
If development time is shorter than the The traditional acquisition process The astonishing success of evolu-
environment change time, the delivered tries to avoid risk and control costs by tionary development challenges our
system is likely to satisfy its custom- careful preplanning, anticipation, and common sense about developing large
ers. If, however, the development time analysis. For complex systems, this pro- systems. We need to learn from these
is long compared to the environment cess usually takes a decade or more. Are systems, because evolutionary develop-
change time, the delivered system be- there any alternatives that would take ment may be the only way to achieve
comes obsolete, and perhaps unusable, much less time and still be fit for use? satisfactory replacements for aging
before it is finished. In government and Yes. Evolutionary system develop- large systems and to create new, unprec-
large organizations, the bureaucratic ac- ment produces large systems within edented systems.
quisition process for large systems can dynamic social networks. The Internet, Evolutionary development is a ma-
often take a decade or more, whereas World Wide Web, and Linux are promi- ture idea that has languished away from
the using environments often change nent examples. These successes had no mainstream practice. In this column,
significantly in as little as 18 months central, preplanning process, only a we will analyze why evolutionary devel-
(Moores Law). general notion of the systems archi- opment does not fit the current com-
The Boeing 777, GPS, and U.S. Census tecture, which provided a framework mon sense and why we need to work to
data systems were developed for stable for cooperative innovation. Individu- change that.
environmentsthey were completed als in the network banded into small
before any significant changes occurred groups to quickly produce or modify Our Current Common Sense
in their requirements. In contrast, the modules in the architecture. They tested From its founding in 1968, the software
FAA replacement system, FBI Virtual their modules by asking other users to engineering field set out to address the
Case File (see www.spectrum.ieee.org/ try them. The systems evolved rapidly software crisis, a persistent inabil-
ity to deliver dependable and usable hold? The first assumption is challenged cycles cannot keep up with real, dynamic
software. Fritz Bauer, one of the fields by the failures of large systems that used environments.
founders, believed a rigorous engineer- the traditional design process and the It may come as a surprise, there-
ing approach was needed. He famously successes of other large systems that fore, that practices for adaptability are
quipped, Software engineering is the simply evolved. The remaining assump- allowed under government acquisi-
part of computer science that is too hard tions are challenged by the increasingly tion rules. In 2004, the Office of Secre-
for computer scientists. Over the years, dynamic environments, often called tary of Defense sponsored the launch
software engineers produced many ecosystems, in which large systems op- of W2COG, the World Wide Consor-
powerful tools: languages, module man- erate. There is no complete statement tium for the Grid (w2cog.org) to help
agers, version trackers, visualizers, and of requirements because no one person, advance networking technology for
debuggers are some examples. In his or even small group, can have complete defense using open-development pro-
famous No silver bullet assessment knowledge of the whole system or can cesses such as in the World Wide Web
(1986), Fred Brooks concluded that the fully anticipate how the communitys re- Consortium (w3c.org). The W2COG
software crisis had not abated despite quirements will evolve. took advantage of a provision of ac-
huge advancements in tools and meth- quisition regulations that allows Lim-
ods; the real problem was getting an System Evolution: ited Technology Experiments (LTEs).
intellectual grasp of the problem and A New Common Sense The W2COG recently completed
translating that understanding into an To avoid obsolescence, therefore, a sys- an experiment to develop a secure
appropriate system architecture.2 The tem should undergo continual adapta- service-oriented architecture system,
tools of 1986, while better than those tion to the environment. There are two comparing an LTE using evolutionary
of 1968, relied on concepts that did not main alternatives for creating such ad- methods against a standard acquisi-
scale up to ever-larger systems. The situ- aptations. The first, successive releases tion process. Both received the same
ation today is much the same: tools are of a system, is the familiar process of government-furnished software for
more powerful, but we struggle with software product releases. It can work an initial baseline. Eighteen months
scalability, usability, and predictability. in a dynamic environment only when later, the LTEs process delivered a
Current software engineering is the release cycle is very short, a difficult prototype open architecture that ad-
based on four key assumptions: objective under a carefully prescribed dressed 80% of the government re-
Dependable large systems can only and tightly managed process. Windows quirements, at a cost of $100K, with
be attained through rigorous applica- Vista, advertised as an incremental im- all embedded software current, and
tion of the engineering design process provement over XP, was delivered years a plan to transition to full COTS soft-
(requirements, specifications, proto- late and with many bugs. ware within six months.
types, testing, acceptance). The second approach to adaptation is In contrast, after 18 months, the
The key design objective is an ar- many systems competing by mimicking standard process delivered only a con-
chitecture that meets specifications natural evolution; the more fit systems cept document that did not provide a
derived from knowable and collectable live on and the less fit die out. Linux, functional architecture, had no working
requirements. the Internet, and the World Wide Web prototype, deployment plan, or time-
Individuals of sufficient talent and illustrate this with a constant churn of line, and cost $1.5M. The agile method
experience can achieve an intellectual experimental modules and subsystems, produced a good enough immediately
grasp of the system. the best of which are widely adopted. usable 80% success for 1/15 the cost of
The implementation can be com- Evolutionary system design can be- the standard method, which seemed
pleted before the environment changes come a new common sense that could embarked on the typically long road to
very much. enable us to build large critical systems disappointment.
What if these assumptions no longer successfully. Evolutionary approaches
deliver value incrementally. They contin- Agile Methods for Large Systems
ually refine earlier successes to deliver Agile system development methods
The astonishing more value. The chain of increasing val- have been emerging for a decade.1,3,6
ue sustains successful systems through These methods replace the drawn-out
success of evolutionary multiple short generations. preplanning of detailed specifications
development with a fast, cyclic process of prototyping
Designs by Bureaucratic and customer interaction. The evolu-
challenges our Organizations tionary design approach advocated here
common sense about Fred Brooks observed that software is a type of agile process.
tends to resemble the organization that The U.S. Government Accounting Of-
developing large built it. Bureaucratic organizations tend fice (GAO) has scolded the government
systems. toward detailed processes constrained on several occasions for its uncommit-
by many rules. The U.S. governments ted lip service to agile processes.4 The
standard acquisition practices, based on GAO believes agile processes could sig-
careful preplanning and risk avoidance, nificantly shorten time to delivery, re-
fit this paradigm. Their elaborate archi- duce failure rate, and lower costs. Many
tectures and lengthy implementation people resist the GAO advice because