Scrum Musings

Scrum Musings
A collection of thoughts on Scrum and Agile processes from Ken Schwaber during 2003 and 2004.
Workload Management ...................................................................................................2 Managing Work ..............................................................................................................5 Managing the Process......................................................................................................7 Scaling Agile Processes Part 1......................................................................................9 Scaling Agile Processes Part II ...................................................................................11 Scaling Agile Processes Part III..................................................................................13 Scaling Agile Processes Part IV..................................................................................15 Scaling Agile Processes Part V...................................................................................17 Software Development A Science or an Art................................................................19 Capability Maturity Model (CMM) and Scrum..............................................................21 Estimates vs Actual .......................................................................................................24 Forming Cross Functional Teams ..................................................................................27 Cross-Functional Teams and Waterfall ..........................................................................29 Silver Bullets................................................................................................................. 31 Realizing Project Benefits Early ....................................................................................33 Spinmeisters and Professionals......................................................................................35 Only Eat When Hungry .................................................................................................37 Yowzah!!! .....................................................................................................................38 For Customers Only ......................................................................................................39 Nothings For Free ........................................................................................................41 Bidding Work................................................................................................................43 Disenfranchised.............................................................................................................45 New Year Predictions....................................................................................................47
Copyright ADM 2004 All Rights Reserved Page 1of 49 12/26/2004
Workload Management
Agile processes introduce the concept of workload management to systems development. Workload management involves controlling development of functionality and release dates in order to optimize the value to the organization of the system being developed. This is different from work management, in which the specific tasks involved in building a system are directed. Agile processes make workload management possible through iterative, incremental development. Development occurs in a series of short iterations, usually between two and four weeks in duration. An increment of functionality is ready by the end of every iteration. The term ready here means potentially shippable or able to be implemented. Ready means complete, including user documentation, and having been tested fully. Traditional development methodologies fully analyze and design a system before coding it. Testing usually follows the coding. It is not until the very end of the project that the system can be implemented. The opportunities for managing this workload to optimize value are limited and are usually not even considerable. However, agile processes make it possible to perform analysis, design, testing, coding and documentation in every iteration. This provides management with many opportunities to do the following: 1. arrange the sequence in which functionality is iteratively developed so that the most valuable functionality is built first; 2. continue to rearrange the sequence of functionality development as the project progresses and business priorities change; and, 3. group increments of functionality into more frequent releases, allowing the business to realize early and frequent benefits. Consider a system that will bring the organization $1,000,000 dollars in benefits in the first two years after its implementation. Using traditional methods, the system would take one year to develop, and its development would cost $400,000,000. Agile processes let us develop and implement the systems functionality selectively and incrementally by doing the following: 1. listing the functionality of the system, with more attention to the highest value and priority functionality; 2. dividing the functionality list into two releases, the first estimated to be ready six month after development begins; 3. using iterative, incremental development to complete the first release within six months for $200,000; 4. allowing benefits worth $800,000 to begin accruing after just six months, with the functionality that will deliver the remaining value scheduled to be developed during the second iteration; and,
5. permitting the second implementation to be deferred if it is not deemed cost effective and the benefits of the first implementation are deemed sufficientfor example, if the development cost of $200k for the less valuable functionality would generate only $200k in benefits. Workload management such as is described in this example is made possible by agile processes. In this case, the customer had an opportunity to realize $200k in benefits six months earlier than would otherwise have been possible. The customer also had the opportunity to choose not to spend an additional $200k for break-even functionality. The time and effort that would have gone into the second iteration could instead be allocated to other higher priority projects. The benefits of multiple releases are somewhat offset by implementation costs. Strategic and competitive systems are able to gain marketplace advantage through such incremental strategies. Imagine that your competition uses traditional development approaches to prepare a single new release or business capability, but your organization uses agile processes to produce early and repeated competitive advantages. If this is the case, then your organization is able to capture the advantage more effectively and thoroughly. An additional benefit of workload management is inventory reduction. As in manufacturing, unfinished raw goods software inventory is an undesirable cost. It may need to be reworked if it has defects. It may never even be used if production costs are too high or demand for the software evaporates. Yet traditional development methodologies amass huge inventories of analysis, design, and coding artifacts even as business changes render them obsolete. The agile approach minimizes the extent to which an organization accumulates such artifacts. Only those artifacts that are necessary to build each iterations increment of functionalitythe highest priority functionalityare built. This role of workload management is a key new role afforded by agile processes. This role is referred to as the Customer in Extreme Programming and as the Product Owner in Scrum. These roles have responsibilities that enable the realization of the benefits of workload management. The Product Owner executes this role through active management of an inventory called Product Backlog. The Customer executes this role through active management of an inventory of Stories. Lets look more closely at Product Backlog. Product Backlog is a simple list of requirements for the system. Each item on the list is a single line in length. Functional requirements, such as the ability to calculate available credit, are listed along with nonfunctional requirements, such as the capacity to handle up to 100,000 simultaneous transactions with sub-second response time. Product Backlog is often maintained in spreadsheet format so that it can be easily manipulated and interpreted.
The Product Backlog is a prioritized list. Items at the top of the list are those that will deliver the most business value. Business priorities can change over the course of the project, and consequently the order of the list can change as well. Dependent functionality, or functionality that is required to support highest priority backlog, is of an even higher priority. Each backlog item is a one-line description of the requirement. An estimate of how it will take developers to turn the functionality into an increment of potentially shippable product is included in each backlog item. The Product Owner doesnt have to specify all the details of every entry in the Product Backlog. The Product Owner extracts requirements from the systems plan, focusing on the highest priority Product Backlog first. At first, the Product Owner needs to list only as much Product Backlog as is needed to drive the first probable release. The lower priority functionality can be itemized and delivered only when it is deemed to be the highest priority available functionality. Even then, its development may be deferred if it costs more than it is worth. Workload management is one of the three key management roles is agile processes. The other two, process management and development iteration management, will be discussed in upcoming reports.
Managing Work
I previously discussed how agile processes facilitate workload management by allowing for frequent, iterative delivery of shippable functionality and enabling customers and product managers to prioritize direct development of top value functionality, iteration by iteration. Who manages the work during each iteration? The agile answer is: the development team! The workload manager indicates what functionality most need to be developed. The development team identifies and organizes the tasks and work necessary to the results of the iteration into a potentially shippable product increment. Collaborating with the workload managers, the development team determines how much priority functionality it believes it can cover in the next iteration. Agile work management is a radical shift from traditional Project Management Institute priorities. These practices call for a project manager to predict and plan all of the work as well as to assign it to individuals, track its completion, and make any necessary adjustments along the way. Agile work management instead follows modern lean manufacturing practices and engineered process controls used in complex development environments. Agile teams have these characteristics: 1. They are cross-functional, containing all of the technical and business domain expertise to take full responsibility for moving from requirements forward to working product functionality. 2. They are limited in size in order to maximize the speed, content, accuracy, and bandwidth of communications. Team size is approximately seven to ten people. When there are multiple teams, the teams get together to synchronize their work on a daily basis. 3. They are authorized to organize themselves, to divide and assign work amongst themselves. 4. They are enabled to let specific tasks require to create an increment of functionality emerge as the iteration progresses; they are expected to be able to make perfection predictions. For the duration of the iteration, the team has the authority to manage itself. Its main goal is to do the best that it can. Applying the technology to the requirements, the team analyzes, designs, codes, and tests. At the end of the iteration, the team demonstrates what it has accomplished. The team uses workstations to show the workload manager the functionality it has created. Only real working functionality counts to the customer; interim artifacts such as models do not count.
Sometimes the team does less than it has predicted it would be able to. Sometimes the team implements the selected requirements even more deeply than it had expected it could. Regardless, the team does the best that it can. For one iteration, the team alone wrestles functionality from complex, sometimes unstable technology and from oftenchanging business requirements. To many, it may seem risky and even foolhardy to trust the team to plan and execute its own work. However, this type of agile development has been successfully used in literally thousands of projects. Two types of productivity occur. First, the project manager doesnt have to try to tell the team what to do, and then keep the plan up to date as changes are required. Second, the team works more effectively without having to rely on external authority for any changes. The U.S. Marine Corp. uses an approach similar to agile processes for battle situations. In Corps Business (Corps Business, David H. Freedman, HarperCollins Publishers, 2000) General Charles C. Krulak , the Thirty-first Commandant of the USMC, describes the new three block war that the corps faces today Marines may confront the entire gamut of tactical challenges within the narrow confines of three continuous blocks. To prepare the marines, the actual fighters, for this situation, the USMC both trains everyone extensively in all potential skills and situations that can be conceived, and then advises the marines on the context, mission, goals, and risks of every situation before they are sent in. But, from then on, the marines are on their own, making their own decisions. Their officers provide as much tactical information as possible, but the ultimate decisions come from the soldiers. As General Krulak says, on the complex. asymmetrical battlefields of the twenty-first century, effective decentralized control and execution will be essential to mission success. This same type of decentralized control and execution by agile teams is required to successfully cope with the complex changing requirements and complex unstable technology required for todays successful systems. These teams manage themselves based on their skills and understanding of the technical and business domains. Weve described two newly designed roles: workload management and work management.
Managing the Process

Weve looked at two of the three management roles in Scrum and agile processes. The Product Manager manages the workload, telling the team what to build. The team manages the work, figuring out how to turn the workload into increments of functionality. What does the project manager do? The project manager is responsible for maximizing the productivity of everyone involved in the project. The project manager doesnt actually do the project work, but he or she is responsible for doing anything possible to make the actual workers as productive as possible. The project manager is the coach. In Scrum, the coach is called the Scrum Master, and in Extreme Programming the coach is actually called the coach. The project manager is similar to a coach for a sports team. The coach doesnt establish the schedule. The coach isnt allowed on the field during play. All the coach can do is ensure that everyone is as well prepared as possible and understands the rules and goals. In Scrum, for example, this means that the Scrum Master ensures that: 1. 2. 3. 4. 5. 6. Everyone involved in and committed to the project knows the Scrum process and its practices and rules; Everyone on the project teams is familiar with the vision, technical and business architectures, development environment, and organizational standards, conventions and practices; The Product Owner builds, maintains, prioritizes and keeps visible the product backlog of functional and non-functional requirements for the project; The development teams build, maintain, and keep visible the Sprint, or task, backlog of work during each iteration; The Product Owner, development team, customers, users, and management meet before each iteration to collaborate and select the most appropriate work for the next iteration; and, The development team presents what it was able to build to the Product Owner, customers, users, and management at the end of every iteration.
A sports coach is responsible for fielding the best team possible for every game. The team members are the best that the coach can field, they are in the best condition possible, they are trained to work together as a unit and individually, and they understand the rules of the game. The Scrum Master has the same responsibilities: 1. Everyone on the project understands the process, practices and rules; 2. Everyone involved in the project, but not actually working on the project, understands how the process and rules apply to them. This usually means training the users, customers, and senior management; 3. Each team member is the best possible person available and has been trained in their functional domain as much as possible; and, 4. The product owner represents the users and customers and has the authority to prioritize the workload. Copyright ADM 2004 All Rights Reserved Page 7of 49 12/26/2004
Prior to the project, the Scrum Master is responsible for assessing the field of play. Is the development environment established? Are all of the engineering practices, standards, and conventions in place? Are the quality, testing, and production environments ready? If not, the Scrum Master ensures that work is staged and prioritized into the product backlog. As the project gets underway, the Scrum Master is responsible for closely watching the development teams. Formally, Scrum affords this opportunity every day at the daily status meeting, and at the beginning and end of each iteration. At the daily status meeting, Scrums rules ask each team member to report if anything is impeding their progress. The Scrum Master is responsible for noting these impediments and doing anything possible to remove them. For example, of needed component software is being held up due to delays in purchasing, the Scrum Master is responsible for facilitating the purchasing process. Informally, the Scrum Master is responsible for being aware of the teams condition, morale, equipment, and everything related to its productivity. For instance, if construction outside the building is disturbing the working environment and making thought impossible, the Scrum Masters job is to fix the problem, however possible. I remember watching Jimmy Williams, a recent coach for the Boston Red Sox, get ejected from the game. His team was dispirited. So Jimmy picked fight when the umpire made a questionable call. After he was ejected, the team was so embarrassed at their spirit compared to Jimmys that they came back and won the game. The Scrum Master is also responsible for measuring project progress. If an iteration is falling behind, the Scrum Master works with the team and product owner to adjust the workload to still meet the goal of the iteration. If functionality isnt being built fast enough to meet a release date, the Scrum Master is responsible for working with the Product Owner and management to adjust goals, functionality, dates, or costs. The Scrum Master inspects the project on an ongoing basis and reports any variances from expectations, then working with everyone involved to make the best possible adjustments. The Scrum Master also produces regular reports that measure expectations to reality, and describes the adjustments that were selected, why they were selected, and how they were implemented. The Scrum Master is the go to person on the project, responsible for everything but with authority over nothing. But, who said that management was easy?
Scaling Agile Processes Part 1

A common and unfounded observation about agile processes is that they dont scale. This belief springs from the early application of agile processes to small projects and the articles describing these projects. Scaling usually refers to the number of people on the project, but Ill extend that definition to include the type of application mission critical, life-critical the type of system - real- time, embedded, n-tier, and even the number of locations involved. An agile project described in the literature is optimized for productivity. It often consists of one small team of experts that is located together in an open space using pair programming with immediate access to the customer, potential users, and decision makers and a bunch of other optimizing practices. Jim Coplien studied the Quattro for Windows (BWP) project at Borland in the early 1990s. The project employed many of what came to be known as agile practices. Indeed, Jims writings1 about the project were the genesis of some of the agile processes. Jim documented the following observations comparing what he saw at Borland to industry averages:
1,000,000 lines of C++ code..........BWP..........Industry standard Time in months.......................31...........>50 Staff................................8............>100 Function points per staff month......77...........2
This type of radical productivity can lead one to extrapolate that a project normally staffed by 500 people could be done with a team of 40 using agile processes. However, as Martin Fowler observed at a recent workshop2, scaling agile projects is the last thing you should do. This doesnt mean that you cant scale an agile project. It means that you should first do everything you can not to scale one. Because, in scaling the agile project, you start to diminish the effectiveness of the very practices that produce such radical productivity. At this workshop, the question was posed as how do you scale agile projects? This question arises from normal management logic of: If we have a project that we estimate at 1,000 function points in size and we want it done in 10 months and our staff average productivity is 2 function points per month, we will need to assign 50 people to the project to get it done in time.
.James Coplien. Borland Software Craftsmanship: A New Look at Process, Quality, and Productivity. Proceedings of the 5th Annual Borland International Conference, Orlando, 1994. 2 Canadian Workshop on Scaling XP/Agile Processes, Banff, Alberta, Canada, February, 2003.
We flipped the question to read, how do you take any large project and reduce it to a small, agile project? In light of the productivity numbers quoted by Jim Coplien, above, and similar productivity experienced repeatedly within the agile community, this seems like a more cost-effective and prudent course of action. For instance, lets use a modest agile productivity multiplier of a factor of 10. Recalculating the example project parameters, the agile answer is: If we have a project that we estimate at 1,000 function points in size and we want it done in 10 months and our staff average productivity is 20 function points per month, we will need to assign 5 people to the project to get it done in time. Of course, agility cant just be declared. A lot of work has to be directed by management to produce an agile project. Management is going to have to be very proactive, getting experts, getting everyone together, providing the best tools, using agile engineering and management practices, and changing development approaches. But the payback seems very attractive. ThoughtWorks is a software development company headquartered in Chicago that specializes in agile software development, using excellent professionals that are well trained and well equipped. ThoughtWorks charges a premium for undertaking projects, but the type of productivity indicated above might mean that any reasonable premium is a bargain. I have managed projects of a very large scale using agile processes. In these cases, reducing the number of people through productivity improvements wasnt an option. I used the Scrum agile process to manage these projects, taking advantage of the coordinating mechanisms and outstanding visibility into project progress and problems provided by Scrum. The worst such project was a Y2K project at a healthcare software provider. Previous attempts to plan, coordinate, and manage this project using traditional project management techniques had failed. Scaling is often thought of in terms of number of people involved in a project, since that often imposes the greatest complexity. The Y2K project had over eight hundred people involved. The project was also very complex in a number of other dimensions usually not found together in a single project. These scaling dimensions include the number of organizations involved (350+), the application (delivery of health care), the criticality (mission critical), the distribution (multiple departments, multiple organizations throughout North America), the type of software (interpretive mainframe, client-server, and web), the type of software work involved (new development, enhancement, maintenance, and bug fixing), the size of the software (over 2500 function points), and the length of the project (over two years).
Scaling Agile Processes Part II

More of my customers have been asking me how to use agile processes, particularly Scrum, to help them manage offshore development. Since offshore development undercuts many of the practices that promote agile productivity, I ask them why they dont just increase the productivity of their teams by thoroughly introducing agility? It seems that offshore development, with its potential for lower unit costs (dollars per programmer day), offers management hope that their losses can be reduced. Since the project is probably going to fail anyway, lets minimize our losses by lowering our investment by using lower priced resources. A more optimistic, agile, way of looking at this problem is to fix the problem at home and increase the probability of success. The agile process sweet spot occurs with teams of seven people, give or take two. These teams can be extraordinarily productive, measurements indicating a potential increase of productivity at least 35 times more than average. Ill describe some of the circumstances that support a team of this size achieving this productivity. Many inadvertent practices reduce this productivity, including scaling, so lets understand how to be as productive as possible before we introduce scaling which reduces team productivity for such goals as quicker time to market. High bandwidth communication is one of the core practices of agile processes. If a team has more than seven people, they tend to need to revert to written documents and formal models to keep everyones activities synchronized. The best communication is face to face, with communications occurring through facial expression, body language, intonation, and words. When a white board is thrown in and the teams work out a design as a group, the communication bandwidth absolutely sizzles. Until the late 1990s, many of the engineering practices have promoted formal documentation of communications, such as formal models, documentation templates, and computer aided software engineering tools. Every time I dont work directly with team members using face to face communications, however, I reduce the communication bandwidth and introduce the probability of misunderstandings. As Im writing this article, Im trying to formulate ideas, understandings, and experiences into words. When you read this article, you try to understand what Im saying within the context of your experiences and context. In the process of narrowing my bandwidth to words, and you trying to expand the bandwidth from words to your understanding, a lot is lost. No matter how well I write and you read. And, most of us are not superb writers and readers. Many of agile practices are aimed at maximizing communication bandwidth. These include: 1. Only using modeling tools and techniques to guide thought processes while on the path to code. Models are not used to document, but to ensure the rigor of the thought process. 2. Colocating teams so that any team member can readily get face to face with any other team members to talk and diagram through a problem. Copyright ADM 2004 All Rights Reserved Page 11of 49 12/26/2004
3. Colocatings team in open spaces to maximize the access within the team. If I want to ask a fellow team member something and I get up from my office, go down the hall, look in his or her office, and find that they arent there, I have both wasted time and lost the thread and depth of my thinking. More than just time was wasted. 4. Colocating teams in open spaces so team members can see each other, see how each other is doing and feeling, and hear when a conversation is occurring in which they want to participate. Privacy is easily obtained by putting on headphones from Bose. 5. Keep iterations to thirty days of less. Longer iterations require communications persistence through such artificial techniques as documentation or modeling tools. If the time between learning a requirement to finishing tested code is kept to under thirty days, the problem and its solution can usually be kept in the mind. 6. Keep the team size as close to seven as possible. Seven minds seem able to attain and maintain a shared mental model of a system and its representation in design and code without artificial aides such as documentation. Misunderstanding and recording time is minimized. 7. Use a shared code library and rigorous coding standards so everyone on the team can readily read and understand the system. If modeling documentation is minimized, the code literally is the design. The code must be easy to read and unambiguous. Variable naming is just one example of these standards. 8. Use agile engineering practices so the team always knows the status of development. Test first development ensures that the code reflects the design and that the code is tested as soon as possible. Source code management, continuous integration, and automated testing suites find errors as quickly as possible. Refactoring keeps the design simple, elegant, and easy to debug. Not writing arcane, heroic algorithms keeps code easy to understand. All of these practices combined mean that when I think I have a working system, it really is a working system that is sustainable and maintainable. This is called an increment of potentially shippable (implementable) product functionality. 9. Hold short daily status meetings. Face to face, team members communicate status and problems with each other. At full bandwidth, the team synchronizes itself daily. These and other agile practices lead to breakthrough productivity. Every scaling practice will reduce the productivity of these teams in support of other goals. Our job will be to understand how to implement these scaling practices as intelligently as possible, so we dont throw out the baby with the bath water.
Scaling Agile Processes Part III Offshore Development

Offshore development benefits from the frequent inspection and adaptation provided by Scrum. There is an opportunity for this at the end of the iteration, at the iteration review. There is also an opportunity for this at each daily status meeting, called a Daily Scrum in Scrum and Standup Meeting in Extreme Programming. However, distances and differences in time zones may make such co-ordination difficult. Regardless, it provides the only benefit afforded by agile to offshore development, so every effort should be made to comply. The daily status meeting is a team synchronizing mechanism. Since agile teams rarely use iterations longer than 30 calendar days, they can co-ordinate their activities at such a meeting without having to rely on the complicated mechanisms of pert charts. Teams synchronize their work every day at these meetings, which are held at the same time and place daily, and rarely last longer than 15 minutes. Each team member answers three questions at this meeting: What have you done since last daily status meeting? This question addresses only the last 24 hours, unless a weekend or holiday has occurred in the interim. Team members only mention the things that they have done that relate to this team and this iteration. For example, the team isnt interested in other work that part-timers might be doing unless it relates directly to their own work. If team members are doing work other than what they had planned to be doing for this iteration, that other work should be identified as an impediment. Anything not related to the teams work is probably an impediment. What will you do between now and the next daily status meeting? This question relates only to this iteration and this team. What is each team member planning to work on? The work that team members expect to do should match the work that has been planned by the team. If team members state that they are going to be doing other work, they should be asked why. The team might need to meet after the daily status meeting to talk about the new work. Other team members have to adjust their work based on the new work. Getting answers to these questions can helps the team and management assess whether the work is proceeding regularly and as expected, or if adjustments are needed. What got in your way of doing work? If a team member was unable to work or anticipates being unable to work on what he or she planned, what got in his or her way? That is to say, what is getting in the way of the team? Each team member has planned and committed to a goal and is empirically figuring out the work to meet the goal. What is slowing down individual team members, and therefore the team as a whole? Although team members have worked within the organization and are used to its culture and style, the team coach or ScrumMaster should encourage them to think outside of the box. If Copyright ADM 2004 All Rights Reserved Page 13of 49 12/26/2004
this were the perfect work environment, what else would it have? More specifically, what could help the team be more productive, both as a group of individuals and as a cohesive team? One of my customers has five development sites throughout the United States. This is a reasonable time-zone difference and number of sites to synchronize through Scrum. However, they also have a development site in Finland and another in India. They are investigating opening still another development site in Bejing, China. Each site can readily have their daily status meeting to synchronize their activities within a team. The Scrum agile process uses a mechanism known as a co-coordinating status meeting that synchronizes the work between multiple teams. It is held immediately after the team daily status meetings, is attended by one member of each team, and co-ordinates the work of the teams. At these higher-level coordinating meetings, the team representatives answer the same three questions indicated above. For larger organizations, multiple levels of this coordination can be used, with progressively higher levels meeting less frequently than one day. The time zone differences make planning a daily synchronizing meeting extraordinarily difficult for this organization. Offshore development violates almost every other agile practice that provides the high productivity and quality. This isnt unique to agile, but is a problem for any development process. For instance, agile processes utilize incremental development, with each iteration developing a complete slice of product functionality - from requirements through design to coding and testing. Offshore development requires the development of requirements and architecture at the customer site, and the detailed design, testing and coding at the offshore site. Then acceptance testing and the round of bug fixes and change orders takes place. The customer must fully define all of the requirements upfront, building an inventory that may go obsolete as business requirements change. While the offshore developers design and code the application, it the functionality also may go obsolete and be unneeded. Another tenet of agile processes that offshore development violates is the ability for the customer to steer the project iteration by iteration, based on inspection of each iterations working functionality. The customer ensures that the top priority functionality is developed first, and may not even have lower priority functionality developed. Without this frequent collaboration between development teams and customers, much that the customer doesnt require may be build regardless, and that which is built may not deliver the top business value. Still another violation of agile productivity practices is that absence of full bandwidth communication between all team members, ensuring that nuances that are difficult to capture in documentation are captured. The moment communication occurs through documentation and models, the change for error occurs. The larger or more complex the project, the greater the chance.
Scaling Agile Processes Part IV Too Large Teams

The optimal size of an agile team is about seven people. With this many people, experts can be combined with non-experts to foster mentoring. With this many people, its easier to include all of the skills needed to effectively produce a complete increment of code at the end of the iteration. One coder, one designer, one tester, one documenter is already four people, so the number seven is quickly reached. Fewer people are more effective, with some people even advocating team sizes of three. In my experience, smaller teams are only effective when the increment purpose is restricted. For example, the increment might not include documentation, or the design work might be minimal. Or perhaps, the team consists of three truly outstanding individuals with all of the skills needed. A problem that occurs more frequently is an oversized team. I recently worked with teams of seventeen and fourteen people while implementing the Scrum agile process. At first, I thought that this might be acceptable; I felt that the teams would self-organize to make the size work. They did! The teams almost immediately started dividing themselves into smaller teams. In effect, the teams said, You, management, arent smart enough to optimize our size, so we are going to optimize it ourselves. You gave us full authority on how to work within the iteration, and were going to do it. We see the right thing to do, and were going to do it. Its hard to fight the creativity these teams demonstrated, especially when they were right. The teams demonstrated the beauty of self-organization and emergence. They determined a flaw in the iteration setup and corrected it themselves. But, what was wrong with an oversized team? When I work with a team of seven people, I can see them bend forward to start sharing ideas. I see them exchange thoughts, work through problems, and learn from each other. When I observed these oversized teams, such an easy interchange wasnt possible. For starters, the room had to be oversized to hold all the people. When someone at the far end of the room would say something, people at the far end of the room had trouble hearing them. Because the number was so great, side conversations tended to spring up; this added to the difficulty of hearing what was being said. So much was being said and so many ideas were presented that it was impossible for the various team members to keep track of everything that was going on. A solution to keeping track of everything could have been documentation. We could have required agenda, time slots for presenting ideas, taking meeting minutes, and providing meeting reports that everyone on the team could read. But that would undercut the value of face-to-face communications and the immediacy of intimate interaction. It would also have imposed an overhead on the entire process, exactly the opposite of what agile promotes.
The larger, seventeen-person team spotted this problem itself and divided itself into four sub teams. These sub teams worked on parts of the functionality that were as orthogonal as possible; normally parsing requirements this way is a ScrumMaster and Product Owner responsibility, but the team proved itself equal to the task. Because a perfect orthogonal solution, with perfect cohesion and no coupling, was impossible, the team on its own devised a solution to keep their work synchronized while minimizing collisions. Each team had its own status meeting, called a Daily Scrum in the Scrum process, where the team spends fifteen minutes daily synchronizing work among team members. The team then held a Daily Scrum of Scrums. Representatives of each team met daily after the individual team Daily Scrums to synchronize work between them. They self organized into self-coordination. The teams presented this approach to its management and me; not for our approval since they were using Scrum and were fully authorized to devise their own solutions - but for our information. I was amazed at their creativity. Not only had the team devised a workable solution, but also it was the same solution formally documented in the Scrum methodology for scaling development projects from the small to the large. Except, the team had never seen the Scrum methodology. Working on its own, the team had reached the same optimized solution within three days.
Scaling Agile Processes Part V Virtual Teams Instead of OffShore

I recently read that over 70% of all IT organizations are planning or are already engaged in offshore development. I see my share of this because many of these organizations are turning to Scrum, an Agile process for managing complex projects that Jeff Sutherland and I developed in the early 1990s. Through Scrums iterative, incremental development practices and daily status meetings, these organizations control and coordinate their onshore and offshore activities. I lalm concerned with offshore development from an agile values standpoint. Aside from tilting development practices back to contracts, documentation, and fixed plans, offshore development reinforces the tendency toward waterfall practices. The business domain experts are in one country, the technology domain experts are in another. Analysis and high-level design are done in one country, detailed design, coding and testing in another. Certified ScrumMaster sessions are used to improve the skills and practices of Scrum practitioners that serve as Scrum project managers (www.controlchaos.com/certifiedscrum). At a recent Certified ScrumMaster session in Milan, Italy, the group kicked around the ideas of Scrum versus offshore development. We were looking for a way to mitigate the damaging effects of offshore development through Agile practices. The conversation strayed to Open Source, a movement for collaboratively developing free software. Scrum has practices and rules for iterative, incremental development of software. Open Source has practices and rules for collaborative development of software by many individuals who rarely see each other. The ScrumMasters wondered if merging the practices of Scrum and Open Source wouldnt lead to an Agile solution to offshore development. They wondered if this solution wouldnt be more flexible and in line with Agile values than the manner in which offshore development is usually practiced today. Open Source values are similar to those embraced by the Agile movement (www.agilemanifesto.org) : The basic idea behind open source is very simple: When programmers can read, redistribute, and modify the source code for a piece of software, the software evolves. People improve it, people adapt it, people fix bugs. And this can happen at a speed that, if one is used to the slow pace of conventional software development, seems astonishing. We in the open source community have learned that this rapid evolutionary process produces better software than the traditional closed model, in which only a very few programmers can see the source and everybody else must blindly use an opaque block of bits. from Open Source Initiative OSI Welcome, www.opensource.org.
One of the largest Open Source sites, SourceForge (www.sourceforge.net) has over 64 thousand active projects. Each project has its project administrator that ensures the integrity of the project work to the project vision, ensures the internal product integrity, and forms and guides teams. This role is similar to that of the ScrumMaster. Staff for each project is selected by the project administrator from his or her pool of usual suspects, those professionals who have previously successfully worked on projects with them. Additional project resources are selected from online Open Source job posting boards. On these boards, interested individuals can express a desire to join a project and the project team can select qualified applicants. As teams form and move forward, the project administrator serves as both the Scrum Product Owner and ScrumMaster. The Product Owner is responsible for setting the project vision and prioritizing the work to deliver it. The ScrumMaster is responsible for administering the process for developing the software. The team makes progress based on individual commitments from team members who often hold full time jobs in addition to their Open Source project. We wondered if the concept of the Sprint Planning meeting and the Sprint Review meeting wouldnt help organize these projects into regular iterations that incrementally deliver functionality. The Sprint Planning meeting would allow people to commit for the next iteration based on availability and skills. The Sprint Review meeting would help the team figure out its real progress and optimize its commitments and composition. As a result of these musings in Milan, the Certified ScrumMasters are developing a new approach to offshore development; call Virtual Scrum. This approach will implement many of the ideas expressed above, fusing Open Source and Scrum into an Agile approach to offshore development. The offshore development can either be rapid and focused, using full time teams, or the development can be asynchronous with part time teams from all around the world.
Software Development A Science or an Art

Computer Science reflects the hope that our field can become a hard science and that one day we will understand the first principles underlying the creation of software. And then there are the agile processes, which rely on discipline, emergence, and creativity. All three of these words are descriptive and suggestive rather than first principles, however. For instance, I know discipline when I see it. Similarly, I know that the universe emerged; I just dont know how. One of the laments concerning our discipline, practice, science, or whatever it is currently being described as is that it isnt as good as the way they do it. Like construction. If only we followed solid practices like the construction industry, wed be ok. I wonder why it is that the construction industry is starting to adopt agile processes then, through the ideas of lean construction and the last planner? I recently saw one of these laments in a group discussion: "A key to solving the problem lies in converting software design and development - historically a solitary art practiced by programming "wizards"- into a formal science and engineering discipline governed by widely accepted principles, methods, and best practices for achieving high-quality results. Engineers would not dream of building a suspension bridge without a detailed blueprint of the technical requirements, state-of-the-art disciplinary knowledge of the principles and materials involved, and cost-effective manufacturing processes to produce lasting structural elements of certified quality. As a pervasive, critical underpinning of our national infrastructure, software must be at least as scientifically engineered." The belief stated underlying this persons comment has led into the rigorous, defined, unworkable methodologies that fail so miserably when applied to complex projects (all the one's that I've seen). We seem myopic, looking at other disciplines and wishing that we had the precise, defined, scientific rigor they had. Yet, the "big dig" in Boston is a wonderful example that this isn't true. The project managers laid out great plans and budgets for submerging an elevated highway through Boston into an underground tunnel. However, reality quickly diverged from plan as they dug, uncovering tremendous differences from what they had assumed. For instance, the bedrock on which a "simple to engineer" bridge was to be placed wasn't as expected, and when they started to use it they discovered infrastructure pressures that rattled nearby subway lines. Day by day, iteration by iteration, they had to compromise and revise their plans based on what they found. The project is now some 300% over budget, and everyone says they are incompetent. I believe the only incompetence was in their believing that they had a science and could predict the unpredictable. When they built one suspension bridge, they vastly overspent because - despite all of the science of bridge building - the stress that the cables put on the bridge required empirical tuning and retuning to work.
Some good books that I can recommend on this subject are "American Ground", describing the deconstructing of the World Trade Center disaster, where everything emerged through daily meetings directed by two people with no authority and no detailed plan. And, "The Emergence of Everything", where Harold Morowitz points out that despite popular perceptions - deterministic science has only proven useful when roughly applied. For instance, Newton's laws are used to describe and predict interplanetary movement. Yet they are applicable for only two bodies. The complexity of what really is occurring invalidates these laws when they try to predict the interoperability of three or more bodies. The idea of observing something and reducing it to a set of laws, principles, and practices has been wildly successful in gaining coarse control over our environment. However, using these same laws to predict how finely grained objects interact as they built up have been horribly unsuccessful. It turns out, as complexity science has observed, that very minute variations in the initial conditions often lead to wildly unpredictable perturbations when scaled into the large. I believe that software development is a wonderful subject - both to do and to study. I believe that we need to understand it better to do it better. I have moved Scrum forward because I find it to be a sustainable, scalable framework for software development. But I do not understand how it works. I don't understand emergence, I just understand the beauty of it and how it unleashes creativity. I don't know how models translate from the mind to the software, and I don't know how we make up for the fuzzy boundaries and undefined or weakly defined aspects of models. But I know with inspection, adaptation, and diligence, that we can build software. In "Corps Business", the approach the US Marines take to complexity is discussed. They believe that the world has gotten so complicated and dense that the marines will find every possible battlefield complexity within a three block area - far too complex and dense for top down direction from generals to privates. To address this, the marines train extensively in every possible technique of warfare, in every conceivable circumstance. Before a mission, they meet and discuss the mission - it's objectives, the desired outcome, the risks, the potential variations and complexities. Then they put the troops into the battlefield and count on them to use their training, their common sense, and their knowledge of the situation to do the best they can possible do. The Marine Corps trains leaders, not scientists.
Capability Maturity Model (CMM) and Scrum

CMM is a framework that describes practices that an organization employs in developing software. CMM consists of five levels, level 1 to level 5. Level 1 means that the organization doesnt have any defined, repeatable, or improvable approach to building software; basically, they hack their way to a solution. At level 5, an organization has a defined, repeatable, and improvable set of practices for developing software. Level 1 is considered an immature organization; level 5 is considered a mature organization. At each level the practices that should be employed are defined as key practice areas (KPAs). Bill Curtis and Mark Paulk from the Software Engineering Institute (SEI) at Carnegie Mellon University developed CMM in the early 1990s. If an organization believes that it has thoroughly implemented the KPAs for a specific level, it can engage someone that has been certified by SEI to assess this. If it is compliant, it is so certified. Certification is a big deal as some companies and governmental agencies wont hire any professional services firm that isnt certified to at least CMM level 3. CMM at MegaFund MegaFund is a large fund management company that spent three years and over $40 million dollars to improve its software development practices until it was certified at CMM level 3. At this level, MegaFund not only had a repeatable approach for managing any conceivable software development project, but MegaFund also had formally defined these practices. Unfortunately for MegaFund, it had not defined practices that addressed a time-critical project that would automate ill-defined requirements on advanced and untried technologies like a project that it unsuccessfully tried to initiate after receiving its CMM certification. When MegaFund brought in Scrum for this project, the project had already been stalled for over nine months trying unsuccessfully to jump the procedural and bureaucratic hurdles the CMM level 3 imposed to unforeseen and undefined circumstances. I gained an unfavorable impression of CMM from this and later encounters with CMM implementations. However, as I was asked more frequently what I thought of CMM and how it related to Scrum, I realized that I needed more information and knowledge. To this end, I set up meeting with Mark Paulk at SEI in the fall of 2002.
SEI, CMM, and Scrum Mark was familiar with Extreme Programming, another Agile process similar to Scrum but more engineering and less management focused. However, he had only heard about Scrum. Mark taught me about CMM and I taught Mark about Scrum throughout the first day. On my side, I was quite surprised and impressed by CMM. Mark related that it was only a framework describing an organizations maturity for developing software. How an organization satisfied that framework was up to the organization. Assessors were supposed to determine if the manner in which the organization satisfied the framework were adequate. This enlightened me. Since almost every organization prior to 2001 used defined software development processes, it would of course build rigid, prescriptive, and defined methods for fleshing out the framework. These practices would suffer the weaknesses of all defined approaches they would only work for instances that had been foreseen by those defining the practices. Since software development is a very complex process, there would be very few actual projects to which these practices would be applicable. Mark then went over KPAs for the various levels with me. We then assessed how Scrum fulfilled the KPAs for each level. Mark was pleasantly surprised. Even though Scrum took an empirical approach, someone employing its practices would satisfy all of the CMM level 2 KPAs and many of the level 3 KPAs. The KPAs that werent satisfied at level 3 were those that addressed institutionalizing the practices. These KPAs were addressed in 2003 when the Scrum Methodology, the Certified Scrum Program, and Project QuickStart were made available as products. To see how Scrums practices implement one of the KPAs, Ill discuss KPA 2.6 Requirements Management. The definition of this KPA is: The purpose of Requirements Management is to establish a common understanding between the customer and the software project of the customer's requirements that will be addressed by the software project The Scrum mechanism for meeting this KPA is the Product Backlog, an openly visible listing of all functional and non-functional requirements maintained by the customers that is used to drive development, Sprint by Sprint. The emergent nature of the Product Backlog, with focus only on the top priority items, maximizes the probability that investment in detailing requirements is of valuable. Lower priority Product Backlog that may never be implemented is ignored until and unless it rises to the top of the Product Backlog list. This KPA is often interpreted as demanding requirements traceability, the ability to show how requirements are fulfilled in the delivered system. The manner in which Scrum addresses this interpretation is by demonstrating within thirty calendar days how every Product Backlog item that has been worked on during the Sprint operates as business functionality, The proof is through an actual demonstration of the functionality. As the customer accepts the functionality as complete, that completed item reduces the Product Backlog.
This completely empirical approach to requirements traceability fully meets the requirements of the KPA without extensive documentation or overhead to the development process. It also provides complete flexibility to manage and trace changes in requirements anytime throughout the project. Scrum addresses the rest of the KPAs for level 2 and 3 similarly, empirically and with a minimum of documentation and overhead.
Estimates vs Actual
Everyone wants to improve the predictability and certainty of software development. Given the number of failures and missed deadlines, how can anyone possibly run a business or anticipate how to use new functionality without knowing when the system will be done and how much it will cost? Software development is a complex endeavor. The requirements are hard to state explicitly. The technology works, sometimes. And the interactions between the team members are sometimes great and more often less than great. How is it possible to precisely estimate the amount of work and predict a date give than level of hubbub and complexity? The only way that I know is to study and detail all of the factors involved and then come up with the best answer possible. The more time that Im given to study the problem, the more accurate my estimate will be. Unfortunately, there is a cost to the time that I spend studying the problem. Now, on one hand, I could just go ahead and work on the problem and see how long it takes to do the software development. Or, on the other hand, I could study the problem, come up with an estimate, and then work on the problem. Either way, the work of understanding the problem is part of the overall effort on converting the vague requirement into working functionality. The question is whether I have expended this effort on something that is really needed by the business, or whether I have expended this effort on something that may be needed by the business. The Standish Group estimates that over 60% of the functionality delivered by successful systems projects are never used. One only has to look at Microsoft Excel to validate this statistic; how often does one use all of the great functionality available on all of the menu options, versus just using the standard spreadsheet capabilities of any competent spreadsheet software? What this leads one to potentially conclude is that we should focus our estimation efforts only on those requirements that are essential to the usability of the software. The other estimation efforts should be deferred, since they are nice to haves, that may be dispensed with to meet either cost or delivery date requirements. This is good. Now I know only to worry about the accuracy of the estimates for functionality and requirements that is of the highest value. Having realized this, I face another problem. In all of my years of systems development, I have never seen benefits estimated with any degree of precision. As a matter of experience, the estimates of cost are always better than the estimates of benefits, which are often stated as, this is important to this customer, or, weve simply got to have that! Does it make sense to continue to only work on one side of the equation, cost, when the other side of the equation is so ill-formed?.
Agile processes require a different type of management than called for by the heavyweight, prescriptive processes. The project manager doesnt tell people what to do, the project manager facilitates people doing their work. At a recent Scrum implementation weve faced this question squarely. The line managers, functional managers, and staff from the Program Management Office were trained as ScrumMasters, the title for a Scrum project manager. For the first five or so Sprints, these new ScrumMasters facilitated the correct implementation of the Scrum process. The two most important assigments were getting the business stakeholders to drive development for ROI every iteration, and to help the team learn how to self-manage. While the new ScrumMasters were implementing Scrum, a lot of other work went undone. Every day and at the end of every Sprint, wed find new work that had to be done, but wasnt part of the stakeholders project. All Teams were working on projects, doing the tactical work of systems development as driven by the business stakeholders. We setup a new Team to do this non-project work, an Engineering Management Organizations (EMO) Team, consisting of the managers. The project Team leads were trained as ScrumMasters. The EMO was given the following initial responsibilities: Backlog Management; Release planning and management; Risk, assumption and ROI management; Scrum management; ......................Product Planning - architecture, technology, workflow, and user interface; Development Practices; Productivity Management; and, Quality Management. From this, the following backlog of work was constructed and selected for the first EMO Sprint. Fix up, estimate and make overall product backlog visible; Create next release plan and backlog; Create next +! release and estimate; Define risks, assumptions, and ROI for each release; Establish productivity metrics, trends and history report; Improve engineering practices through coding standards, refactoring, code reviews, and daily builds; Improve defect rate through test driven development and test driven design; Improve defect management; and, Define management reporting and initiate.
An important rule for the EMO team was that it research subjects and define future work but couldnt do work. All actual work would be done by the development Teams, as prioritized and funded through the overall product backlog. The stakeholders and EMO had to prioritize where the money would be spent. At the same time, we ensured that no strategic organization was formed that would tell the teams what to do. The wonderful news is that there is truly meaningful work for engineering management once the non-Agile stuff is stripped.
Forming Cross Functional Teams

A cross-functional team consists of people from all of the disciplines necessary to accomplish the work. The entire team, not specific individuals, is responsible for the success or failure of the effort. Agile development teams are cross-functional. They are responsible for the creation of an increment every iteration. If the increment isnt successful, the team has failed, not individuals in the team. For instance, if user documentation is part of an increment, the team collectively is responsible for that user documentation being completed as part of the increment. If it isnt there, it isnt the fault of the documentation person on the team; it is the fault of the entire team. As the team moves forward during an iteration, its members plan and work together. They lay out the tasks that each of them will perform to successfully build the increment. People with particular expertise take a lead role in that part of the increment construction, such as the people with design expertise taking a lead in how to describe the increments user interface. The technical writer will take a lead role in figuring out the work for building the user documentation. However, it is the responsibility of the team as a whole to complete all of the work and for the completeness of the entire product. I recently saw a team where the technical writer felt he was behind and letting the team down; he felt this because the user documentation wasnt complete at the end of the iteration. He felt guilty and was working overtime and weekends to make of for this. This was wrong and an incorrect understanding of the nature of a cross-functional team. He is only a member of the team, and the team is responsible for building the entire increment, documentation included. If there were overtime to be worked to build user documentation, everyone on the team should have been working it. Then everyone on the team should have discussed how to avoid that crunch during the next iteration, how to start addressing the documentation component of the product earlier in the iteration to avoid the last minute crunch. Cross-functional teams usually have to be built; most organizations dont already have them. Building such a team is difficult since it usually cuts across several embedded understandings. The first understanding is that there are areas of functional expertise, such as analysis, design, programming, testing, and documentation. People who follow a career path in each functional area are the experts and are expected to be the only people who perform this type of work. Others are deemed not capable of performing work outside their area of functional expertise. To exacerbate this problem, most organizations are used to a waterfall methodology for software development. The analysts analyze the problem and describe it; then the designers use the analysis to create a design, the programmers take the result of the design and create code, and so forth. The consequence of this is that when a cross-functional team is formed from people with such a functional orientation, they operate as a mini-waterfall within the iteration.
The analyst starts, performing the analysis of the problem. While the analyst is analyzing, the others try to find things to keep busy until it is their turn. One by one, each gets a waterfall turn to apply their expertise. Finally, the technical writer gets to start the documentation, usually with no time left. I help teams become cross functional by asking the analyst how the other team members can help? The analyst is surely the expert, but how can the analyst direct the others. By directing the others to do analysis, the whole process is sped up, everyone is aware of the results of the analysis, and the need for analysis artifacts is minimized. If this shared work occurs throughout the iteration, the progress is more rapid and cross-functional training occurs. Everyone pitches in. The time-box of the iteration helps the team realize the benefits of this approach, since a strictly partitioned functional and waterfall approach usually fails to deliver a completed increment within the time-box.
Cross-Functional Teams and Waterfall

I was teaching a class on how to be an agile project manager last week. These classes are called certified ScrumMaster classes; attendees discuss how to implement Scrum, one of the agile processes, into their environment. Most of the time is spent discussing the unique difficulties that are expected in the attendees organizations. The topic of greatest interest at this class was cross-functional teams. Scrum and agile processes are iterative, producing an increment of product functionality that is potentially shippable at the end of each iteration. The people who do this work, creating this increment, are the people comprising the agile team. These teams are small, ranging from five to ten people. This team is considered the heart of the agile process, and they are orders of magnitude more productive than traditional software development teams. Drawing from the principles of lean manufacturing, the teams are empowered to figure out how to do their work themselves, and then proceed to do it; that is, they rely on creativity to generate productivity. After all, who knows better how to do the work than the people doing it? Agile teams are cross-functional. This means that the team consists of people with all of the skills necessary to create the increment of functionality every iteration. In many instances, this means that people with analysis, design, testing, coding, and technical writing skills are put together into the team. The team selects how much work it can handle for the iteration, and then proceeds to build that functionality. The greatest impediment to teams working together is the legacy of the waterfall process. A team that is used to waterfall development works in fits and starts. The analyst does the analysis and creates a requirements document. The designers then take over and write up the specifications document. The coder then writes the code. The tester tests the code. And, when everyone else is done, the technical writers starts on the online help and documentation. While each person does their work, the rest sit around, waiting, doing busy work, or cleaning up previous increments. The project managers tell the teams that they should act cross-functionally, they should forgo the tradition of waterfall. The team may try to do this, to work together, but tradition undercuts their efforts. The analyst thinks, Im really the only qualified person to do this, and if I dont clearly document the requirements everyone else on the team will make mistakes! Not only that, but the analyst has a functional manager and a career path that rewards how well she does this analysis. Even the modeling processes and tools reinforce waterfall, starting at the high level and gradually decomposing into code. How do we get the teams to operate cross-functionally, as a team rather than as a group of individuals working in a sequential waterfall? What can management do? The answer is hard to accept do nothing. We often think that teams consist of primitive individuals without the intelligence to figure things out on their own, that they must be told what to do. If we flip this belief and rely on the native intelligence and maturity of the individuals comprising the teams, they almost always come through. It is hard to wait for this selforganization to occur, but patience has its own rewards. Copyright ADM 2004 All Rights Reserved Page 29of 49 12/26/2004
The team starts the first iteration in waterfall mode and is disappointed at how little it can accomplish. Usually, at the end of the iteration the coding is incomplete and no testing or documentation has been done. The team thinks about this and realizes that it would be more efficient if the analyst were responsible for analysis, but used everyone on the team as his or her legs to get it done. By doing so, the tester is aware of what must be tested early in the iteration as well as helping with the actual analysis. Also, since everyone is doing the work, no documentation is needed since they are already aware of the requirements. And this proceeds from analysis to design, from design to coding, and so forth. The entire team is responsible for the results of the iteration; functional specialists teach everyone how do help with their area of expertise, magnifying the productivity of the team. Consequentially, everyone on the team becomes cross-trained and can fill in for each other. Most project managers are used to telling people what to do. If a problem exists, study it and direct people to fix it. Self-organization is much more difficult. We must wait for it to occur, and it cant be hurried. Sometimes we can help team members have insights through anecdotes, metaphors, or just conversation, but we cant make a team do something as complicated as cross-functional work by directing it to do so. The project manager may help the team get to this point by questioning it, Gee, would you be able to do the analysis faster if everyone on the team helped, with you directing? But the project manager cant tell the team to be cross-functional; the team must realize how to do this and do it themselves.
Silver Bullets
I dont know if other industries fall prey to the silver bullet myth as hard as systems development. The belief that there will be some magical incantation, tool, or facility that will kill the difficulty of software development has haunted our industry as long as Ive been developing software, and I remember the 60s. In some ways the belief in the silver bullet was good; it fueled the growth of whole tool industries remember Computer Aided Software Engineering tools from Index Technology, Knowledgeware, and a host of other companies? Remember models that were supposed to generate code that could be maintained from the model? Remember heavyweight methodologies? A test for the presence of a silver bullet is, does using this somehow make the practice of building software for me and those who work with me substantially easier in a way that I dont fully understand? If the answer to this is yes and this thing costs money and is being sold by a company, inspect it thoroughly before buying and implementing it. Im not against magic or simplification. Our jobs of building quality software are hard enough that I believe that we can be forgiven for looking for ways to simplify them. We can even be forgiven for spending lots of time and money acquiring and trying to use silver bullets. In the process, we usually learn something useful, like how to graphically represent our ideas methodically and rigorously. What we wont be forgiven for are both the degree to which these silver bullets have reduced our ability to produce quality software on time and with required capabilities, and the degree to which theyve made it so that we speak separate languages from our customers. I once even taught a course in data modeling to a group of actuaries at an insurance company so that they could understand their systems. Agile processes are sometimes thought of as silver bullets. All I have to do is implement Scrum throughout my organization and things will be better is becoming a mantra, following closely on the heels of all I have to do is implement XP and things will be better. This statement is usually made by senior IT or software engineering management after one or several projects is miraculously turned around and becomes vividly successful after XP and/or Scrum is applied. The problem is that the success of the few is difficult to replicate in the many. The reason is the shortage of people who know what they are doing.
The idea behind most silver bullets is that almost anyone can successfully build systems, as needed, using the silver bullet. Agile processes come in with a different tenet: software development is difficult work and here is a framework and set of practices within which to engage and manage teams in this difficult work. No attempt is made to say that the work is easy or can be done by junior, indifferent, or incompetent employees. Instead, agile processes simply make the impact of unskilled professionals very apparent very quickly. Skilled professionals generate quality systems using agile processes. Unskilled professionals produce terrible systems using agile processes. What agile processes bring to this game is that the bad work of the unskilled professionals becomes quickly apparent and must be dealt with; it wont go away and wont become invisible. Many methodologies keep unskilled work hidden until the end of the project, and some even keep the results hidden until someone attempts to sustain, maintain, or enhance it. Agile processes show the bad quality every day and at the end of every iteration. Agile processes are hard work. If everything that isnt going well is put in managements face every day and at the end of every iteration, life isnt very pleasant. It isnt that all of these problems werent always present; its just that now they are out in plain sight and pointed out to everyone very frequently. Part of my work helping people implement agile processes is getting them through the despair they initially feel when they see all the problems. I call these problems the good news and the bad news: the good news is that you can now begin methodically fixing these problems; the bad news is that you know that they need to be fixed (and so does everyone else). Agile processes keep the bad news visible until it is fixed. If developers arent testing their code, this is evident at least every iteration, and often every day. If developers arent checking in and building their code frequently, this becomes visible every day. These problems stay visible until engineering practices are implemented to fix them. Management is on the hook to get them fixed as well as to devise strategies to remedy the problems until they are fixed. Agile processes ruin perfectly good management jobs, where managers sit in corner offices reading reports and presuming good progress until the project fails. Agile processes stick the bad news in the face of managers daily; the plausible deniability offered by most silver bullets is totally removed. The good news is that you can build great software predictably with agile processes. The bad news is that it is really hard.
Realizing Project Benefits Early

Any systems development process that provides for early realization of project benefits and maximized return on investment is valued. ThoughtWorks develops systems for its customers using Agile processes. In a recent study by Forrester Research, ThoughtWorks customers identified early realization of benefits as a primary reason why they were pleased with their relationship with ThoughtWorks (posted at ThoughtWorks web site at http://www.thoughtworks.com/forrester_tei.pdf). All Agile processes use iterative, incremental techniques. Software is developed during iterations that last from one week to four weeks. At the end of every iteration, an increment is delivered. This increment is a complete piece of the entire system and is potentially ready to implement, should the customer so choose. This means that every increment contains all aspects of the final product, including tested code, documentation, and user help. When the application calls for more incremental product, this is also included. For instance, FDA Life Critical applications must have requirements trace ability, demonstrating how the initial requirements are implemented in the finished product. This trace ability is included in every increment delivered at the end of every iteration. Having inspected an increment of the system at the end of an iteration, customers can choose to implement them before they had otherwise planned. TransCanada Pipelines (TCPL) in Calgary, Alberta chose to do so after just one iteration. The project was intended to automate title change feeds from all the provinces and states which TCPLs pipelines crossed. After the first iteration, the paper feed from Alberta was automated into an XML feed with a partial database and change management screens. Since over thirty-percent of all changes were from Alberta, when the customer saw this one feed working, she choose to implement it immediately. The additional cost of this early implementation and realization of benefits more than offset the cost of the implementation. Agile development processes create opportunities for customers. They can implement one or more increments of functionality at any time. They can also make other investment choices such as increasing or canceling funding of the project. When they inspect what the team has developed at the end of every iteration, they have all the information they need to justify such decisions. If good engineering practices have been used to build each increment of functionality, the cost of implementing it is relatively small. If marginal engineering practices have been employed, all defects must be fixed during the implementation cycle. Such increased implementation costs discourage customers from calling for implementations because of the increased cost of each implementation. Because of this, part of implementing an Agile process is improving the engineering practices of the development organization. We want the customer to be encouraged to call for early realization of benefits.
Some systems development processes, such as waterfall, defer the implementation of the system until preplanned milestones. All of the requirements are defined at the start of a project, and the project converts all of these requirements into functionality that is implemented at the end of the project. However, studies have shown that over 50% of all functionality developed using this approach is rarely or never used. This indicates a possibility of reducing costs of a project by 50% of more and still receiving the same benefits. This is an increased return on investment. Agile processes use a prioritized list of requirements to drive each iteration. At the start of the iteration, the customers and the team get together. The customers identify and describe the requirements that are of highest value to them. Highest value can mean that these, if implemented, would have the highest benefit to the business. Or, highest value can mean that these requirements are the riskiest and need to be resolved prior to incurring further project costs. In either case, the customers can stop funding as soon as they evaluate that the return on investment is maximized. This approach was initially described in the Dynamic Systems Development Methodology as receiving 80% of the benefits from 20% of the functionality.
Spinmeisters and Professionals

I conduct workshops at which people learn how to manage Agile projects. Because some of this knowledge is subtle, I use an exercise in which students, grouped into teams, collaborate with the commissioner of Major League Baseball, Bud Selig, to develop and implement a facility for selling and buying unwanted baseball game tickets. The name of the facility is MLBTix. The exercise is set up for Bud to have the problem, the team to have the technical expertise to solve the problem, and the two of them to talk about what can be done. Bud wants the MLBTix up by opening day, and he asks the team if they can do it. In the vast majority of cases, the teams tell Bud that they can have MLBTix up by opening day. Under questioning by Bud, they tell him that they are absolutely certain, there is no shadow of doubt, and that he has nothing to worry about. Agile processes depend on collaboration between development teams and customers. Since the job of systems development is very complex, both sides collaborate throughout the project. They talk about new things the customer needs, either because the customer sees new ways in which the system can deliver value or the business climate has changed. They talk about technical problems that are slowing progress, and alternative approaches that can salvage the date. They talk about interpersonal dynamics that are hindering project progress and how to resolve them. Throughout the project, the team and the customer are partners in optimizing the success of the project. However, partnerships are based on trust and trust flourishes in trust and honesty. I recently had surgery on my right eye. Prior to the surgery, the physicians at Massachusetts Ear and Eye discussed every aspect of the surgery with me, including all possible complications and adverse outcomes. I had to sign a form indicating that this information had been disclosed to me. Then, just prior to surgery, I had to use a pen to make a mark above my right eye to indicate which eye I expected to be fixed. The entire process was intended to make all pertinent facts regarding the operation visible; nothing was hidden, everything was in the open. Fortunately, this great institution pulled off the operation without a hitch. However, if they hadnt succeeded, I might be disappointed but not furious. I knew the possibilities, we had discussed them, and one of them occurred not the one I might have desired, but one that was within the realm of the known. During the workshop, I ask the teams why they absolutely assured me that they would successfully deliver MLBTix on time. I ask them if they have doubts. I ask them how they can be sure without having performed some type of scalability testing. We have to be positive, we want the customer to select us, why should we scare the customer off? These and similar phrases indicate that the teams feel different than the assurances they provide to Bud. To drive home the lesson, circumstances are described in a later exercise that cause them to fail to deliver the system on time. Yet, on questioning the teams, they rarely feel remorse for having misled Bud, for having not informed Bud of all possible outcomes.
Why do teams act this way? Why does our entire profession act this way? Can we call ourselves a profession if by comparison to the professionals at Massachusetts Eye and Ear we are only spinmeisters? Over the last twenty-five years, weve been subjected to enormous and contradictory forces. On one hand, systems development has become more and more difficult and complex. On the other hand, we rarely have a good idea of whether we will succeed or fail in meeting a time and cost until near the end of the project. Further complicating the situation, we are under pressure to clean up our act and do better. Doing better is often interpreted as knowing what we are doing, being able to predict a firm cost and delivery date. These request or demands are usually accompanied by such phrases as, dont you know what you are doing. We usually dont know what we are doing. Given the complexity of our projects, the uncertainty of the technology, and the changes in the requirements it truly is impossible to estimate the cost and delivery date at the start of a project. But, in the face of demands that we do so, we act as though we can. Hence, such phrases as absolutely certain even though we arent absolutely certain. Into this mix come Agile processes which are complexity tolerant. Customers and teams work together to ascertain what can be done rather than holding each other accountable for the impossible. But, in my experience, the habits of the past are hard to break. We are used to pretending we can be certain in the face of uncertainty, and being open and professional about our uncertainty is hard to do.
Only Eat When Hungry

Agile software development - only eat that for which you hunger, only maintain that which you need. When I go to the window of a fast-food restaurant, I evaluate what I want to eat in light of how much money is in my pocket. At restaurants, I usually spend whatever what I want costs, because payment is flexible through the use of a credit card. By fast-food is still cash-only, limited by what is in my pockets. In traditional systems development, customers identify what they want - the requirements of the system - and are told what the cost will be and the date on which the system will be delivered. For fast-food, this is analogous to driving up to the window, ordering, and then being told to pull over and wait until a specified time when we could get our food; during that time we could figure out how to get the money to pay for the estimated cost. Imagine buying systems functionality for a variable cost. Agile processes let the customer state the functionality that they want and how much money they want to spend. The functionality is delivered to the customer at the end of every iteration, during which the team cooks up a way of delivering the functionality. The customer looks at what was delivered and decides whether he or she is satisfied. If they want the functionality in more depth, they can order more stuff built into the functionality in further iterations. If the functionality is pretty complicated (like a sour-dough bacon burger cooked medium well, but the bacon is well done, and the roll is toasted) the functionality may take several iterations before the first digestible portion is ready. However, we still let the customer inspect the "food in progress" to maximize the likelihood that it will be what they hunger for. Traditionally, we list all of the requirements for customer functionality and deliver all of it. This is like a fixed price dinner, where we get all of the food even if we are full and sated after only the appetizer. Agile processes let us state the desired functionality (we are hungry) and then order requirements "a la carte", one at a time, until we are satisfied. Since the requirements can be prioritized, teams can iteratively deliver only increments of the requirements that are most appetizing throughout a project. Since we are sating the customer by delivering increments of functionality, the customer can dictate when they are sated, or when they have spent all that they want, and then consume the functionality as delivered. They eat what they hunger for, no more, no less. This simple analogy of systems development to dining works not only at the consumption level, but at the maintenance level. If we eat in all you can eat restaurants, we get fat, have to buy new clothes, and our health suffers. If we consume fixed requirements systems, we have to maintain and enhance all of the functionality, even the stuff that we infrequently or never use.
Yowzah!!!
I recently helped two divisions in two leading software companies implement Scrum for mission-critical projects. One project had three months left until it was due, and the second project wasnt due until the first quarter of 2005. In both cases, I met with the division vice president first. Each had already experienced Scrum elsewhere in their company and were optimistic that it would help them realize their objectives hitting the date with a competitive product. When I arrived at each software company, I spent time with directors, program managers, and project managers prior to beginning training. In each case, I felt that they were less optimistic about the project than their vice presidents. So, as an experiment, I had all the software managers in each of the companies get together. I asked them to close their eyes; I then asked those who thought the product would be released on time to raise his or her hands. In each case, the vice president was the only person to raise his hand. I relayed the information to everyone. In one case, the vice president ignored the information and plowed on. In the other case, the vice president spent the next two days working with the managers to figure out what could be done to improve the odds. What is it in software planning where the only person who believes the plan is the person who has committed it to management. Why is it that the commitment doesnt come from bottom up, so the vice presidents (in these cases) are speaking based on everyones beliefs and commitments? In both of these cases, the products were mission critical and the dates had been set by management. However, in an Agile process, the degree of functionality to be delivered by that date could have been established by everyone. Equally, everyone could have shared the tough news of the date and collectively tried to figure out how to make it happen, perhaps without ruining their personal lives and the quality of the product. Last I heard, one of the two companies was well on their way to figuring out, collectively, how to meet the date. The other company was in a death march, with managers and developers floating their resumes while the vice president resolutely plowed ahead while figuring out his alibi and excuses.
For Customers Only

Have any of your software development projects surprised you, either because they failed utterly, didnt come in on time, were of low quality, or took longer to deliver than you expected. You may want to take comfort in the knowledge that you werent singled out, that everyone else who initiates and funds software development projects has it better. Most of you share a common experience. In the political arena, you would have been spun. In Las Vegas, you would have been betting against the odds. In crime, you would have been conned. In a relationship, you would have been cheated on. If you have teen-agers, you would have been lied to. Underlying it all is the thread that your software development project team worked, at both a conscious and unconscious level, to keep you in the dark, out of the loop, and deceived. I run a class that teaches project managers to manage projects using Agile processes. Agile software development requires you, the customer, to actively collaborate with the development teams to optimize the value you get from your investment, to get the functionality that you need to meet your objectives. At this course, there are a number of exercises to explore how Agile project managers will facilitate this happening. In the exercises, a difficult project is initiated. There are many risks in the technology as well as difficult choices to be made in how to support the business goals with the technology. The point of the exercises is to create a scenario where the development teams actively collaborate with you to help you minimize your risk while maximizing your value. Many people in these courses have excelled. However, a disturbing number of these project managers are unable to help you understand your risks and alternatives. Not because they arent aware of them. Not because they dont know that they project may not succeed or meet your expectations. But, because they are afraid to tell you the truth. Even while fully understanding these risk, these people will tell you that they are absolutely confident that they can deliver the project on time with what you need. Words like certainly, positively, no-problem, slam-dunk, and without a doubt slip from their lips even though their minds and experience tells them otherwise. When I ask them why they mislead you and dont share their opinions with you, these people that you will entrust your success and money to say that they dont want to discourage you, that they want to put a positive spin on things, and that you wouldnt work with them if they werent positive. They tell me that you are so dumb that you would select someone who tells you no problem if they raised the specter of risk and doubt.
I ask these people how they would feel if they were treated the same when buying something themselves. Perhaps they enter a restaurant, a very expensive restaurant, or order a steak. The waiter and the chef know that the beef is old and that it comes from a herd where mad-cow disease has been spotted. Yet, they figure that what you dont know wont hurt you , that your actual chance of getting ill is pretty low, and they probably will be elsewhere if you do become ill. All of them tell me that they would be furious! I ask them where they get the nerve for assessing your risks for you and gambling your money in the face of uncertainty. What I hear back is a combination of fear, uncertainty, and bad-habbits. Except for the newest project managers, the software development profession is coming out of a period of twenty years when it was at least difficult and many times impossible to tell you if your project would succeed. You werent being lied to; the project managers juist didnt know. Worse, because of the process used to manage systems development, they didnt have any way to determine if your project would be successful or not until well into the project and your money. They lied to cover up the appalling trust that in the face of your probability of success only a desperate person would fund a project. This has led to a state where many venture capitalists and enterprises are turning to offshore development. These peers of yours have told me a number of times that they are doing this not to reduce their cost of a successful project, but to limit their losses. If the project is going to fail anyway, better to lose $500k than $1,500k. Agile processes provide an opportunity to turn around this unfortunate situation. Month by month all of the information is available for you to maximize your return on investment and to optimize your risk strategies. But, only if they arent hidden from you. Although alow and painful, and in the face of a history of hiding the trust, we are developing project managers that are confident of what they can and cant do with your project. Look for them. Dont look for the person that tells you what you want to hear, even though you know it is impossible, that your project is no problem and that the project manager is absolutely confident. Would you buy a used car from such a person?
Nothings For Free

Agile development yields productivity gains. Teams of developers, managing their own work in collocated spaces, outperform externally managed developers sitting in cubicles or offices. These teams are at least twice as productive. This productivity is coupled with other benefits, such as more creativity in deriving solutions to problems, higher quality by cross-checking each others work, and better morale through the socialization. All of these benefits have seemed to be free! Put the teams together, get their commitment to a goal, and have them work within a time-box. What has been lost by taking the team members out of their individual offices? What has been lost asking them to manage themselves as a team? Conflict! Unsurprisingly, it turns out collocated teams working under commitments and deadlines often have different values and opinions about the best way to accomplish their work. They sometimes come to work in a less than collaborative mood, resembling Attila the Hun rather than Tom the adult who happens to be an programmer. Conflict is inevitable. However, nothing in the Agile vocabulary says what to do when it happens. To date, weve all been willing to take the benefits without anticipating this cost. No wonder, for who likes to deal with conflict? I was at a client recently when a conflict occurred. An analyst was upset that a programmer had taken liberty with an agreed-upon design without consulting her. The programmer felt that the change was so minor that he was well within his rights by just going ahead with it. She felt that he was disrespectful of her work and responsibilities. As a result, voices were raised, nerves were frayed, and tears were spilled. Other members of the team tried to mediate, to soothe the conflict, without avail. The conflict ended in both of the participants fleeing the team room to a private space where they could get away from the other, lick their wounds, and emotionally recover. They were both torn between feeling they were right and being embarrassed that they had openly fought in front of the others. They both didnt know what to do. I am by no means trained in resolving conflict. When I entered the team room, everyone kept their eyes on their workstations. They were very upset by what had happened. I felt that we had to talk about this, so we gathered together and shared how the team mebers felt. They all felt embarrassed that they had witnessed the conflict. They also felt bad that they couldnt have helped those having the conflict to resolve it, or to avoid it. They were also agitated and worried that this would upset their family, their team, and they didnt know what to do to resolve the problem.
We decided that we couldnt ignore the conflict, that leaving the bad feelings unaddressed would create a poisonous atmosphere in what until then had been an easy and collegial atmosphere. Worse yet, we all acknowledged that if many more unresolved conflicts occurred, the team might not be able to work together. So we forged the following rules. 1) We accepted that conflict is a natural event when people work together; 2) When a conflict occurs, no work will continue until the conflict is resolved no sweeping conflict under the carpet; 3).The first thing to do to resolve the conflict is for everyone in the team to describe how they feel (the guys had a hard tie with this at first, mistaking analysis of the situation with feelings); and 4) Everyone would work together on a solution. We hypothesized that if no solution was forthcoming, it was probably because emotion was still clouding our thinking, so we would go through steps 3) and 4) in a loop until the solution was derived and everyone was able to report that they were feeling ok. Im certainly not a conflict resolution specialist, so I recommended to this organizations management that they bring in a specialist to work with each team. The specialist would teach the teams how to resolve conflict in a more effective way, one that could handle more types of conflict than they team and I had dealt with. When I implement Agile processes, I sometimes help the organization develop rules of conduct, or etiquette. Sometimes I do this because ill-will already exists and I dont want it exacerbated. Other times I help build the rules because I feel that the people are truly at a loss on how to get along; sometimes this is because of different cultural perspectives, sometimes it is simply because they have spent so much time working alone that they have forgotten, or never knew, how to work in teams. Some of the rules weve devised are: 1. Never use the work you because the other person may feel on the spot and defensive. 2. Never refer to history (three months ago, you said .!). 3. Be on time for meetings; if you are late, apologize and pay a late penalty. 4. If everyone is talking at once, use a pen to determine who talks. Whoever is holding the pen talk, everyone else listens. 5. Everyones opinion is important and needs to be understood and taken into account. 6. No name calling. 7. So forth, depending on the organization, team, situation, and past experience. In retrospect, Im surprised that I didnt foresee this cost. The team members had previously been kept in relatively conflict-free situations, with their management being responsible for resolving anything that came up. Now, this was up to the teams.
Bidding Work
We are often asked for estimates to build a system. Even though the system is complex, we are prodded, what will it take. And, to our regret, the estimate once out of our mouth becomes a contract. I had an experience recently where another professional showed me another way, and I learned that I was pleased with his approach. The exterior of my house needed painting. I called in three painting contractors, and I thought my experience with them may be of interest to the Agile community. The three contractors all came to my house, apprised it, and provided estimates. The high estimate was $15k, the medium estimate $12k, and the low estimate 7k. All were fixed price estimates good for thirty days. No estimate took more than one hour to prepare, and I walked the house and I answered any questions they had. I was therefore surprised at the fixed-priced bids, since I knew my houses exterior had some unique attributes that none of the contractors had encountered previously. My house is clad with DryVit, a highly insulating foam board contruction technique usually reserved for commercial buildings due to the skill needed to apply it The DryVit is then covered with a proprietary sealing polymer and then given a final color coat of acrylic paint. The paint application has to be carefully applied since it tends to soak in more than other paint. So I was perplexed and somewhat uneasy that these contractors thought they could fix-bid such a complex project; or, maybe they thought it was simple? Twenty days after the last bid was submitted, I was driving home on a limited-access highway. The speed limit was 55 mph. Suddenly, an immaculate, white panel truck passed me on the left, going at least 80 mph. As it disappeared, I was able to make out the name on the side, Noe Montenegro, Professional Painting. I was impressed. Here was a guy in a hurry who nevertheless cared about appearances. When I got home, I looked up his telephone number and asked him to come over the bid on the job. Noe was a young, intelligent man. When he came to the house, he spent time looking at the exterior before even ringing the doorbell. When I came outside, he asked very penetrating questions about the exterior, its construction and composition. I gave him all of the material and information I had, and he left. The next day he stopped by and told me that he had been doing some research. The research had led him to understand the type of acrylic paint required for my house, as well as the difficulties and complexities of applying it. Noe said that, even though he and his crew were great professional painters, they had no experience with this type of exterior and were uncomfortable submitting a fixed price bid. Noe said that if he tried to cover his uncertainty with a high bid, it might be too high. Similarly, if he made incorrect assumptions, he might underbid the work and have to take a loss.
After talking for awhile, we reached an arrangement. I would pay Noe and his crew $65 per hour plus materials for painting the front of the house. Then he would give me a fixed price bid for the rest of the house, based on his new knowledge and experience. I felt comfortable with this because if Noes price was too high or his competence too low, I was free to not use him after the front of the house was done. Also, I would have that increment of work done and could build on it with any other contractor. When complete, the time and material and fixed price remainder of the work cost me $8500, and was for excellent workmanship. Noe even cleaned the windows. I added an additional $1,000 to the check, as I was thoroughly impressed with his work as well as his honest approach to bidding on it. I told him that I was going to use this experience as a story; he just shrugged and said, Thanks.
Disenfranchised
As I help others learn, implement, and use Scrum, I run into a recurring problem. The people I work with are beset by authority that they feel is indifferent, impeding, or hostile to the successful use of Scrum, and the resultant development of successful systems and products. They feel that their management doesnt understand, care about, or support their efforts. Management (sometimes called senior executives) is inflexible and intolerant, continuing to make demands inimical to the successful and proper construction and deliver of systems. Marriage counselors have identified a theme common to people coming to therapy to improve their marriages. Each party is sure that their marriage will succeed if the other person - often called, the truculent rat will only change their behavior and ways. This attitude is the greatest impediment to martial bliss. So, a common these in both dysfunctional marriages and organizations is a problem can only be solved by the other party changing their stripes. Although this certainly simplifies the problem to the person identifying this course of action, it makes them helpless, powerless, and without an effective course of action. By stating that the problem can only be solved by changes in the other party, the person places themselves and their success completely at the mercy of the understanding and whims of the other party. They have lost control of and their ability to affect change and make progress. I discussed this problem at the recent Agile Business Conference in London with several other practitioners They immediately recognized it as a common psychological phenomena. They described it not only as a problem, but as an addiction. Once started, blaming others for lack of progress or success becomes a comfortable way of dealing with difficult situations and problems. The problem has been identified; maybe some effort has been made to communicate it to the other party. Now the person can sit back and wait for the other person to change. Sure, the problem doesnt improve, work is miserable, and the products are unsatisfactory, but we did our best. It is the other parties fault.
Ive pointed out to people caught in this form of thinking that they are dooming themselves to long term failure and impotence. Fortunately, there is a course of action a hard course of action, but a very possible course of action. Change yourself, change the way you behanve!! Take the initiative, figure out what can be change, and start changing. Sometimes this has no affect on the other party, but often changing your actions provides an education and opportunity for the other party. For instance, if we tell our customer that to deliver the additional functionality will require another month and miss the deadline, they often insist on the deadline and the functionality, regardless. Show them a burndown graph, show them that what they ask is as impossible as driving a 18 horsepower car 900 mph. Dont think that cutting quality, of the product and your life,will solve the problem. It will simply addict the customer to unreasonable requests that can only be met by us acting unprofessional. We will be condemned to building low quality product and our customers being unaware of the long term risk and costs of such reduced quality. We can overcome habits with time, but the reward had to be adequate. In this case the reward is quality products that are sustainable and maintainable, customers who trust and respect our advice (even though sometimes they may not like to hear it), restoration of the reputation of our profession, and pleasure and satisfaction in our work. Quite a reward, quite a lot of work. Go for it!
New Year Predictions

Three years ago, only the early adopters used Agile and those with systems projects that had failed and the companies would try anything. Over the last year, Agile has become an acceptable alternative to traditional methodologies and in-house processes. Some of the acceptance has come because of the word-of-mouth success stories spreading through the industry. At the same time, members of the Agile Alliance and Scrum Alliance have been working with development organizations, helping them understand how to make Agile work to optimize their development projects. For instance, over the last eighteen months, over one thousand three hundred project managers have been trained to manage Scrum projects, and are now called Certified ScrumMasters. In the case of Scrum, Ive worked with project managers, development managers, and leads to implement Scrum bottom up within their organizations. Ive also worked with senior management to implement Scrum top-down in their organizations. The latter has occurred either with visionary executives, or as executives have noticed something different happening in their organizations as bottom-up projects succeed. Either way, they wanted to capitalize on Agile. 2005 is around the corner and Im taking some time off to regroup, assess what has happened, and to plot out future directions. Below are some of my predictions based on my experiences over the last fifteen years, and based on where I think Agile will go, based on my experiences at over two-hundred organizations using Scrum and XP practices: 1. 25% of the people will be able to do the same work as now. Average productivity increase in organizations is 200%. This in itself is impressive in an industry that tries for 5% gains, and indicates the degree to which the previous processes were counterproductive. In addition, companies that Ive worked with have built more valuable software. By prioritizing their requirements based on value and risk, these companies have been able to achieve their objectives by often implementing less than 50% of the requirements that they previously implemented. Value driven development is possible using Agile, and these companies not only have significantly reduced the amount of work to achieve value, they no longer have to maintain and sustain the low value requirements. These productivity improvements will significantly increase over the years as the engineering practices and management control of ROI through value driven development becomes more sophisticated. The bottom line the same work that we do today can be done by less than 25% of the current work force. Of course, this may cause work to expand as the value of our work and the trust in our profession grows.
2. Only 1/3 of the companies that develop software will adopt Agile because it is very hard to implement. The degree of organizational change required will prove unacceptable to any organization that doesnt require on software and embedded software as the lifeblood of their companies. If these companies dont adopt software, their competition will take their business. Already, the successful and thorough implementation of Agile has mostly occurred in high technology and financial companies. 3. 2/3 of companies that develop software will outsource their work to companies that use Agile. The degree of change will required by Agile will prove unacceptable to these companies. However, the cost of the current failure rate will also prove unacceptable, and they will outsource their work to organizations that providing the quality of software at the cost enabled by Agile processes. Ive already seen companies that manage their offshore development using Scrum to frequently inspect the progress and to regularly take delivery of increments of functionality. 4. Business models for cooperative work will emerge and become dominant. A question that has plagued Agile since its inception is what about fixed price, fixed date contracts how does Agile handle them. The answer, that Agile handles them exactly as traditionally, with a heavy investment in up-front analysis and design to derive the price and date, works but soon is discarded by organizations using Agile. As trust between customers and developers grows, a new business model of collaboration has emerged, with teams of developers working on a time and materials basis as directed by the customer until customer has enough value. The hands-on control provided to customers by Agile is more than enough to create a trusted relationship similar to Japanese keiretsu relationships. 5. People will become more important than tools and processes. The experts have dominated the software development industry, providing methodologies and tools that were supposed to provide answers and silver bullets to the developers. Unfortunately, our industry is so complex that these methodologies and tools have often stifled as much as they have improved. Agile follows the lean manufacturing mantra, the people who best know how to do something are those doing it. These people will devise open source tools, or adapt commercial tool s to their need rather than being driven by their functionality. 6. People will enjoy and take pride in being software development professionals. This is the raison d'tre for many of us implementing and espousing Agile. We intend to turn restore our profession from one that is accused of lying and cutting corners, people whose manufacturing yield is unacceptable in other industries, into a modern profession that we can all enjoy and into which our children cant wait to enter. Having said this, I guess I have to do what I can to put my effort where my mouth is, all within the understanding and humility that this is an inevitable swing of the pendulum. Ken Schwaber

Scrum Musings

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Scrum Musings

Uploaded by

Copyright:

Available Formats

Scrum Musings

Copyright ADM 2004 All Rights Reserved Page 1of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 2of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 3of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 4of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 5of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 6of 49 12/26/2004

Managing the Process

Copyright ADM 2004 All Rights Reserved Page 8of 49 12/26/2004

Scaling Agile Processes Part 1

Copyright ADM 2004 All Rights Reserved Page 9of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 10of 49 12/26/2004

Scaling Agile Processes Part II

Copyright ADM 2004 All Rights Reserved Page 12of 49 12/26/2004

Scaling Agile Processes Part III Offshore Development

Copyright ADM 2004 All Rights Reserved Page 14of 49 12/26/2004

Scaling Agile Processes Part IV Too Large Teams

Copyright ADM 2004 All Rights Reserved Page 15of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 16of 49 12/26/2004

Scaling Agile Processes Part V Virtual Teams Instead of OffShore

Copyright ADM 2004 All Rights Reserved Page 17of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 18of 49 12/26/2004

Software Development A Science or an Art

Copyright ADM 2004 All Rights Reserved Page 19of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 20of 49 12/26/2004

Capability Maturity Model (CMM) and Scrum

Copyright ADM 2004 All Rights Reserved Page 21of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 22of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 23of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 24of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 25of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 26of 49 12/26/2004

Forming Cross Functional Teams

Copyright ADM 2004 All Rights Reserved Page 27of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 28of 49 12/26/2004

Cross-Functional Teams and Waterfall

Copyright ADM 2004 All Rights Reserved Page 30of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 31of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 32of 49 12/26/2004

Realizing Project Benefits Early

Copyright ADM 2004 All Rights Reserved Page 33of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 34of 49 12/26/2004

Spinmeisters and Professionals

Copyright ADM 2004 All Rights Reserved Page 35of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 36of 49 12/26/2004

Only Eat When Hungry

Copyright ADM 2004 All Rights Reserved Page 37of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 38of 49 12/26/2004

For Customers Only

Copyright ADM 2004 All Rights Reserved Page 39of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 40of 49 12/26/2004

Nothings For Free

Copyright ADM 2004 All Rights Reserved Page 41of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 42of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 43of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 44of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 45of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 46of 49 12/26/2004

New Year Predictions

Copyright ADM 2004 All Rights Reserved Page 47of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 48of 49 12/26/2004

Copyright ADM 2004 All Rights Reserved Page 49of 49 12/26/2004

You might also like