Agile Transition Strategy (Draft)

Ajit Alwe

Table of Contents
1. The Need - Why Agile....................................................................................................................... 4 1.1 1.2 Other Characteristic of Short Release Cycle time: ..................................................................... 4 Caveat ...................................................................................................................................... 4 Learning faster NOT working faster. .................................................................................. 4 Learning Is Not Restricted to Engineering.......................................................................... 5 Examples of actions that result in Longer Release Cycle Time ........................................... 5 Cycle Time Measure to surface the problem ..................................................................... 6

1.2.1 1.2.2 1.2.3 1.2.4 2.

What is Agile? How does it address the need of achieving Shorter Release Cycle Time? ................... 7 2.1 2.2 2.3 2.4 2.5 Phase Gate Process .................................................................................................................. 7 Iterative and Incremental ............................................................ Error! Bookmark not defined. Definition of Agile from Cycle Time Perspective ........................................................................ 8 Agile Practices that provides nested feedback loops ................................................................. 9 Fast feedback gives a sense of control. (Human Side of feedback) ...................................... 10

3. 4.

Using Release Cycle Time as an attribute to define Agile maturity of Product Teams ...................... 10 Practices that need to be put in place before Product team attempts Shorter Release Cycle. ......... 10 4.2 WIP limits - The magic the enable Shorter Cycle Time............................................................. 12

5 6

Implementations Steps for teams to adopt the above .................................................................... 15 Governance ................................................................................................................................... 16 6.2 6.3 6.4 6.5 The Role of the Agile Executive ............................................................................................... 16 Governing Rules as Maturity Increases ................................................................................... 17 Governing Rules for Agile Managers ....................................................................................... 18 Governing Rules for Agile Engineers ....................................................................................... 19

Moving towards Shorter Release Cycle Time .................................................................................. 20 7.2 7.3 7.4 Moving to Quarterly Release Cycle ......................................................................................... 20 Moving from Quarterly to Monthly Release Cycle................................................................... 21 Monthly to Weekly ................................................................................................................. 22

Advantages of Agile/Short Cycle Time ............................................................................................ 23 8.2 8.3 More Complete Learning Loops .............................................................................................. 23 Process focus reduces the time to detect and correct problems ............................................. 23

8.4 Business Strategy and Organization Are driven by what adds value to Revenue: Paying Customer ........................................................................................................................................... 24

8.5 8.6 8.7 8.8 8.9 8.10 8.11

Agile Continuously Improve the process that generates Result ............................................... 24 Drive Information Flow Across and Within Functions .............................................................. 24 Open Sharing of Information .................................................................................................. 24 Making Decision Locally in Real Time ...................................................................................... 25 Employee Are Appreciating Resources.................................................................................... 25 Empower People and Build Trust ............................................................................................ 25 Increased Sense of Accomplishment....................................................................................... 25

Fundamental progress has to do with the reinterpretation of basic ideas - Alfred North Whitehead

1. The Need - Why Agile

Business cycle can be thought of as a learning cycle. Each time a company defines, develops, and deploys (for customer use) a product, it completes the entire business cycle (Release Cycle Time) or Validated Learning Loop. The learning s gained are analyzed with the intent to improve strategy in the next business cycle. Therefore Shorter release cycles time shortens the time it takes
between an idea how to make money, and finally making money with that idea.

1.1 Other Characteristic of Short Release Cycle time:

y Short release cycle significant differentiator between product development organizations is the length of the learning Loop: the length of the software release cycle time. That is, shorter the release cycle time, more the learning Loops. Short release cycle time allows to use the ability to learn and change quickly as a competitive advantage. Short release cycle provides a Tough-to-Copy Organizational Capability. This needs brief explanation Illustration: A competitor may see that you are first to market and even understand the basics of short cycle time strategy but it cannot copy the process that enables you to get there. This highlights the fundamental difference between the results and process orientations. For example, one can buy a Hewlett-Packard laser printer, plug it in, and use it immediately. At the same time, one cannot buy their capability to develop that printer in record time. Developing Short release cycle time capability takes time; once you have it, it cannot be copied, even if others know you have it. In contrast, a single hot product may give a one-time boost to a firm s profits, but its advantage fades as competitors respond with their own.

y y

1.2 Caveat
1.2.1 Learning faster NOT working faster.

Having said that a sustainable shorter release cycle time can be achieved only by learning faster NOT by working faster. If we double the speed of an inefficient (covered in section 1.2.3 below) product development process (which includes Sales/Marketing, PM, Architecture, Dev, Test, PS), the inefficiencies will be doubled. Process/work design and governance metrics that enable learning has to be put in place before we target reducing of short cycle time. Companies with short cycle time revere learning as the only truly inexhaustible source of competitive advantage. The rate of knowledge production is limited by how fast the organization learns. Therefore, as one reduces release cycle time, the importance of learning becomes even greater. i.e.

learning of the customer/end user preferences, product design/architecture, configuration, deployment process , technical debt, defect density, team organization, bottle neck, information flow across various team etc.. 1.2.2 Learning Is Not Restricted to Engineering

Any CEO, COO, CFO reading this should immediately be alerted to the implications process improvement in engineering will necessarily drive process improvement throughout the whole business. The implications are not to be underestimated. Failure to improve the maturity of a product marketing group will produce poor quality data and poor quality input to the engineering organization. That data is used to make hard decisions about investment in engineering and the product being produced. Product Management is, therefore, the next big challenge for the high technology industry. One of the best ways to achieve Short Release Cycle Time is to build only what the customer needs, wants, and values and to not waste energy creating features the customer ignores. What this should also be telling CTOs, VPs of Engineering, and line managers within the engineering organization is that their success relies on the success and competency of their colleagues elsewhere in the business. They will need the support of senior executives who understand value chains and the necessity of having several departments cooperate and work as a team. Cross functional team working is essential to the success of a Short Release Cycle Time business model. In their enthusiasm, executive leadership slip into compressing time instead of redesigning core processes. Senior management must watch this at every step of the way. Project leaders have to shift beyond tightening schedules to fundamentally reconceptualizing the product development process as a learning laboratory. The primary obstacle to real-time learning is the perceived is the perceived risk. Discussing mistake when they happen is difficult. People are disappointed and sometimes angry; the degree to which one deems the environment safe either inhibits or spurs the discussion. 1.2.3 Examples of actions that result in Longer Release Cycle Time Example or Comment features or services the customer doesn t really want large engineering documents, more detailed designs than can be quickly implemented duplication of data for clarification, documents, approval, components, other groups to finish something giving a specification from an analyst to an engineer giving a Architecture Power Point/specification from an Architect to an engineer giving a UI mock-up from an UX to an engineer giving a component to another group for testing

Non-Value-Adding Action 1. Overproduction of solutions or features, or of elements ahead of the next step; duplication

2.

Waiting, delay

3.

Handoff

4.

Extra processing (includes extra processes), relearning, reinvention

forced conformance to centralized process checklists of quality tasks recreating something made designs documented but not built things built but not integrated or tested interruption multitasking on 3 projects partial allocation of a person to many projects testing and correction at-the-end to find and remove defects is not a value action; it may be a temporarily necessary people only working to single-speciality job title, or ? do people have the chance to change what they see is wasteful? information spread across many separate documents communication barriers such as walls between people, or people in multiple locations The estimate cannot increase; the effort estimate is what we want it to be, not what it is now proposed. We re behind schedule, but we ll make it up later.

5.

Partially done work, work in progress (WIP) or design in progress (DIP) Task switching, interrupt-based multitasking

6.

7.

Defects, testing and correction after iteration/release of the features

8. Under-realizing people s potential and varied skill, insight, ideas, suggestions

9. Knowledge and information scatter or loss

10. Wishful thinking (for example, that plans, estimates, and specifications are correct )

"Simple, clear purpose and principles give rise to complex and intelligent behavior. Complex rules and regulations give rise to simple and stupid behavior." ~ Dee Hock

1.2.4

Cycle Time Measure to surface the problem

Cycle time is easy to measure and use throughout the organization. If you define the end points of a process and have a watch/calendar, you can measure cycle time. Because the measurement is that easy and can be done within any function, team, or level in an organization, people can use a common language to discuss business issues. Cycle time also avoids many of the definitional issues that accompany measures such as quality. Whenever one tries to define what a defect is, particularly in upstream

knowledge work, theirs is potential for debate. Cycle time is much easier to understand and cycle time delays illuminates the bottle neck (mentioned in section 1.2.3).

2. What is Agile? How does it address the need of achieving Shorter Release Cycle Time?
2.1 Phase Gate Process to Overlapping Incremental Process

Waterfall to Incremental overlapping process

Waterfall

Time

Incremental overlapping process

Time

2.2

Definition of Agile from Cycle Time Perspective

Agile is set of values, principle and practices (practices are noted in table below) which help develop software in iterative and incremental manner using cross functional teams. This provides the ability to manage Business and Organization as an interdependent system using Cycle Time Measures. (Note: How Phase Gate approach increases the Release Cycle time and how to design overlapping process is explained in Process Traceability document) Agility throughout organization means focusing attention on understanding how the markets, Strategy and organization operate as a system of interdependent structures rather than as a collection of independent elements. By using Cycle time as a Central measure, Agile Organization highlights deficiencies that may be hidden between functions in a traditional way of working.. For example, let s say development reduces its development cycle time for a single feature/User Story from 2 to 1 week. If Tester still requires 2 weeks for regression of the same feature/User Story, the development may be

off the critical path, but overall Cycle Time doesn t change. The goal is to reduce total system cycle time rather than changing the critical path of tasks. Fast cycle time development is always limited by the slowest element in the development process. Using cycle time measures also provides a quick and painless way to identify and measure defects. To achieve speed, one cannot repeat tasks two, three, or more times. First-time yield is essential for speed. Thus, whenever cycle times grow, one can be confident there is probably rework occurring due to mistake or defects. By managing through cycle time, process defects within and between functions are surfaced earlier. Because cycle time can be used as a common measure for every organizational process regardless of function or level, it serves to integrate the organization. Senior management can monitor its decision cycle time, just as an engineer can monitor his cycle time on Code (Class methods in class). Using common measures such a cycle time and defects (as identified by cycle time delays) keeps the business management process simple. One only has to use two basic measures to ensure that work is done quickly and correctly. The table below shoes how Agile uses time- and scale-sensitive practices and dependencies, to provide an efficient feedback engine. Its nested feedback loops, each one optimized for the size of decision involved, don t just hurry feedback; they do so in very cost effective ways. Very small decisions, such as those made when writing statements and methods in a program, are very cheap to validate, so we feedback continuously, minute-by-minute, through interactions within programming pairs and through unit testing. Larger decisions, such as the selection of features to help solve a business problem and the best way to spend project budget, are quite costly to validate. Therefore projects needs to validate those decisions somewhat more slowly, through day-to-day interaction with customers, giving the customer control over each iteration s feature choice, and by providing a release (for production use, if desired) every few weeks at most.

2.3

Agile Practices that provides nested feedback loops

Feedback Loop Length ( also cycle time to Validate the
Artifact/Decision )

Software Artifacts /Decisions

Agile Practices

Solutions Priorities, Estimation Features Architecture

Months Weeks Weeks Days

Short Release, Minimal Marketable feature Relative Point Estimation, User Stories, Story Mapping, User Stories, Automated Acceptance Test, Specification by Example Collective ownership, SOLID principles. (Architecting for Testability, Architecture based on Use Cases and not dependent on technology) Emergent Design, TDD, BDD, Refactoring, CI, Mocking, SOLID principles SOLID principles, Clean Code

Design Classes and Interfaces

Hrs Minutes

Minutes, Seconds Statements and Methods Unit Testing, Clean Code, Pair programming

2.4

Fast feedback gives a sense of control. (Human Side of feedback)

The human brain makes attributions of causality when there is a short elapsed time between cause and effect. If we push a button and light goes on quickly, we assume that pushing the button made the light go on. If we push a button and it takes 5 seconds for the light to go on, we assume that the button did not turn on the light. Fast feedback causes people to perceive the cause and effect relationships in their actions. When people see that their action causes consequences, it changes their behavior. They feel a sense of control, and this causes them to take even more control of the system. Instead of being the victim of a vast monolithic system, they become aware of a subdomain where they have power and influence. Victims often can remain victims because they assume they have no control over outcomes. They have the steering wheel in their hands, but they don t turn it. When people begin to believe that the steering wheel they are holding can actually turn the car, they start steering the car. Thus, the human effects of fast feedback loops are regenerative. Fast feedback gives people a sense of control; they use it, see results, and this further reinforces their sense of control.

3. Using Release Cycle Time as an attribute to define Agile maturity of Product Teams
1. Quarterly Release Cycle Time 2. Monthly Release Cycle Time 3. Weekly Release Cycle Time Note: Release Cycle Time = Time required to Ship a set of features/work Items.
Work Item Cycle Time (Feature/User Story/Defect)= The time it take to complete single work item from Analysis to Acceptance stage.

To move shorter release Cycle Time, there are some practices product team needs to start doing, while stop doing others. These will be covered later in this document.

4. Practices that need to be put in place before Product team attempts Shorter Release Cycle.
Before we think about moving to shorter release cycle (quarterly, monthly, and weekly), FICO development teams must first put in place process that allows tracking requirements through the Release life cycle. It must be possible to track ideas through Analysis, Design, Coding, and Testing. This is a fundamental stepping stone towards manageability and profitability. Introducing end-to-end traceability in the software development lifecycle is a nontrivial first step. The manager (Dev, QA, PM) must achieve these basic step: 1. Analysis Ability: Decompose system input into basic units of measurement. It is vital that the product team gain a basic proficiency in the conceptual model of backlog,

2.

3. 4. 5.

Specification by example, Story Mapping. Need a basic grounding in UML domain modeling. Product Requirement Capturing - The Agile Way.docx provides way to organized decomposed information End-to-End Traceability: Implement system for capturing & monitoring measurements. Represents the first true steps on the road to an organization that shows ability to continuously improve and produce better and better financial results. In order to show those results, it is necessary to measure. (see Designing End-to-End Process Traceability.docx) Develop a Strong Configuration management capability Develop capability to deploy effectively Stabilize System Metrics: Inter relationship between these data points will help demonstrate basic statistical process control show that system is stable against a target and within a tolerance. Stabilizing system metrics is a demonstration that software development can be brought under control. It is demonstrating that it can deliver conformant quality. It is showing that it can meet its customer expectations within some acceptable tolerance. (see Designing End-to-End Process Traceability.docx)

The concept of measurement as uncertainty reduction and not necessarily the elimination of uncertainty. Uncertainty Reduction point of view is what is critical to business. Major decisions made under a state of uncertainty (of Information) can be made better, even if just slightly, by reducing uncertainty. Such an uncertainty reduction can be worth millions. Measurement Definition - a quantitatively expressed reduction of uncertainty based on one or more observations. *observations that quantitatively reduce uncertainty.

Observations of data points that predicts the Cycle Time of new feature.

Little s Law
Cycle Time =

Work in Progress (WIP) Throughput

WIP = no. of work items on which work has started but not yet finished. Throughput current rate of completion of features over
period of time. Cycle time How long will it take for new feature to complete. *This law holds good only when system is in statistical control. i.e. no. of people remain same and throughput is stabilized

* Little s Law used in Queuing theory the mathematical analysis of how stuff moves through a system with queues was developed to understand and improve throughput in telecommunication systems systems with lots of variability and randomness similar to software product development.

Cycle time is the time that a single work item (represented by a User Story, Feature, Prod. Defect, etc.. ) spends in a system in various work states (Analysis, Design, Code, Test, Regression, Acceptance) till it reaches the accepted state. We want to reduce Cycle Time of Work Item, so we can get more work done i.e. increase throughput. As you can tell from the equation, reducing WIP (work in progress) is one way of achieving that. Which means we can reduce Cycle Time without changing either demand (backlog item or Inflow of work) or Capacity (adding people or enhancing people s skills or adopting best practices of development) by just reducing WIP to an optimum limit.

4.2

WIP limits - The magic the enable Shorter Cycle Time

It is easier to start work than it is to finish it. It is easier to grow WIP than it is to reduce it.

Less is more We've all heard Less is More and have certainly experienced that. That's an earthy way of stating Little's law. We can get more done, when we are focused and working on fewer things. Why? We are not wired to work on multiple tasks at the same time. When we say we are multitasking, we are really switching our attention from one task to the other i.e switching contexts. Context switching is a waste. Studies have shown that with every additional task taken up there s a 20% loss. If we are working on three different tasks, we ve lost almost half the time in just context switching. Ok, we should limit WIP. But how do we determine the WIP limit? We experiment. If we set the limit too high, we will continue working on multiple tasks. If we set the limit too low, we may cause bottlenecks and affect the flow of work in the system. We want the WIP limit to be optimal so we can have a smooth flow. This needs to be empirically derived from the work flow data. There are advantages to starting with a lower WIP limit Yes, they ll cause starvation and pain. But lower limits will expose potential problems in the system. The industry uses the analogy of the WIP limit with lowering the waterline. Lowering the waterline exposes the rocks (i.e. bottlenecks and constraints). When the problems are exposed, the team can work to remove the bottlenecks and constraints until the work flows smoothly again. And so on and so forth. It creates slack When everybody is working all the time, there's not much time to introspect, retrospect, experiment,

innovate or even have lunch. As a rule of thumb, lead times go up when utilization crosses around eighty percent. High utilization is not good. We've probably experienced this when the highways begin to fill up with cars or when the CPU on the computer gets toward its capacity. How do we create slack? We could hire additional members on the team. Imagine going to the folks that handle the purse strings and telling them that we want ten developers in the team but we only want to keep eight developers working because we've heard that utilization at above eighty percent is not good. Good luck with that :) Or - we could apply WIP limits If we have a team of ten and we've put an overall WIP limit of 8, and assuming only one person works on an item at a time, we've taken away work from a couple of developers. In other words, we've intentionally created slack (or excess capacity). And slack is good because it will allow these two "idle" developers to do a few things such as pair up with other developers or help resolve issues at bottlenecks. And funnily enough, this will help get more work done. It's an enabler for a "Pull"system A pull system is one where the downstream process (Testing, Regression is downstream process, Requirement from client, Analysis are upstream process) pulls in work only when it's ready to process it. This will result in producing only what's needed and reduce work-in-progress WIP i.e. it will help result in a lean outcome and short cycle time. When we have WIP limits, we cannot take on work unless we have a "slot" available on the work station. We are consciously defining our capacity to take on work. Imagine if we did not have limits. Work would be "pushed" to us by the upstream process - whether we were ready or not. And most likely, it would just wait to be acted on. That's a waste. WIP limits make Process a "Pull" system. Pulling work through its entire lifecycle, causing the work to flow more smoothly and at a higher rate. Shines a light on the activities that are normally hidden. This visibility is at two levels; for individual activities, and also over the lifecycle as a whole.

It's an enabler for Kaizen (Continuous Improvement) It's not that we don't want to improve, but it's usually that we don't know where to start. Or what our bottlenecks/problems are. Lower work limits are a great enabler for continuous improvement (kaizen). When you run into the bottleneck, resist the temptation to immediately raise the WIP limit. Get a conversation started in the team and look to resolve the bottleneck. Pound the rocks to submission. And if you still have a bottleneck, raise the limit to enable flow. Trust the work flow data. The data does not lie.

Use a work-in-progress limited pull system as the core mechanism to: 1. Expose system operation (or process) problems and 2. Stimulate collaboration to improve the system. Metric the indicate process problems:

Elapsed time means how many days the feature took to cross the various state (Analysis, Design, Code, Test, Regression, Accepted i.e. is cycle time).

Touch time means how many days the feature was actually worked on (or touched ). In other words, it s how many days that card spent in work in progress columns, as opposed to queue/buffer columns.

This gives us very interesting data such as Hey, feature X was only two days of work but took twenty days to cross the board! That s only 10 percent process cycle efficiency!

Use metrics primarily to support process improvement, not to drive it.

The Healthy mix of leading and lagging Indicators: leading indicators predict future system behavior. y y Leading indicator : Work In Progress size, Queue* Lagging indicator: Throughput, Cycle Time

Queue = a small buffer between two processes to create the appearance of instant availability to the downstream process. Eg. Dev ready , Build Ready State in the diagram

5 Implementations Steps for teams to adopt the above

1. Put in place Configuration management capability. 2. Put in place capability to deploy effectively, QA should be able to deploy the build by only one input from development i.e. Tag name and deployment guide 3. Determine a set of unique types of customer valued work that flow through the organization, examples include bug, change request, or feature. 4. Organize the Feature information as described in the document - Product Requirement Capturing - The Agile Way.docx. Or the QC needs to have mapping between Requirement and Test cases, and requirement organized as end to end business scenario, which is critical prerequisite for prioritizing. 5. Size the work Items. a. For release planning, the Size Estimates needs to be computed at least on the information of Feature Acceptance Test (refer Backlog Meta Model in Product Requirement Capturing - The Agile Way.docx ) It is recommended that size unit of the work item to be bucketed into relative size for example Small, Medium, Large. Where small work item can be implemented end to end within one week, Medium between 1 to 2 weeks and Large more than 2 weeks. b. For planning cadence, Size Estimates needs to be computed based on Story Acceptance Test (refer Backlog Meta Model in Product Requirement Capturing - The Agile Way.docx ) It is recommended that if the work item size exceeds 10 days, it needs to be further broken down into multiple scenarios in the planning meeting. 6. Choose starting point from where you want to control system performance, i.e. between.
Customer Need or Idea, Marketing Req., Analysis, Design, develop, Test, Deployment - by PS team, Customer UAT. Identify what is upstream of that point and the upstream stakeholders. Define

some exit point beyond which you do not intend to control system performance. Identify what is downstream of that point and the downstream stakeholders.

7. Map End-to-End Process Traceability between start and end point identified in step 6 to Implement system for capturing & monitoring measurements. (see Designing End-to-End Process Traceability.docx) I. Visualize the work flow for different types of work. This will represent the underlying model of your system. II. Meet with team members, upstream and downstream stakeholders to discuss and agree on policies around WIP limits, prioritization meetings, and release mechanisms. III. Agree on the Input (Planning) and Output (Demo) and Retrospective cadence. IV. Use Spread Sheet/ Electronic Tool to visualize the workflow. This will cover the workflow that exists between the input and exit points defined in step 6. V. Create CFD to track and report the same. VI. Begin to visualize the flow through the system and study the system performance and capabilities. The purpose is to understand the system operation not affect performance or implement control. 8. Agree to have a standup meeting every day in front of the Electronic Tool or Spread Sheet, as a daily opportunity to drive improvements. Agree to have a regular operations review meeting for longer term retrospective analysis of process performance. 9. Educate the team on the new board, WIP limits, and pull system operation. (see Designing Endto-End Process Traceability.docx)

6 Governance
The essence of Agile software development is that it is highly delegated. Many Agilists prefer the term, "self-organizing." The point is that plans should be made at a high level and the desired adaptive behavior should be left alone to emerge from the system.

6.2

The Role of the Agile Executive

The role for the executive is, therefore, to establish the simple rules governing the system of software production. The executive must determine the rules, and then leave the system alone to adapt accordingly. If the desired adaptive behavior does not emerge, then the rules must be tweaked. Executives must resist the opportunity to interfere, control, or micromanage the process. Each layer of management must delegate responsibility for the activities beneath them to the staff doing the work. 6.2.1 Three levels of system control. 1. Executives: Select the system (or method) of production. Define the governing rules. o Use a work-in-progress limited pull system as the core mechanism to:  Shorten Release Cycle time
y Cycle time = WIP/Throughput

Stimulate collaboration to improve the system

Set WIP limit Process Cycle Efficiency = (Touch Time/Elapsed Time ) * 100

Expose system operation (or process) problems

y

Elapsed time means how many days the feature took to cross the various state (Analysis, Design, Code, Test, Regression, Accepted i.e. is cycle time).

Touch time means how many days the feature was actually worked on (or touched ). In other words, it s how many days that card spent in work in progress columns, as opposed to queue/buffer columns.

2. Managers: Set/adjust process tolerances. Adjust attributes of variable rules. 3. Engineers: Adjust performance by responding to feedback from system metrics.
6.2.2 Govern the System with Anticipated ROI

The answer to this is provided by the ROI metric. It is known that system rules must not encourage local efficiency or local throughput. Line managers must understand their place in the value chain and be focused on overall system production. However, managers must be remunerated based on ROI. Only ROI shows a more positive value for riskier projects assuming the riskier project has a higher intrinsic value. What is more, the incentive must be based on expected ROI, rather than actual ROI. If managers are measured based on actual ROI, they would be carrying all of the risk. This would make them risk averse. The desired result would not be achieved. The manager would still prefer the less risky project with the higher throughput rate. By using expected (or anticipated) ROI as the measure of incentive, it is possible to encourage managers to take the risks that the business needs to take. The business as a whole will carry the risk.

6.3

Governing Rules as Maturity Increases

Setting governing rules based on potential Throughput necessitates the ability to measure and track the value of Throughput throughout the system. The Agile manager must be capable of measuring the functional requirements at ideation and tracking them through to delivery. Measuring functional requirements allows the Work-in-Progress (WIP) of ideas captured within the software production system to be quantified. If WIP can be measured, the rate of transformation of WIP the Throughput can also be measured. If the WIP can be quantified at ideation, it is possible to measure how long it takes for a single functional requirement to pass through the system, that is, Cycle Time.

6.4

Governing Rules for Agile Managers

Managers needs be measured initially by their ability to implement end-to-end traceability. Managers must set the goal of implementing requirements tracking systems for the purpose of tracking the flow of value through the system. In order to do this: y Understand the different work item type and do functional decomposition of each work item. o Specification by Example and Backlog Meta-model to do base level of analysis. Depending on the skill set and maturity of the product team, the manager may first need to implement a skills gap training program on analysis techniques. The team must be able to identify what they are measuring.

y y

Once it is possible to measure Work in Progress (WIP), throughput, and cycle time, the focus must change to improving the throughput and reducing the cycle time and WIP by setting ever more difficult targets. This will focus the managers' thinking and require them to identify constraints and then exploit and elevate each in turn. The governing rules for managers must be based on the throughput of the whole system, not just the local part (i.e. feature complete, Test complete etc..) of the value chain for which a single manager is responsible. The effect of focusing the managers on the overall system production is to force all the managers in the value chain to work together as a team. If they are measured based on local throughput (local optimization) i.e. QA throughput, UX throughput, Architecture throughput, Dev throughput etc.. , the organization is likely to become dysfunctional. This resolves one of the fundamental problems in organizational hierarchies, that there is no such thing as a management team! In Peopleware, DeMarco and Lister observe that a group of line managers with a similar rank in an organization chart are not a team unless their goals and values are commonly aligned. Finally, when the system is working close to maximum capacity, the governing rules must change to measure the manager based on ROI. The manager must be allowed to let the throughput fall, if it produces a potentially greater ROI.
6.4.1 End-to-End Traceability

The next goal for the Agile manager involves achieving end-to-end traceability with the goal of demonstrating ability to measure WIP, Throughput Rate , and Cycle Time. The goal should be measurable based on the coverage of projects in the software production system and the compliance with data input for metrics tracking. It will be necessary to audit the figures in order to assess the manager's achievement. The long-term target is full traceability for all WIP in the software production system.

6.4.2

Continuous Improvement The Learning Organization

The manager should be asked to make improvements in the metrics. This will cause her to focus on identifying the system constraints and deciding how best to exploit each in turn, then on mechanisms for subordinating everything else to the decision, and later on making a business case for investment to elevate the constraint. Every time a constraint is elevated, the metrics should improve until a new constraint is encountered. Action should be taken to remove the new limiting factor, that is, elevate the next constraint. This may require actions you have not yet considered, . . . or choices in rewards and norms.

6.5

Governing Rules for Agile Engineers

Engineers must be encouraged to focus on exploiting themselves as a capacity constrained resource. This is referred to as "self organization" by many Agile methodologists. The concept is that the team members themselves organize to optimize their work. The business must set goals designed to create behavior that is in the best interests of the overall system. It may be unfair to expect every engineer to see and understand the big picture. It may not be necessary to ask engineers to use system thinking. They need not understand the Theory of Constraints. Instead, they must be told how management intends to measure them. Expect their behavior to modify accordingly. The correct metrics to exploit the engineer as a resource are Cycle Time and quality. Engineers should be asked to minimize their local cycle time rather than maximize their throughput rate. If they are rewarded for throughput rate and that rate begins to greatly exceed the rate of others in the value chain, they will create a downstream stockpile of WIP and may erode upstream buffers. Minimizing cycle time the time to perform their stage of transformation of the client-valued functionality shows that engineers can be responsive and Agile without building WIP downstream or depleting it upstream. Engineers should be encouraged to do high-quality work. Quality should not be sacrificed by reducing cycle time. It is desirable to minimize the quantity of WIP returning through the system. It is understood that rework reduces the overall system production. Poor quality will affect production rate, WIP levels, and cycle time. By asking the engineer to minimize cycle time and maximize quality, for example, to minimize bugs per client-valued function whilst also minimizing the time to create that client-valued function, the business is setting objectives for engineers that are aligned with the management objectives of improved production rate, minimized WIP, and short overall lead times. In turn, these objectives are aligned with the overall executive metrics .

7 Moving towards Shorter Release Cycle Time

1. Quarterly Release Cycle Time 2. Monthly Release Cycle Time 3. Weekly Release Cycle Time Note: Release Cycle Time = Time required to Ship a set of features To move shorter release Cycle Time, there are some practices product team needs to start doing, while stop doing others. These will be covered later in this document.

7.2

Moving to Quarterly Release Cycle

Start Doing Automated Acceptance tests based on Specification by Example technique Continuous integration Refactoring
Subscription

1. Automated Acceptance tests based on Specification by Example technique

The first problem encountered is that at a yearly or six month release cycle time, integration/Regression testing generally takes about two to three months. This makes quarterly releases quite a challenge. For starters, the bulk of the integration testing is going to have to be automated. However, in most cases code base is very difficult to test automatically, because it wasn t designed or written to be tested automatically. Specification by Example provides technique to think of automated tests as executable specifications that become living documentation. It provides guidance on both the benefits and the pitfalls of creating a well architected integration test harness.

Continuous integration
Once you have the beginnings of an automated integration test harness in place, you may as well start using it frequently, because its real value is to expose problems as soon as possible. But you will find that code needs to be done in order to be tested in this harness. Thus all teams contributing to the release would do well to work in 2-4 week iterations and bring their code to a state that can be checked by the integration test harness at the end of every iteration. Once you can reasonably begin early, frequent integration testing, you will greatly reduce final integration time, making quarterly releases practical. Refactoring: One cannot make all the design decisions up-front. In quarterly deployments there is not enough time any more for a two month design phase in the software development project. Hence practice of Refactoring provides more economical and better solution to mitigates the technical debt risk. Subscription Sales and support models are generally based on annual maintenance releases. Moving from annual to a quarterly release, support model will have to change for two reasons: 1) customers will not want to purchase a new release every quarter, and

2) you will not be able to support every single release over a long period of time. Consider quarterly private releases with a single annual public release, or may move to a subscription model for software support. In either case, it would be wise not to guarantee long term support for more than one release per year, or support will rapidly become very expensive.

7.3
Start

Moving from Quarterly to Monthly Release Cycle

Stop
Separate Q/A department Multiple deployed versions (dev, QA, cert) Design document Change requests Analysis team Build team

Developer testing Stand-up meetings Cards on a wall Pay per use

The organizational structure does not scale up to reach monthly releases cycles. This is where technical changes alone are not enough. Having a separate QA department might get a company out of the chaotic ways to work for longer release cycle (yearly or more than yearly). But a QA department brings in distance between the introduction of a defect, and it detection. With an organization not caring about quality, a separate QA department may help, but at the sacrifice of some overhead. When going to the shorter deployment cycle, a separate QA department though provides too much overhead. In regard to multiple installed versions, when moving from quarterly to monthly deployments, you cannot maintain multiple versions in parallel, as it costs too much overhead. You cannot expect to have ten different versions deployed, and get any new work done while maintaining these. The good news is that with monthly deployments no one has to wait more than a month for the new version of the software. If there is a defect in one version, the bug will be fixed within a month, and be shipped to the customer. For most customers this is an acceptable amount of time to wait. Instead of design documents, better team communication need to be designed. The big change request process is critical for an annual deployment. If you have a less rigid change process on a monthly cycle, you can get the changes in place within a single month. For the analysis and the separate build team, you cannot allow the organizational distance to take over on a monthly cycle. You still needs the skills in place, just as with testers, but you need shorter feedback loops. Stand-up meetings reduce the amount of meeting overhead. Instead of planning documents, cards on the wall provide a very low overhead planning technique. Since the plan gets updated multiple times per day, the overhead of re-planning needs to get as low as possible. Developer testing become crucial, the number of defects in a software has to be dramatically low for the shorter deployment cycle. When you are going to deploy monthly, you have to switch between design and implementation way more often. From a business standpoint, monthly releases tend to work best with software-as-a-service (SaaS). First of all, pushing monthly releases to users for them to install creates huge support headaches and takes

far too much time. Secondly, it is easy to instrument a service to see how useful any new feature might be, giving the development team immediate and valuable feedback. The pay-per-use model has a risk in the longer deployment cycle. If you make a mistake on the quarterly deployment cycle, you lose a lot of money with a pay-per-use business model. If you do the same with a monthly deployment cycle, the loss is less dramatic. Money is the best form of feedback. The person giving you the money, gives up something in order to get your service.

7.4
Start

Monthly to Weekly
Stop
Test team Up-front usability design One-way data migration Release branch Patches

Live, 2-way data migration Flow based approach Defect zero Temporary branches Keystoning

At a weekly or daily cadence, iterations become largely irrelevant, as does estimating and commitment. Instead, a flow approach is used; features flow from design to done without pause, and at the end of the day or week, everything that is ready to be deployed is pushed to production. This rapid deployment is supported by a great deal of automation and requires a great deal of discipline, and it is usually limited to internal or SaaS environments.

When moving to weekly deployments, more pressure is put onto the technical skills and the organizational structure. The organizational distance needs to be closer, and you can allow fewer defects in your software. Therefore you need a two-way data migration, in order to rollback data changes after a faulty deployment. He referred to a client who had a defect database with some of them being a year old. The amount of waste produced by managing the defects got into the way to go for the shorter deployment cycle. In order to get faster, you need to get rid of your defects. Beck claimed that teams deploying weekly need to have zero defects in their development process. He referred to zero defects as being the Inbox Zero of defect management. For weekly deployments, having a weekly planning cycle gets into your way. On the things you should stop with weekly deployments, he claimed that testers need to be embedded completely into the development team. The organizational distance of a test team is too large to keep up the pace. You also need to stop working with release branches, and patching the defects you found. Up-front usability design is also not possible within the time given for a weekly cycle. An other way of building new features in your software is to build them right into the mainline/trunk/master, but use the principle of *keystoning to only expose this new feature at the last moment when it is done.

*A keystone is the final top stone in an ark keeping both sides from collapsing. Once an ark is created it should be self supporting i.e. done.

8 Advantages of Agile/Short Cycle Time

8.2 More Complete Learning Loops
The business cycle can be thought of as a learning cycle. Each time a firm defines, develops, and introduces a product, it completes the entire business cycle or LEARNING LOOP. Firm with short cycle time learn faster than their competitors because they have more complete opportunities: Each time the total cycle is completed, one learns the consequences of decisions made earlier in the process. What appeared to have been good decision during the product specification phase may not pan out that way.

We are most alive when we are learning, creating and discovering.

Learning loops also occur on an individual level. A firm with an average 2-year development cycle will provide a young engineer with two complete development opportunities in 4 years, where as a company with 6 months development cycle time will provide 8 complete effort. By completing more cycles, the entire organization learns more about the development process and implements more incremental improvements. Learning and speed feed on each other and become a development breeder reactor. (A breeder reactor is a nuclear reactor that produces more fuel than it uses.)

8.3

Process focus reduces the time to detect and correct problems

Process orientation that illuminates root causes of problems earlier and therefore enables one to address them quickly. A central value of the process focus is that it reduces the time it takes for the organization and individuals to learn. The firm that learns faster than its competitors will be able to improve itself constantly and eventually to lead.

Time to Detect

Time to Correct

Time to Verify

Problem Introduced

Problem Identified

Fix Designed & Implemented

Fix Verified

The learning cycle has four major components: 1. Time to detect. Time from an error creation to its discovery

2. Time to correct. Time required to correct known errors 3. Time to verify. Time it takes to confirm that the correction has fixed the problem. 4. Time to err. The sum of above time. It is another way to think about total cycle time.

8.4

Business Strategy and Organization Are driven by what adds value to Revenue: Paying Customer

The business of business is to provide customers the maximum value for a price. Agile Organization define their business strategy and structure their organization based on the process that deliver value to their customers. We call these value delivery processes. They constantly work with their customers to hone and adjust their definition of value added as markets and technology change. As change occurs, they do not hesitate to adjust strategy or structure to be in alignment with the new definition.

8.5

Agile Continuously Improve the process that generates Result

Every result a business achieves is the output of a process. Most of management s time is spent comparing actual outputs to desired or expected results. When a mismatch occurs, focusing people s attentions on the mismatch itself may be motivating, but it does not provide much insight into what caused the mismatch. Short Release Cycle organizations treat results that do not meet expectations as symptoms of process errors. Specific to new product development, Short Release Cycle companies continually redefine the set of process at play in the development process, from overall strategy definition to product specification and final execution. What can we do to reduce the overall development cycle time and increase the time spent adding value? is asked regularly. Time is the common denominator and helps illuminate quality and cost issues. For example, multiple rework and testing cycles takes significant time, cost money and are symptomatic of a poorly

8.6

Drive Information Flow Across and Within Functions

Data are the raw material of product development. Development speed is paced by how fast one transforms data into

8.7

Open Sharing of Information

Speed and need to know mentality cannot co-exist. In a need-to-know organization, information is hoarded to the benefits of some and the detriment of many. The person without information cannot take responsibility for actions, whereas person with information cannot help but take responsibility. Information provides critical feedback, which unleashes learning and speed. Information sharing means more than publishing figures. Unless the context and meaning are clear, distributing information is wasted effort. Avoid information overload by defining the purpose behind the figures and only communicating the critical set of measures required, rather than everything they possibility can.

8.8

Making Decision Locally in Real Time Employee Are Appreciating Resources

It is literally impossible to be fast if people have to seek approval at every step along the way.

8.9

Requires speeding up the rate of learning within an organization, investments that support learning can pay significant dividends. People are the only company asset that an redesign processes for increased speed. If it is assumed that the half life of useful knowledge continues to erode, then the organization that does not develop its people will become technically obsolete.

8.10 Empower People and Build Trust

One cannot be fast without empowering people to take action based on their own judgment. Rather than preaching empowerment, Agile Organization demands speed, which in turn drives empowerment. This in turn eliminates the philosophical debate and focuses empowerment on making process move faster to achieve results quicker. Shorter Cycle time requires real-time, multifunctional input to speed products through development; people have more substantive contact with people from functional perspectives besides their own. Initially this is uncomfortable, but very quickly both parties find that their stereotypes of others are not completely true. Over time, the cross-functional interaction builds more trust and less defensiveness, which ultimately results in faster detection and correction of problems without blame. Increased trust and loyalty is a common outcome of working in short cycle environment. 8.11 Increased Sense of Accomplishment People develop an increased sense of accomplishment because their efforts reach a tangible conclusion that is visible to colleagues and friends at release. The decision to defer making decision is by no means cost free. Once identified, issues are pushed quickly to resolution. As cycle time drops, people get to work on more new products, which generates its own excitement. Short cycle time is honest and down to earth because there is simply not time for politics. Possibly the most important and least expected benefit of becoming an Short Cycle Time competitor is that the firm s deficiencies are rapidly and continuously exposed. Increasing development speed has the same impact on an organization that stress testing does on new product: it quickly illuminates problems that heretofore were buried under other problems. Short Release Cycle will not cure all of the company s problems, but it will do an excellent job of exposing them.

RT @cyetain: The most important product of #knowledge is #ignorance JW:How do you mean that? #learning what you don't know? Everything that needs to be said has already been said. But since no one was listening, everything must be said again. -- Andre Gide doing good engineering is not primarily making good decisions, it's seeking good feedback which lets you quickly discard bad decisions kent beck