Introduction to Agile Methods

Ned Horvath Massey University Prepared for New Zealand Police 20 Jul 07

Agenda
Recap Agile motivation & history SCRUM Some case studies and lessons learned Agile applied to business processes

You can never step in the same river twice - Buddhist proverb

Waterfall v. Agile
Predictive v. Adaptive Perfection v. Approximation Right once v. Continuous improvement Process artefacts v. Running system Centralized control v. Distributed control Delivery times measured in months or years v. weeks or months Discipline v. Discipline

Agile is for
Small teams (<40) Goal is on-time, on-budget, good quality Poorly-understood requirements Not safety-critical Experienced contributors

Cultural Requirements
Continuous communication between Suppliers and Users Sponsors who reward business value and value the product Delegation to front-line teams Negotiation permitted

Agile Manifesto
Customer satisfaction by rapid, continuous delivery of product Working product is delivered frequently (weeks rather than months) Working product is the principal measure of progress. Even late changes in requirements are welcomed. Close, daily, cooperation between customers and suppliers

Agile Manifesto ctd

Face-to-face conversation is the best form of communication. Projects are built around motivated individuals, who should be trusted Continuous attention to technical excellence and good design. Simplicity Self-organizing teams Regular adaptation to changing circumstances

Agile Practices
Frequent Customer-Supplier Communication Get something delivered, then refine Timebox
Fixed length 2-4 week development iterations 1-3 development interations per release iteration

YAGNI (You Aint Gonna Need It) / Just in Time Redesign to meet new/evolving requirements Continuous Integration & Testing
(Automated) Unit Tests part of deliverables (Automated) System integration tests updated every iteration (Every week? Every day?) Customer owns acceptance testing

Good, Bad, and ? (Customer perspective)

+ Receive value early and often + Change orders are OK + Can respond rapidly to changing conditions + Transparent development process + Stop when needs met No set end of project, so no set price ? You set and change development priorities ? Continuously engaged with supplier team

Good, Bad, and ? (Supplier perspective)

+ Customer resolves ambiguous requirements Customer can change & reprioritise requirements + Deliver value early and often + No schedule slips + No death marches ? You own the estimates ? Transparent development process No job security (except by delivering value) ? Customer owns the priorities ? Stay engaged with customers

Good, Bad, and ? (Management perspective)

+
Customers and Suppliers are accountable for success!

Agile Software Methodologies

[Rapid Application Development (RAD)] Dynamic Systems Development Model (DSDM) Extreme Programming Scrum Crystal Agile Unified Process (AUP) Lean

Scrum

Scrum Roles
Small (6-15) Team ScrumMaster (guide, provide cover, deal with non-technical obstacles. Not a manager) Product Owner (Customer or Marketing) Other Stakeholders: Sponsor, Press, Meta Team - larger teams comprise small scrum teams for components (up to 800 contributors claimed by Schwaber)

Scrum Activities
Project Planning Sprint (Iteration) Definition & Completion Daily Sprint Status Meeting

Scrum Project Planning

Define Scope Identify Stakeholders Secure Sponsor and Customer buy-in Identify Product Owner, Team & ScrumMaster Draft Requirements
Define Project Backlog

Draft Release Schedule

Sprint Definition & Completion

Sprint Planning Meeting Product Owner, ScrumMaster, Team; other Stakeholders welcome Establish/Agree Sprint Goal: deliver some explicit functionality Create sprint backlog: prioritized list of estimated, assigned tasks
One sentence; Product owner defines, team accepts Product owner will not expand/reprioritize during the sprint

Artefacts: Blocks list, burndown chart

Items from previous sprint backlog (e.g. bugs, incomplete features) New features from Product Backlog to be delivered (implemented and tested) New features for next sprint that need research (definition, spikes) Delivery Preparation actvities [Budget 75% of available time net of planned unavailability]

Sprint Review Meeting Review and demo accomplishments Update Release [and Product] Burndown Chart

Scrum - Daily Sprint Status

15 minutes (ideally: standing up) Team speaks, ScrumMaster facilitates, others listen only What have you done? What will you do? What obstacles are in your way?
Identify and assign - dont handle!

ScrumMaster records obstacles, updates blocks list and sprint burndown chart
see spreadsheet example

Sprint Retrospective
Team & ScrumMaster Every 2-4 iterations Identify chronic obstacles Identify process improvements [Define process improvement plan]

Spikes
A Spike is a bit of exploratory work, the aim of which is to understand enough to estimate a task, pick a technology, or assess a tool for fitness for a purpose.
Typically timeboxed to 8 hours or less The outcome is a recommendation or estimate Any code resulting from a spike is usually thrown away

Fail early and often - avoid analysis paralysis

Use Cases (Cockburn)

Product Planning: Scope
Exhaustively Identify Actors, Events, and Goals
Actor: verbObject Speaking as <role>, I want to

Ask Why? Capture the outermost summary use cases to see who really cares. Revise, repeat.

Product Planning: Releases

Group summary use cases into prioritized usable collections Verify draft release structure with stakeholders (expect to revisit this frequently!)

Use Cases ctd

Flesh out sea level use cases breadth-first
Capture stakeholders and interests, preconditions and guarantees Write the main success scenario (MSS) 6-9 steps, each: Show a goal succeeding Highlight the actor' intention, not the user interface s details. Have an actor pass information, validate a condition, or update state. Exhaustively list the extension conditions (exceptions) Write the extension-handling steps minimum guarantee

Review with customers

Use Cases + Scrum

YAGNI: Use Cases get fleshed out only as the Product Owner identifies them as candidates for a near-term Sprint Customer-Developer Communications: Customers can read and criticize (and even write) use cases Use cases UML Sequence Diagrams
Define (system) public interfaces Invent/revise internal components/classes, define public interfaces Updated UML class diagram - identify refactorings Design directly traceable to requirements

Use Cases + Scrum ctd

Use cases Use cases Use cases Use cases UI design System test design (black box) Acceptance test design Documentation & Training

Use case driven supports YAGNI Use cases identify functional requirements only

Case Study 1: Airline Ops

Customer: Southwest Airlines, 2001 Developer: CALEB Technologies Problem: Recover Crew after disruption Talented but inexperienced developers led by experienced leader (but no previous Scrum experience) Inexperienced QA team led by experienced leader Optimization experts (OR Ph.D.s) with no formal software engineering training

Airline Ops ctd

Initial use cases created by customer (1 yr)
Clear where understood, vague elsewhere Extremely vague around recovery strategies

Use cases supplemented with explicit problem scenarios (Acceptance Test Framework) > 20 stove pipe legacy systems, customer beginning to integrate with J2EE.

Airline Ops - Results

Managed to 80% of budget target (time and money) Close collaboration to define/refine Use Cases 3 week sprints Use cases drove UML design, system testing, UI design, user documentation Close collaboration with QA shortened ALL phases. First delivery to customers at 4 months Customer deliveries at 6 week intervals Delivered product was operational at 13 months (original budget 15 months)
Acceptance testing commenced immediately

Opt Lessons Learned

Customer-developed Use Cases provided a framework, hard questions required close interaction of subject matter experts Scrum spike procedure very effective at failing early and often Scrum daily meetings & short timeboxes were particularly valuable with inexperienced personnel Integrated Development/QA team kept cycles tight Scrum Master communication skills are decisive

Case Study 2: Password Sync

Customer/Developer: Sun Microsystems, 2002 Problem: single sign on across LDAP servers Talented, experienced developers, no previous Scrum experience except leader Intimate experience with one LDAP implementation

Sync ctd
Tight timeframe and budget (4 developers, 1 QA, one lead, 4 months) No formal scope, requirements Geographically distributed team, multiple demands on personnel Lots of pre-existing (but poorly understood) development process Already late

Sync - Issues
Scope ill-defined
Target LDAP implementations? Target substrate (JMS, DBMS)? Boil the ocean - initial deliverables included logging framework and other cool subprojects

Lead had little project management experience Management wants it yesterday Culture rewards innovative individuals

Sync Restart after 3 months!

Pulled customer (marketing), test, development, management into one room, defined scope Feasibility study (2 weeks) Use Case list, defined initial substrate Created road map buy-in (?) Instituted Scrum process (elevated Scrumexperienced lead) Instituted regular conversations with marketing

Sync - Results
First customer delivery 4 months after restart
Scope insufficient! Developed 2 interested outside customers

Second delivery 9 weeks after first Subsequent deliveries at 6 week intervals Operational at targeted customers at third release (8 months after restart, 11 months after initial start) Resources redirected after 5th release (11 months after restart)

Sync Lessons Learned

Scope is decisive Budget (even if you have to make one up) Beware customer surrogates Use Cases successfully defined the functional boundaries Non-functional requirements (list of supported systems and substrates) was the real challenge; resolved with spikes Scrum allowed a track record of failure to be transformed into a series of milestones that could be hit Morale improved once Scrum instituted

Agile at BT Exact (IT)

May 2004: 8k employees, 6k contractors, 24 timezones The Mandate: every project delivers value every 90 days Training a challenge The Agile Road Show - half-day events to promote awareness, held at BT Exact' principal UK offices. s
Agile Program Days - one-day "deep dive" events to provide more in-depth education on Agile delivery and to collaboratively explore how these approaches could be applied to the project and program in question. Agile Learning Projects - a series of project-level engagements were created to provide learning opportunities and success stories.
Ref: http://tinyurl.com/25zxvg

BT Exact ctd
Problem: Too Few Coaches
Pair Coaching Apprentice Coaching Agile Boot Camps

Results
ISO 9001 certification Jan 2006. Assessor: This was a fundamental change within the business. We were impressed by the scale of the transformation and how fast the organisation implemented the changes.

Agile for Business Processes

Agile is becoming a buzzword (and a trademark: BT Agile is CRM outsourcing) BTs rollout followed Agile principles:
People to people communication Training driven by problems from attendees

One size does NOT fit all (Cockburn: Crystal) Consistent with current management theory (Drucker, Peters) and Quality process [Ask Stephen about Telecom]

References
Cockburn, Alistair, Writing Effective Use Cases, Addison-Wesley 2000, ISBN 0-201-70225-8 Schwaber, Ken, Agile Software Development with Scrum, Prentice-Hall 2001, ISBN 0-130-67634-9 Boehm, B. & R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, AddisonWesley 2004. ISBN 0-321-18612-5. Ambler, Scott, Making the Unified Process Work in Practice, http://www.ambysoft.com/unifiedprocess/ Linda Rising and Norman S. Janoff, The Scrum Software Development Process for Small Teams, IEEE SOFTWARE July/August 2000

References ctd.
Cohn, Mike, Agile Estimating and Planning, Prentice-Hall 2005, ISBN 0-131-47941-5 M. Poppendieck and T. Poppendieck, Lean Software Development, Addison-Wesley, 2003, ISBN 0-321-15078-3 http://www.ControlChaos.com/ http://www.ScrumAlliance.org/ http://AgileAlliance.org http://AgileManifesto.org Other books by Cockburn, Schwaber

Q&A

Scrum

Scrum on One Page

Crystal is a family of methodologies.

Technologies change techniques. Cultures change norms. Distances change communication.
Choose a M ethodology according to (project size, systemcriticality, teampriorities)
(defects cause loss of...) . . . Prioritized for Legal Liability

Criticality

Life (L)
L6

Prioritized for Productivity & Toleran ce

L20

L40

L100

L200

L500

L1000

Essential money (E) E6

E20

E40

E100

E200

E500

E1000

Discretionary m oney D6 (D) Comfort (C)

C6

D20

D40

D100

D200

D500

D1000

C20

C40

C100

C200

C500

C1000

1- 6
Alistair Cockburn

- 20

- 40

- 100

- 200

- 500

- 1,000 Slide 11

Num of people involved +20% ber

Hum and Technology, Inc., 1998-9 ans

Not even conceivable to have a single, common methodology. Every project is slightly different and needs its own.
- Alistair Cockburn

Benefits from Implementing Agile

1. 2. 3.

Deliver benefits early (First Iteration is demonstrable) Avoid significant rework by only doing just-in-time detailed design Avoid dead-end design decisions by managing with LPM decisions and trade-off matrix Raise quality by moving testing forward in the process Become responsive by supporting scope adjustments every iteration Become reliable by instituting regular heartbeats to the team Increase estimating accuracy by working in small chunks Decrease risk by always having working software Increase throughput via real-time visibility Increase team moral by dropping the death marches.
- Ryan Martens, Rally SDC

4. 5.

6. 7. 8. 9. 10.

Four Paths to Great Software (Consistently Responsive)

Culture of Discipline

Hierarchical

High

Great Organization

Agile Organization
Path 1 Agility Agility & Innovation
Low

Waterfall
Path 3 Agility & Discipline

Agile Development

Path 4 Scaling & Extending Agile

High

Path 2 Discipline

Chaos
Bureaucratic
Low

Solo Virtuosos
Start-up
(Adapted from Collins Good-to-Great Matrix of Creative Discipline, 2002)

Evolution of Agile inside a team

1.

Rollover

Implement daily stand-up meetings, create an automated build process to always be close to shippable code

2.

Apply Agile Project Management (APM) practices

Create and track a Backlog of stories, a Release, and its Iterations through to Acceptance Create short iterations and release small batches

3.

Extend APM upstream into requirements elaboration

Capture use cases, scenarios for richer requirements statements, and build parent-child relationships to pull testing forward Build tight link to the customer for feedback

4.

Extend APM into test to increase quality measures

Track test case results to story cards, manage defects from iteration to iteration Implement an automated acceptance testing framework

Technical Team Disciplines

1. 2. 3. 4.

5.

6. 7.

Develop coding and naming standards Track remaining effort on tasks Automate the build process and share results Implement a Testing Framework that includes unit test, application tests and GUI tests Develop a continuous integration process and server Move toward collective code ownership Cross-train through pair programming to increase flexibility, domain knowledge and reduce risk

Resources to Self-teach Agile

Suggested Process
Internal champion Suggested reading Brown bag discussions Guest lectures Local Agile/XP user groups Agile Conferences
Agile Development XP / Agile Universe Software Development

Suggested Reading
Home of SCRUM Agile & Iterative Development

www.controlchaos.com/ www.craiglarman.com
Lean Software Development

www.poppendieck.com/
Agile Project Management

Where to start on the web

www.agilealliance.org
Articles, events & user groups Great resource for getting started with Agile

www.jimhighsmith.com/
Rally Whitepaper
Tactical Management of Iterative Development: Achieving Competitive Advantage, www.rallydev.com

Professional Help
Training Opportunities Lean Manager & Practitioners Classes
www.poppendieck.com/co urses.htm

Mentoring Programs Individual consulting:

Ken Schwaber Mary & Tom Poppendieck Jim Highsmith Mike Cohn Or see www.agilealliance.org

SCRUM Master Class

www.controlchaos.com/ce rtifiedscrum/

eXtreme Programming
www.xprogramming.com/ xpmag/services.htm

Rally Agile Pilot Program

1-day team workshop Developing the Agile Management Team Delivering the first Agile iterations

Rallys 1-day Agile Workshop

11 Ways for Agile Adoption to Fail

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

Ineffective use of the retrospective Inability to get everyone in the planning meetings Failure to pay attention to the infrastructure required Bad ScrumMasters Product Owner is Consistently Unavailable, or There are Too Many Owners Who Disagree Reverting to Form Obtaining Only "Checkbook Commitments" from Executive Management Teams Lacking Authority and Decision-Making Ability Not Having an Onsite Evangelist for Remote Locations A Culture that Does Not Support Learning Denial is Embraced Instead of the Brutal Truth
- Jean Tabaka, http://tinyurl.com/2og2f7