What is an Agile Tester?

Colombo Agile Conf, June 2014

Consultant
www.crisp.se Henrik Kniberg
henrik.kniberg@crisp.se Father
@HenrikKniberg

Agile & Lean coach

Author
 Agile tester mindset

00:22
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
Henrik Kniberg
Case study: Game development company
Design-ready games Production-ready games
Game backlog
15 12
8

Concept Resource Graphics Sound Test &

Dev
pres. planning design design deploy
1m 6m 1w 6m 6m
4h 1d 1m 3w 3m 3w
(1m+2m)
Actual work: 3 months
Time to market: 25 months
Before
Design-ready games Production-ready games
Game backlog
15 12
8

Concept Resource Graphics Sound Integr. &

Dev
pres. planning design design deploy
1m 6m 1w 6m 6m
4h 1d 1m 3w 3m 3w
(1m+2m)
Actual work: 3 months
Time to market: 25 months

After Game team

(graphics, sound, dev,
test, deploy, etc)
Cross-functional game teams

Time to market: 3-4 months

7 times Better Less More
faster games Planning fun
 Cross-functional teams
We’re slow!
I’m fast! 6 months

Joe Dave Lisa Release

3 months

We’re alot faster!

Joe
I’m a bit
Dave slower

Lisa Release

January February March April May June July

Henrik Kniberg
 Test phase Test team

Tester role

Henrik Kniberg
 Test phase

Req Code Test

Henrik Kniberg
 We own
Test team
quality

Programmers Testers

Henrik Kniberg
 I own
quality
Tester role

Tester

Dev team

Henrik Kniberg
 Role Competency
We own
quality

Cross-functional development team

QA = Quality Assurance
= Quality Assistance

Henrik Kniberg
 Roles

DB

GUI
Design

Backend Test
Henrik Kniberg
 Competencies

DB

Design
GUI

Backend Test
Henrik Kniberg
 Knows a bit about many things

Cross functional team Knows a lot

Doesn’t mean everyone has to know everything

about one
thing

I can do Java, but

Product I’m not so good at it. Skills Needed to implement Top X backlog items
Backlog
Java DB Web Test Domain CM I don’t know CM
I’m good at at all. But I’m
Lisa Test! willing to learn!

Joe
Fred
Jenny

David
Erik
I won’t even go
near a
database!

Henrik Kniberg
 ensure
How do you know that your product works?
Who are the
stakeholders?

1. Understand the What need do they have,

that we want to solve?
problem
How will we know
when we’ve solved
it?

How will we know if we’re

moving in the right direction?

2. Iterate until Minimize the distance to MVP

you’ve solved it
Deliver, measure, adjust
continuously
Henrik Kniberg
 Typical activities

00:22
Henrik Kniberg
 Make sure backlog items are testable & valuable

As a buyer
How to demo:
I want to save my shopping cart
1)  Enter store
so that I can continue shopping
2)  Put a book in shopping cart
later
3)  Press ”save cart”
4)  Leave store, and enter it again
5)  Check that the book is in my
cart

Henrik Kniberg
 Find defects early!

Cost of
defect

Age of defect
Henrik Kniberg
 Sit with the developers

Developers Developers & Testers

Meeting room
Testers

Henrik Kniberg
 Shorten the feedback loop
People
(# of handoffs)
2 3
1
Maker User
5

# of 4
3
handoffs
2
1 Time
(Feedback delay)
0

minutes hours days weeks months years

Length of feedback cycle

Henrik Kniberg
 Push for Continuous Delivery

Build Test & Deploy to Deploy

Automatic staging
integrate to prod

Manual Code & Manual

commit test

Henrik Kniberg
 Create a shared vocabulary Unit
Test?

Integration
Quality?
test?

Acceptance
test? Technical
debt?

Henrik Kniberg
Set working agreements for test automation
AT and UT complement each other
Each user story has at least •  Acceptance tests: coupled to UI, decoupled from internal design
one black-box acceptance test •  Unit tests: coupled to internal design, decoupled from UI

Complex code has

unit tests

We know which code isn’t covered

Almost all code is and have a good reason for it
covered by SOME kind
of test (AT or UT)

Henrik Kniberg
 Do & teach exploratory testing

!
?

Henrik Kniberg
 Done includes ”no added technical debt”

Backlog Ready In Ready for

for dev progress production
- No added tech debt

Henrik Kniberg
 Example: Test automation backlog
Step 1: Decide what needs to be tested
•  Change skin
•  Security alert
•  Transaction history 
•  Block account 
•  Add new user 
•  Sort query results  
•  Deposit cash  
•  Validate transfer  

Henrik Kniberg
 Step 2: Classify each test
Pay every
Pay once
time

Manual Test Automation

Test case Risk
Cost Cost

Change skin low 0.5 hrs high

Security alert high 1 hrs high

Transaction
history med 3 hrs low

Block account high 5 hrs low

Add new user low 0.5 hrs low

Sort query
results
med 2 hrs medium

Deposit cash high 1.5 hrs low

Validate high 3 hrs medium
transfer

Henrik Kniberg
 Step 3: Sort the list

Manual Test Automation

Test case Risk
Cost Cost
Automate first!
Block
account high 5 hrs low
Validate
transfer high 3 hrs medium
Transaction
history med 3 hrs low
Automate later
Sort query
results med 2 hrs medium

Deposit cash high 1.5 hrs low

Security
alert high 1 hr high
Add new Don’t bother
user low 0.5 hrs low
automating
Change skin low 0.5 hrs high

Henrik Kniberg
 Example: Tech backlog
Product backlog

Tech backlog

Henrik Kniberg
 Reserve X% of team capacity for the tech backlog
Product backlog

80% Sprint

Tech backlog

20%

Henrik Kniberg
 Example:
Spotify

00:22
Henrik Kniberg
 Like a magical music player in which
you’ve bought every song in the world!
Play Everywhere!

Henrik Kniberg
 >400 people in tech

Stockholm

Gothenburg

San Francisco
New York

36

Henrik Kniberg
 > 50 squads

Henrik Kniberg
Autonomous Squad
Cross-functional, co-located, self-organizing team

Henrik Kniberg
39
 Squads are grouped into Tribes

Tribe Tribe Tribe

Tribe Tribe Tribe

Henrik Kniberg
Each Tribe is a lightweight matrix focused on delivery
Vertical = Delivery.
Horizontal = knowledge sharing & personal development
Tribe Tribe

Chapter Chapter

Chapter
Guild Chapter
 Reality is messy

Henrik Kniberg
 Decoupling to enable
frequent releases

Client App squad

!#?

Feature squads

Henrik Kniberg
 Self-service model
Client App squads

Infrastructure squads Enable &

support

IOS Android Desktop Web

Enable & support

Feature squads

Enable &
support

Henrik Kniberg
 Release trains & Feature toggles

Henrik Kniberg
 Failure Recovery is more important
than Failure Avoidance

Failure Avoidance Failure Recovery

Henrik Kniberg
 “Limited Blast Radius” via decoupled architecture

Henrik Kniberg
 ”Limited Blast Radius” via gradual rollout

Henrik Kniberg
 Trust > Control
100% control = 0% motion
If everything’s under control,
you’re going too slow!

- Mario Andretti

Henrik Kniberg
 Idea/Problem
“Radio you
can save!”

Narrative &
Prototypes & “Follow your
favorite artist”
Metrics
A/B stats

Build MVP

Tweak Deploy

Analyze data
Henrik Kniberg
 Example:
Big Government Project

00:22
Henrik Kniberg
 Team structure - before
Requirements 3 Test
analyst Development team
team teams

! !
#% #%

!? !?

52

Henrik Kniberg
 Next 10 User
Ideas Features Development System
features acceptance Production
test
test

53

Henrik Kniberg
 Team Next 10 Dev Ready for Sys test
swimlanes features in progress sys test progress

54

Henrik Kniberg
Team Next 10 Dev Ready for
swimlanes features in progress sys test

Dev Team 1 Dev Team 2 Dev Team 3

Henrik Kniberg 55
 ”Daily cocktail party” 9:15 – 10:15

56

Henrik Kniberg
 10:00 – 10:15

Project sync

9:45 – 10:00 9:45 – 10:00

Requirements Dev sync

Test sync
sync

9:30 – 9:45 9:30 – 9:45 9:15 – 9:30

Feature team 1 Feature team 2 Feature team 3

57
Henrik Kniberg
 Example: Measuring velocity by counting cards
Count
cards

Velocity per week

58

Henrik Kniberg
Example: Release planning using a burnup chart
All of these
will be done

Total
# of Week
delivered Some of these
features will be done,
but not all

None of these
will be done 60 60

Henrik Kniberg
 ”Oh no, bottleneck
in System Test!

FLOW

61

Henrik Kniberg
 Tech stories
Next 10
features

Next 5 tech
stories

62

Henrik Kniberg
 Bottleneck

”Let’s stop
building new
features”

”... and focus on

test automation!”

63

Henrik Kniberg
 Everyone
doing tech
stories

64

Henrik Kniberg
 Top 3 recurring bugs

65

Henrik Kniberg
 Definition of
”ready for
development”

Definition of
”ready for
system test”

66

Henrik Kniberg
Henrik Kniberg 67 67

Henrik Kniberg
 Release
Before:
Test at end Test Fix %&@#!

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Now: Release
Test continuously

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Test Fix

Test at end:
%&@#!
Test continuously:

Test Fix Time

saved!

68

Henrik Kniberg
 Bug fixing process
Bug found!

Yes Write sticky-note,

Blocker? find developer,
fix now!
No

More important Yes Replace one of

than any of the the other
current top 30? top 30 bugs
with this one
No

Ignore
it

Don’t log it.

Fix it NOW!

69

Henrik Kniberg
 Three input queues

Next 10
features

Next 5 tech
stories

Next 5 lower
priority bugs

70

Henrik Kniberg
 Wrapup

00:22
Henrik Kniberg
 What is an Agile Tester?

•  An agile team member with testing expertise

•  Helps the team become quality-aware
•  ... and learn how to deliver better stuff

Henrik Kniberg
 Quality Quality
Mindset Assurance Assistance
Automatic
Manual test
test
Functional Exploratory
test test

Requirements Customer
needs
Late Early
involvement involvement
Long Short
feedback loop feedback loop

Find defects Prevent

Henrik Kniberg defects
 Agile is a direction, not a place

The important thing isn’t

how you work.

The important thing is

how you improve the way you work!

Henrik Kniberg