Software Testing

Software engineering
processes

Course objectives
Become familiar with the software
engineering processes
Have an image of the main software
life cycles
Place the testing processes into the
right place
Describe few classes of software
projects and their specifics

References
I. Sommerwille, Software Engineering, 8th
edition, Addison Wesley, 2006 (chapter 4)
G. Everett, R. McLeod: Software Testing. Testing
Across the Entire Software Development Life
Cycle, Wiley&Sons ,2007 (chapter 2)
C. Kaner, J. Falk, H.Q. Nguyen: Testing computer
software, Second edition, Wiley & Sons, 1999
(chapter 3)
K. Benk, C. Andress: Extreme Programming
Explained. Embrace Change. 2nd edition,
Addison-Wesley Professional, 2004
S. McConnell, Rapid Development:Taming Wild
Software Schedules, Microsoft Press,1996

Software projects
There are a potpourri of software projects
No single structure or process that
optimally applies to the requirements and
environments for all sorts of projects
The discipline and the art of managing
projects makes the project success
What makes the software projects
different?

Software Project Management

Software Engineering
What is software engineering?
is an engineering discipline that is concerned
with all aspects of software production from
the early stages of system specification to
maintaining the system after it has gone into
use.[Sommerville, Software engineering, 8th
edition, 2007]
is the application of a systematic, disciplined,
quantifiable approach to the development,
operation, and maintenance of software[IEEE,
"Guide to the Software Engineering Body of
Knowledge" (February 6, 2004)]
Software Project Management

Typical Phases in Software

Requirement analysis and Specification

Design
Implementation
Testing
Maintenance
Life cycle:
Establishes the order these activities are
performed
Establishes the criteria for moving from one
phase to the other
Software Project Management

Software specification
The process of establishing what
services are required and the
constraints on the systems operation
and development.
Requirements engineering process
Feasibility study;
Requirements elicitation and analysis;
Requirements specification;
Requirements validation.
Software Project Management

Requirements Validation
Showing that the requirements define what the
customer wants
Checks:

Validity checks compared to stakeholder needs

Consistency checks related to possible conflicts
Completeness checks related to the coverage
Realism checks ensuring that they could be
implemented
Verifiability proved by the potential of writing
acceptance tests

Techniques:
Reviews
Prototyping
Test case generation
Software Project Management

Software design
The process of converting the system
specification into an executable
system.
Software design
Describes software structure that realises
the specification;
Describes data which is part of the system
Describes interfaces between the system
components
Describes the algorithms
Software Project Management

Design process activities

Chapter 4,10,11,12 Sommerville
Architectural design: sub-systems identified and
documented
Abstract specification: abstract specification of
the services and services constraints per sub
system
Interface design: Unambiguously describes how
the subsystems interact
Component design: allocation of services per
components
Data structure design: data structures used in
implementation being detailed
Algorithm design: algorithms being detailed
Software Project Management

10

Programming and
debugging
Translating a design into a program and
removing errors from that program.
Programming is a personal activity there is no generic programming
process.
Programmers carry out some program
testing to discover faults in the
program and remove these faults in the
debugging process.
Software Project Management

11

Software testing
Verification and validation (V & V) is intended to
show that a system conforms to its specification
and meets the requirements of the system
customer.
Involves checking and review processes,
system testing and acceptance testing.
System testing involves executing the
system with test cases that are derived
from the specification of the real data to
be processed by the system.
Acceptance testing involves executing
tests intended to asses the way the
customer requirements were implemented.
Software Project Management

12

Software evolution
Software is inherently flexible and can
change.
As requirements change through changing
business circumstances, the software that
supports the business must also evolve and
change.
Although there has been a demarcation
between development and evolution
(maintenance) this is increasingly irrelevant
as fewer and fewer systems are completely
new.
Software Project Management

13

Life cycle choices

Life-cycle: a particular way of performing
activities related to different software
projects phases
Main idea: keep the focus to the goal
Possible goals:

Improve development speed

Improve quality
Improve project tracking and control
Improve client relations
Minimize overhead
Minimize risks
Support software evolution
Software Project Management

14

Waterfall Project Phases

Software Project Management

15

Waterfall
First applied software model
Basis for many other models
Orderly progress from initial concept
to system testing and maintenance
Review at each phase end. Return to
previous phase if no acceptance.
Document driven
Software Project Management

16

Waterfall
Phases are discontinuous: do not
overlap
Suitable for stable product definitions
and technologies
Find errors at early stages
Minimizes planning overhead
Suitable even for un-experienced
teams
Software Project Management

17

Waterfall Drawbacks
Risks not explicitly addressed
The product is not visible until final stages
The stability of requirements is not a valid
assumption in software development
Discovering missing features at the end of
the cycle is very expensive
Lack of flexibility
Costly
Longer schedules
Software Project Management

18

Waterfall testing
Testing at the end of the lifecycle
No interaction between testing activities
and the development activities
Consequence: keep the testing and
development departments separate
Software testing is considered as being a
destructive process
Programmers should not test their own
programs
Programming organizations should not
test their own programs.

V-Model

20

V-Model testing
Start testing activities early in the lifecycle
Clearly defines four kind of testing
activities and their chaining:
Unit testing
Integration testing
System testing
Acceptance testing

Testing plans affects the development

plans

Risk reduction: moving to

iterations

*Reproduced from the IBM testing

course

Spiral models

Spiral
Boehm 1988
(http://www.computer.org/portal/cms_docs_computer/co
mputer/homepage/misc/Boehm/r5061.pdf)
First major iterative model: planning spread across
iterations
Enhance waterfall by early risk identifications
Process is represented as a spiral rather than as a
sequence of activities with backtracking.
Each loop in the spiral represents a phase in the process.
Each spiral iteration consists in a major risk elimination.
No fixed phases such as specification or design - loops in
the spiral are chosen depending on what is required.
The last spiral iteration may be a waterfall cycle

Software Project Management

24

Spiral
The result of each iteration are more refined
prototypes until the released version
Advantages:
Early addressing of risks: as costs increases the
risk decreases
The management control is at least as much as
in the waterfall case, improved by the existence
of checkpoints at the end of each spiral
iteration

Disadvantages
Rather complicated to manage
Can be difficult to define milestones to indicate
if you are ready to move to the next iteration
Software Project Management

25

RUP phase model

Phaseiteration

Inception

Elaboration

Construction

Software Project Management

Transition

27

RUP phases
Inception - Define the project scope, gain
agreement on project objectives, baseline
the product Vision
Elaboration - Address key technical risks,
produce an evolutionary prototype,
baseline the Architecture
Construction - Iteratively and
incrementally develop an operationaly
complete product
Transition - Deliver the product into the
live end-user environment
Software Project Management

28

RUP good practice for business

driven development
Adapt The Process More processes is not
necessarily better
Balance Competing Stakeholder Priorities Manage often conflicting requirements
Collaborate Across Teams - Proper team
organization and the setting up of effective
collaborative environments.
Demonstrate Value Iteratively - Develop
software iteratively
Elevate Level Of Abstraction - Use
component-based architectures

Focus Continuously On Quality Verify software quality

Software Project Management

29

Practices and Anti-patterns

Practices:
Ensure team ownership of quality for the product.
Test early and continuously in step with
integration of demonstrable capabilities.
Incrementally build test automation.

Anti-patterns (compare this with the V-model)

To peer-review all artifacts and complete all unit
testing before integration testing.
To conduct in-depth peer-review of all
intermediate artifacts, which is counter
productive because it delays application testing
and hence identification of major issues.

RUP disciplines

Business Modeling: business processes modeled as business cases

Requirements: identification of actors and the development of system
use cases
Analysis & Design: a design model is created using architectural
models, component models, object models and sequence models
Implementation: The components are implemented and structured
into sub-systems

Test: iterative process carried out in conjunction with the

implementation. System testing is performed after
implementation

Deployment: a release is created, distributed and installed on users

systems
Configuration & Change Management: controls change to, and
maintains the integrity of, a projects artifacts.
Project Management: Provide a framework for managing software
intensive projects. Provide practical guidelines for planning, staffing,
executing, and monitoring projects.
Environment: Focuses on the activities and tools necessary to
configure the process for a project

RUP

Software Project Management

32

RUP

Software Project Management

33

RUP-Test Discipline
Finding and documenting defects in software
Generally advising about the perceived software
quality
Proving the validity of the assumptions made in
design and requirement specifications through
concrete demonstration
Verifying the software product functions as designed
Validating that the requirements have been
implemented appropriately
Focuses primarily on the evaluation or assessment of
quality realized through a number of core practices
Acts in many respects as a service provider to the
other disciplines

Agile Development

Customer centered: www.agilemanifesto.org

Extreme Programming XP
SCRUM
Highly iterative
Adapts to requirements changes
Focus on team communication
Good for small/skilled teams
Pair programming
Refactoring
Test driven development
Active customer involvement
May not scale for large projects

Software Project Management

35

Manifesto for Agile Software Development

We are uncovering better ways of

developing
software by doing it and helping others
do it.
Through this work we have come to
value:
Individuals and interactions over
processes and tools
Working software over comprehensive
documentation
Customer collaboration over contract
negotiation
Responding to change over following a

Kent Becks assumptions

If code reviews are good, we'll review code all the time (pair
programming).
If testing is good, everybody will test all the time (unit
testing), even the customers (functional testing).
If design is good, we'll make it part of everybody's daily
business (refactoring).
If simplicity is good, we'll always leave the system with the
simplest design that supports its current functionality (the
simplest thing that could possibly work).
If architecture is important, everybody will work defining
and refining the architecture all the time (metaphor).
If integration testing is important, then we'll integrate and
test several times a day (continuous integration).
If short iterations are good, we'll make the iterations really,
really shortseconds and minutes and hours, not weeks
and months and years (the Planning Game).
Software Project Management

37

XP Promises(Beck)

To programmers:
They will be able to work on things that really matter, every day.
They won't have to face scary situations alone.
They will be able to do everything in their power to make their system
successful.
They will make decisions that they can make best, and they won't
make decisions they aren't best qualified to make.
To customers and managers:
They will get the most possible value out of every programming week.
Every few weeks they will be able to see concrete progress on goals
they care about.
They will be able to change the direction of the project in the middle
of development without incurring exorbitant costs.
In short, XP:
reduce project risk,
improve responsiveness to business changes,
improve productivity throughout the life of a system,
add fun to building software in teams

Software Project Management

38

XP phases
Architectural Spike, where a prototype is created
to validate the primary concerns of an iteration:
reduces risk
Release planning phase, where the customer
writes stories, the programmers estimate them,
and the customer chooses the order in which
stories will be developed;
Iteration phase, where the customer writes tests
and answers questions, while the programmers
program. Versions are generated at few weeks.
Release phase, where the Programmers install
the software, and the Customer (hopefully)
accepts the result.
Software Project Management

39

XP
http://www.extremeprogramming.org/map/project.ht
ml

Software Project Management

40

XP Iterations

J. Shore, S. Warden: The Art Of Agile Development

Software Project Management

41

XP testing
Test early. This principle states that you must not wait
with testing until the entire system is assembled. Instead,
run test cases as soon as a unit is implemented, and
assemble your system out of carefully tested units.
Test first. Write test cases before implementing the unit.
This is useful because test cases can serve as
specifications. Any program feature without an
automated test simply doesn't exist.(K. Beck)
Test often. At the minimum, run your tests with each
release of the system. Better yet, run your tests with every
change.
Have others test. Always have someone independent
test your program, and be open to criticism
Mostly perform Unit tests and Acceptance testing
Agile vs RUP:
http://www.agiledata.org/essays/rup.html

SCRUM
SCRUM: eight players rugby team with
different roles acting together for a specific
goal
Agile methodology
Iteration based: 1-4 weeks iterations
(Sprints)
Small teams: 7-9 people
SCRUM: practices and roles
http://www.infoq.com/minibooks/
scrum-xp-from-the-trenches
Software Project Management

43

http://scrumtraininginstitute.com/home/stream_download/scru
mprimer

Software Project Management

44

SCRUM Roles
Product Owner: business representative

identifying product features

translating these into a prioritized list
deciding which should be at the top of the list for the next Sprint
continually re-prioritizing and refining the list.

Team: programmers, architects, testers, etc

builds the product that the Product Owner indicates
cross-functional it includes all the expertise necessary to deliver the
potentially shippable product each Sprint
self-organizing with a very high degree of autonomy and
accountability

Scrum Master
helps the product group learn and apply Scrum to achieve business
value
help the Team and Product Owner be successful
cannot be the same with the product owner
he/she is not the leader of the team

The Scrum Master is NOT the project manager!

Software Project Management

45

Product backlog
Product vision: product owner building the
product backlog

customer features but also technical tasks

user stories or use cases
continuously updated by the Product Owner
items having a business value estimate
helping maximizing the ROI

Release Backlog: the subset of the

Product Backlog that is intended for the
current release
Software Project Management

46

Sprint backlog

Creating the current sprint plan from the product backlog

Sprint planning meeting part one: Product Owner the Team and the Scrum
Master
Focuses on what are the PO priorities
Provides the team with insight about the goal and the context of the
sprint
Establishes what a done item mean (e.g. coded to standards, reviewed,
implemented with unit test-driven development (TDD), tested with 100
percent test automation, integrated, and documented)
Sprint planning meeting part two: the Team
focuses on detailed task planning for how to implement the items that
the Team decides to take on
the Team selects the items from the Product Backlog they commit to
complete by the end of the Sprint, starting at the top of the Product
Backlog
It is updated every day with the remaining time for each task
Sprint Burndown chart: estimate of how much work (measured in person
hours) remains until the Teams tasks are finished.
It is a downward sloping graph that is on a trajectory to reach zero effort
remaining by the last day of the Sprint
Software Project Management

47

Daily Scrum
Short (15 minutes or less) meeting that happens every
workday at an appointed time.
Everyone on the Team attends.
To keep it brief, it is recommended that everyone
remain standing.
It is the Teams opportunity to synchronize their work
and report to each other on obstacles.
No management involved.
Each member of the Team reports three (and only
three) things to the other members of the Team:
what they were able to get done since the last meeting;
what they are planning to finish by the next meeting;
any blocks or impediments that are in their way.

Software Project Management

48

Sprint review
After the Sprint ends;
The Team and the Product Owner review the
Sprint;
An inspect and adapt activity for the product;
Product Owner learns what is going on with the
product and with the Team;
The Team learns what is going on with the
Product Owner and the market;
Attended by: Product Owner, Team members, and
Scrum Master, plus customers, stakeholders,
experts, executives, and anyone else interested.
Software Project Management

49

Sprint retrospective
Inspect and adapt regarding the process.
Its the main mechanism for taking the
visibility that Scrum provides into areas of
potential improvement, and turning it into
results.
Its an opportunity for the Team to discuss
whats working and whats not working, and
agree on changes to try.
The Team and Scrum Master will attend, and
the Product Owner is welcome but not
required to attend.
Software Project Management

50

Release sprint
Final sprint to complete the release
Integrate some elements, polish, etc
Sign of some development weakness
as ideally at the end of each sprint
the product should be good enough
for use

Software Project Management

51

Common system types

Business Systems

Mission-Critical Systems

Embedded Life-Critical Systems

Internet site

Embedded software

Avionics Software

Intranet site

Games

Embedded software

Inventory management

Internet site

Medical devices

Games

Packaged software

Operating Systems

Management Information Systems

Software tools

Packaged software

Payroll systems

Web services

Source: McConnel, Code Complete

Software Project Management

53

Life cycle model

Business Systems

Mission-Critical Systems

Agile development (Extreme

Programming, Scrum, timebox
development, and so on)

Staged delivery

Evolutionary prototyping

Evolutionary delivery

Embedded Life-Critical Systems

Staged delivery
Spiral development
Evolutionary delivery

Spiral development

Software Project Management

54

Project planning and

management
Business Systems

Mission-Critical Systems

Embedded Life-Critical Systems

Incremental project planning

Basic up-front planning

Extensive up-front planning

As-needed test and QA planning

Basic test planning

Extensive test planning

Informal change control

As-needed QA planning

Extensive QA planning

Formal change control

Rigorous change control

Software Project Management

55

Requirements and design

Business Systems

Mission-Critical Systems

Informal requirements specification Semiformal requirements

specification
Design and coding are combined

Embedded Life-Critical Systems

Formal requirements
specification

As-needed requirements reviews

Formal requirements inspections

Architectural design

Architectural design

Informal detailed design

Formal architecture inspections

As-needed design reviews

Formal detailed design

Formal detailed design
inspections

Software Project Management

56

*Coding, Testing and

Deployment*
Business Systems

Mission-Critical Systems

Embedded Life-Critical Systems

Pair programming or individual

coding

Pair programming or individual

coding

Pair programming or individual

coding

Informal check-in procedure or no

check-in procedure

Informal check-in procedure

Formal check-in procedure

As-needed code reviews

Formal code inspections

Developers test their own code

Developers test their own code

Developers test their own code

Test-first development

Test-first development

Test-first development

Little or no testing by a separate

test group

Separate testing group

Separate testing group

Separate QA group

Informal deployment procedure

Formal deployment procedure

Software Project Management

Formal deployment procedure

57