Professional Documents
Culture Documents
Kees Blokland
Martin Pol
Jeroen Mengerink
Test consultant, Teacher,
Researcher at Polteq
EuroSTAR
Software Testing
C o n fe re n c e
EuroSTAR
Software Testing
Community
1. Preface
PAGE
2. Introduction
PAGE
Chapter 5 Author:
explains theKees
riskBlokland
based strategy to
August 27,
2013
determine Version:
the test measures
that
could help
mitigating cloud related risks. Two risk areas
3. What is Cloud
Computing?
With cloud computing, a world of new opportunities that can be fitted into the current IT landscape arise. For a
good understanding of the terms and concepts used in this book, a practical definition for cloud computing is
essential. For this, we use the definition that is put forward by the American National Institute of Standards and
Technology (NIST).
PAGE
Author:
Kees Blokland
Version:
The use and development of software continually changes, and that affects the activities and position of the test
PAGE
Author:
Kees Blokland
Version:
August 27, 2013
Testing Cloud Services. How to test Saas, Paas and Iaas
ensure continuity in production. Where there is knowledge of the exact changes to the customers own software, in
cloud computing the customer can be confronted with unannounced and hardly documented changes. The
so response
of might
the service
are in
affected
and in production,
there test
is a job
thatnecessary
needs toas a backstop.
continuous
system integration
will be
The times
situation
even arise
production
thatbe
makes
another implementation.
service or service This
supplier by
necessaryfor
because
showsathat
these otherexample,
customers.
This testing
introduces
donechoosing
during service
guarantees
in continuity
arerole
not of
met.
performance risk. The chosen implementation
is like the
traditional
the test manager:
In addition to a role during service selection and in production, there is a job that needs to be done during service
model also affects the risks, especially the
testing and giving advice on getting information
implementation. This is like the traditional role of the test manager: testing and giving advice on getting
severity
The has
security
risk (see
in the
systemssystems
into production
(operation).
All in
information
into production
(operation).
Allall,
in all, the
role ofof
thethe
testrisks.
manager
broadened
figure
private
cloud
is
less
than
in
the
public
cloud
the
role
of
the
test
manager
has
broadened
below).
Broader
rolerole
of the
testtest
manager
Broader
of the
manager
PAGE
Polteq
Pagina | 7
From the essential characteristics of cloud computing, a number of risks can be immediately determined. For
Functionality
Maintainability
Legislation and regulations (example in
this e-book)
Suppliers and outsourcing
PAGE
5.1Performance
Risks
Risk
5.1.3, 5.2.4
PAGE
Risk
5.1.3, 5.8
5.1.3, 5.8
Subject:
Concept
Eurostar
Testing Cloud Services.
How to test Saas,
Paase-book
and Iaas
Author:
Kees Blokland
Version:
6. Test Matters
6 TEST MEASURES
PAGE
8
To address the risks in the previous chapter, test measures are needed. In describing the test measures, we pay a
lot of attention to test management. In addition, for certain test measures, test specification and test execution is
during
selection
To
addressTesting
the risks
the previous
addressed.
Cloud in
Services
presents a chapter,
pick and mix of testTesting
measures:
depending
on the(with
risks,an
oneexample
or more
measures
are
picked
and
mixed.
The
starting
point
for
this
chapter
is
the
outcome
of
the
product
risk
analysis
of
is this e-book)
test measures are needed. In describing the
the service. The result is a list of relevant detailed risks with appointed classification (high/medium/low). For each
appropriate
Testing test
performance
(with
example
test
measures,
weallows
pay the
a lot
attention
to
risk group,
this list
test of
manager
to determine
measures. In
this an
chapter,
we will
describe
the
following
test
measures:
is this e-book)
test management. In addition, for certain test
Testing security
Testing
for
manageability
Testing availability/continuity
a pick and mix of test measures: depending
Testing availability/continuity
Testing functionality (with an example is
on the risks, one or more measures are picked
Testing
due
to
legislation
and
regulations
Testing migrations
the outcome of the product risk analysis of the
Testing in production
Pagina | 11
PAGE
Generally, during selection, two or more service providers are considered. Too many options cost too much time
Subject:
Concept
Testing Cloud Services.
How to test Saas,
Paas e-book
and IaasEurostar
Author:
Kees Blokland
Version:
PAGE
10
6.2.1
6.2.1Testing
Testingelasticity
elasticity
Boundary
valuecan
analysis
at thepart
boundaries
of scaling up
scaling down
andinvoicing)
(financial)
handling
become
of testing
Process cycle test for the administrative process (this will include pay-per- view invoicing)
elasticity.
The approach to this test comprises executing
a load test with the load increasing beyond
the scaling boundary (scaling up) and then the
load decreasing below the scaling boundary
Polteq down). Manual actions may need to
(scaling
be performed as part of the test procedure
in a process cycle test. Behavior on possible
boundaries of packages also needs to be
addressed.
The following test approach covers a situation
Pagina | 14
Combining test
test techniques
techniques
Combining
PAGE
11
Author:
Kees Blokland
Version:
PAGE
12
Which variations can be applicable (see figure
below)?
Browser types (IE, Safari, Chrome)
Variations of apps
Different operating systems (Windows,
iOS, Linux, etc.)
Subject:
Concept e-book Eurostar
Different mobile devices (tablets,
Author:
Kees Blokland
smartphones)
Version:
August 27, 2013
Different versions
of hardware
PAGE
13
Author:
Kees Blokland
Version:
7. Completion
7 COMPLETION
How can a book that is about something that still needs to reach full development be concluded? For starters, by
establishing that practice will show to what extent the test measures fit and which further innovations are needed.
More risks will undoubtedly be identified, existing risks will reveal to be less relevant than imagined, and the test
measures will be adjusted accordingly. Further developments will be followed and various media additions to this
book will appear. Testing Cloud Services offers the entire approach, from identifying risks to taking test measures.
With all the risks and test measures that are described in this book, the impression might be that going to the
PAGE
14
No Risk, No Test
Example. Based on the extent of the security
risks, one can choose from the following
options:
Biographies
PAGE
15
Blokland/ Mengerink/Pol, Testing Cloud Services, How to Test Saas, Paas & Iaas, Rockynook,
ISBN 978-1-937538-38-5
w w w. e u r o s t a r c o n f e r e n c e s . c o m