Agile

Performance
Testing
Lokesh Gulechha
Associate
Abstract
Increase in competition and leaps in technology
have forced companies to adopt innovative
approaches to have a system which performs well.
Today performance testing happens just before
product going live with very short timeframe
allocated for it which leads to missing performance
problems. With small extra efforts, efficiency of
performance testing can be increased significantly
Cognizant Technology Solutions and these extra efforts usually pay off multi-fold,
Plot No. 26, which can be achieved using Agile Performance
Testing.
Rajiv Gandhi Infotech Park,
This paper addresses performance testing using
MIDC, Hinjewadi,
agile methodology, its approach, advantage, risk
Pune-411057 (INDIA)
mitigation and many others.
+91-20-22931100
Table of Contents

1.0 Today’s Market need............................................................................................3

2.0 Current Performance Testing..............................................................................4
2.1 Objective...............................................................................................................................4
2.2 Approach..............................................................................................................................4
2.3 Risks......................................................................................................................................4
2.4 Drawbacks............................................................................................................................5
3.0 Agile Performance Testing..................................................................................6
3.1 Objective...............................................................................................................................6
3.2 Approach..............................................................................................................................6
3.3 Benefits.................................................................................................................................9
4.0 Effort Metrics for a Release...............................................................................10
4.1 With No Change in Requirement......................................................................................10
4.2 With Change in Requirement............................................................................................10
Note: Value may vary based on the data used for calculation of effort saving.12
5.0 Risk Mitigation....................................................................................................12
6.0 Advantage ...........................................................................................................13
7.0 Pre-requisite, Challenges and Risk..................................................................13
8.0 Conclusion..........................................................................................................14
9.0 About Author.......................................................................................................14

Agile Performance Testing

8
 AGILE PERFORMANCE TESTING

1.0 Today’s Market need

Increase in competition and leaps in technology have
forced companies to adopt innovative approaches to
have an application which performs well. Having
more features and functionality may not necessarily
translate into success for a system - what ultimately
count is how stable the consumer finds these
features.

Thus performance testing is the testing conducted to evaluate the performance of

a system with specified performance requirements.

Today performance testing happens just before product going live with very short
timeframe allocated for it. This may lead to missing performance problems and
propagate to production and have unhappy customer.

To have customer satisfaction in regards to performance, one can have Agile

Performance Testing which states: -

With small extra efforts, making the process more agile, efficiency of performance
testing increases significantly – and these extra efforts usually pay off multi-fold
even before the end of performance testing. Agile Performance Testing

8
2.0 Current Performance Testing

2.1 Objective
 To validate/determine the performance behaviour of the system with respect to
the end-user experience.

2.2 Approach

2.3 Risks
 When major issues are found at the end of test life cycle, it becomes much more
expensive to resolve them
Agile Performance Testing

 Major performance issues identified during later stages may hold up important
releases and delay the project delivery

8
2.4 Drawbacks
 Performance tuning is applied to server-configurable performance parameters
(Hardware/Software). Thus, scalability issues are addressed by expensive
hardware/software resizing techniques
 If code optimization is done, then one needs to go through the complete Testing
Lifecycle.

Agile Performance Testing

8
3.0 Agile Performance Testing

3.1 Objective
 To determine the performance bugs/bottlenecks in key business processes as
early as possible during the project development cycle
 Identify, isolate and fix performance bottlenecks at the component/code level

3.2 Approach
Agile Performance Testing is a three phases approach: -

 Unit Level Tuning – For optimizing

o Execute tests to isolate and fix bottlenecks at code level

 Component Level Tuning – For testing Components

o Execute tests to isolate and fix bottlenecks in application components
level.

 Application Load Tuning – For testing Application Flows

o Test the critical application flows for user-experience under normal and
peak loads

Agile Performance Testing

8
Let’s have detail look at each of the phase: -
Unit Level Tuning

Component Level Tuning

Agile Performance Testing

8
 Application Load Tuning

Agile Performance Testing

8
3.3 Benefits
 Iterative performance testing leads to better code, optimized for performance

 Early detection of performance bugs/bottlenecks reduces the effort & duration for
performance tuning and retesting of application

Agile Performance Testing

8
4.0 Effort Metrics for a Release
What will be the efforts required to incorporate Agile Performance Testing?
To evaluate this, effort required is calculated based on following criteria during a
release: -

 With No Change in requirement

 With Change in requirement

4.1 With No Change in Requirement

Testing Lifecycle Current Approach Agile PT Approach
Unit Testing 10 10
Unit Level Performance Tuning 0 2
Integration/Functional Testing 3 3
Component Level Tuning 0 2
UAT Testing 4 4
Application Load Testing 2 (Script Re-Used)1
Tuning Performance bottleneck at 1 0
Code Level.
Tuning Performance bottleneck at 0 1
Component Level.
Unit Testing after Tuning 2 Not Required,
Integration/Functional Testing after 1 because Code Level
Tuning Tuning has been
UAT Testing after Tuning 2
already done in
step2
Application Load Testing after Tuning (Script needs to re- (Script Re-used)1
create due to code
tuning)2
Total Efforts Required (Man Day) 28 24

Saving in Efforts = 28 – 24 = 4 Man Days = 14% (approx.) for one module

Agile Performance Testing

4.2 With Change in Requirement

When change in requirement is invoked, efforts required for testing lifecycle is less
as compared to the efforts for first time testing of an application. And also Unit level
tuning for CR depends on how changes made impact the working of application.
Example: -
Considering 2 CR’s are made, there are following 3 scenarios for effort saving: -

8
Scenario 1:

» Performance Testing (Unit Level Tuning) is performed in DEV Environment

considering major changes impacting the application for both the CRs.

Effort Saving
35

30

25
Saving in %

20

15

10

0
First Run Run after CR1 Run after CR2
Testing Phase

Scenario 2:

» Performance Testing (Unit Level Tuning) is not performed in DEV Environment

considering minor changes not impacting the application for both the CRs.

Effort Saving
50
45
40
35
Saving in %

30
25
20
15
10
5
0
First Run Run after CR1 Run after CR2
Testing Phase
Agile Performance Testing

Scenario 3:

» CR1 - Performance Testing is performed in DEV Environment considering

major changes impacting the application.

» CR2 - Performance Testing is not performed in DEV Environment considering

minor changes not impacting the application.

8
 Effort Saving
50
45
40
35
Saving in %

30
25
20
15
10
5
0
First Run Run after CR1 Run after CR2

Testing Phase

Note: Value may vary based on the data used for calculation of effort saving.

Agile Performance Testing

5.0 Risk Mitigation

Agile Performance Testing has positive outcome in risk mitigation with respect to: -

 Customer Satisfaction
Customers/users have higher expectation about the quality of application.
 Code Tuning

8
 Unit Level tuning is done early so no need for code optimization at later stage
on Testing Lifecycle.
 Application Rejection
Reduces risk of the application being rejected because of issues such as Memory
Leaks, Hard Coded Values in code, Database Locking
 Release Dates
Iterative Testing leads to constant awareness of the application performance and
therefore more confidence in meeting release dates.
 Performance Bugs
Less Number of Performance bug(s) in subsequent phase(s).
 Performance Testing Efforts
Performance Script can be re-used thus saving 60% of the efforts of Performance
Cycle.

6.0 Advantage
One Good decision at Start can cover a lot

 Earlier detection of performance bottlenecks

» Reduces the effort and duration for performance tuning and retesting
» Project management has constant awareness of the application
performance and therefore more confidence in meeting release dates.
 Faster resolution of identified issues
» Bottom-up approach to identify bottlenecks helps narrowing down the
root-cause faster
 Better performance tuning
» Development team gets more time for performance tuning activities
 Cost Effective

» Identified bottlenecks are resolved much earlier in Testing Lifecycle

Agile Performance Testing

saving expenses required to a great extend.

7.0 Pre-requisite, Challenges and Risk

 Pre-requisites
» Full time Performance Tester.

» Must be technically sound.

8
 » Closer interactions between developers & testers
» Access to monitor all the servers.
 Challenges
» Defining accurate performance requirements.
 Risks
» Incorrect performance requirements may jeopardise the testing

8.0 Conclusion
Applications are backbone of business; hence performance of application becomes
one of the key differentiators in success or failure of business. Thus performance
testing is conducted to evaluate the performance of a system with specified
performance requirements.

In current approach, performance testing happens just before application going live
with very short timeframe allocated for it. When major issues are found at the end of
test life cycle, it becomes much more expensive to resolve them which may hold up
important releases.

This calls for the solution which states - “Investigate performance early and Validate
performance last”

Agile Performance Testing does just that. With Agile Performance Testing, Project
management is constantly aware of the application performance and therefore more
confidence in meeting release dates.

Agile Performance Testing has positive outcome in risk mitigation because systems
are getting more and more complex with a dynamic technology landscape and high
standards of customer satisfaction

One Good decision at Start can cover a lot Agile Performance Testing

9.0 About Author

A Cognizant India Test Engineer with over 3.5 years of proven Quality and Test
management experience in the Financial Services sector. Consistently developed

8
and implemented new ideas and techniques which have lead to dramatic Quality
improvement within the projects. Published following whitepapers:
1. Customer Satisfaction through Quality Index
http://www/projectperfect.com.au/info_quality_index.php
2. Sanity Testing
http://www.stickyminds.com
3. Software Testing Metrics
http://www.stickyminds.com/s.asp?F=S14789_ART_37
To reach email at: - Lokesh.gulechha@cognizant.com

Agile Performance Testing