Professional Documents
Culture Documents
김범수
bskim@kbscom.co.kr
한국비지네스써비스 ( 주 )
Purpose of this Presentation
Educate SE’s on the LoadTesting
Methodology presented to the customer on
the first day of a QuickStart
Ensure that appropriate expectations are set
with customers - PRIOR TO TEST !!!
Better understand what it takes to perform a
successful LoadTest
LoadTest Implementation
Process
Execution Results
Preparation Development
Analysis Summary
,
Start Finish Go
Testing Testing Live!
Planning
Objectives / Goals of LoadTest
System Analysis and Design
LoadRunner H/W and S/W
DB Refresh Strategy and SUT
Monitoring Tools
LoadRunner Product Training
Planning - Objectives/Goals
FOR EXAMPLE:
Stability - Will 500 concurrent users crash
the system?
Reliability - Do all the Business
transactions work properly under load?
Performance - Is the performance
acceptable according to the specifications?
Planning - Analysis/Design
Designing a Scenario:
– Who are the users?
– How many users are there?
– What do they do?
– How often do they do it?
What do you want to measure (transactions)
– Entire Business Process (Post Journal Entry)
– All SQL traffic
– Specific DB inserts or updates
Planning - Analysis/Design
Task Distribution Diagram
Typical Work Day
12 AM 6 AM 8 AM 12 PM Emulated 5 PM 8 PM 10 PM 12 AM
Hour
System Backup Create Invoices (Accounts Payable)
400 500 Delivery Due List
15 20 50 5 20 2
Create Sales Orders
Good Issued
35 15 20 70 20
Display Sales Orders
5 15 20 5
Transferring Materials to the Vendors
15 10
Billing Due List
Changing Sales
Orders27 60
GUIvirtualuser
DBvirtualuser
Host1–
DBv users
Host2– WAN
DBv users
Ap plication
Server
Host3–
GU Ivuser
Controller
Database
Server
Planning - SUT
Scripting and Execution environment
Stable environment
– configuration
– data (loaded with Master and Transactional)
– completed System and Functional Testing
DB refresh strategy
– Between runs ?
Planning - Monitoring Tools
LoadRunner Monitoring Tools
Database Tools
Network Sniffers
(H/W) Operating System
Planning - Who?
Project Lead -
– Objective, Goals and Project Plans
LoadRunner Team -
– Analysis, Design and Training
Technical System Staff -
– System Under Test
Functional Experts -
– Analysis and Design
Development
LR Script Development
– GUI - WinRunner
– DB - VuGen
“Test” Data creation and preparation
LR Script Testing
Script Development
Understand all Business Process
– Navigation and all Data input/output
Basic Script Recording
Enhance Script
– Enter transactions and rendezvous
– Parameterize and variablize data
– Error checking routines
– Correlate Queries
Test Data - Creation
Build Data files to support multiple users
– Extract existing data from DB
– Build data as necessary - LR Scripts
Data “Prep” Phase
– In SUT create and modify data as needed to
support large load test
– Create backup of DB to refresh from
Script Testing
Single User - Single Script
– Test all the data combinations
– Identify new exceptions or correlation's
Multiple User - Single Script
– Table locks
– Data integrity
Multiple User - Multiple Scripts
– Data issues among different BP’s and users
Development And Testing -
Who?
LoadRunner Team -
– Create and test both scripts and data
Technical System Staff -
– Create data and DB strategy
Functional Experts -
– Create scripts and data
Execution
Three types of execution
– 20% of Full Load - flush out transaction
problems, effectively debug transaction
– Full Load - flush out system problems,
effectively debug system
– 120% of Full Load (Overdrive) - provide a
margin of error as well as future scalability
Execution
Should be an iterative process:
– Refresh DB - Baseline
– Run Scenario (LoadTest)
– Analyze Results (LR, DB, OS, Network)
– Tune System
Execution - Who?
LoadRunner Team -
– Run Scenario and pinpoint problems
Technical System Staff -
– Tune and Debug System problems
Application Developers -
– Tune and Debug Application problems
Common Challenges:
– Analysis and Design - time consuming, difficult
to analyze user activities
– Data Refresh Strategy - time consuming,
schedule into system activities, disk space
– Functional Expertise - hard to find, needed
during script recording and data management
– Debug/Tune System - very specific expertise,
time and availability, ROI !!!
Wrap-Up
LoadTesting is NOT trivial
Minimal 3 weeks with very narrowed scope
and full systems and functional support
Engage “Consulting Group” to participate
in conference calls with customers to
discuss implementation issues if necessary
Wrap-Up
Excellent products do not guarantee
successful customers…
LoadRunner QuickStarts Do