Express functional testing approachdoing more with less

Anuj Magazine anuj.magazine@citrix.com

http://anujmagazine.blogspot.com Twitter: anujmagazine

Click to edit Master subtitle style

2012 Citrix | Confidential Do Not Distribute

Have you

Ever been asked to test a project with no or very less or not to your satisfaction requirements specifications ?

Ever been involved in testing a project which had no test cases or a very little time to create them ? Ever been involved in testing a project in which features/implementation changes dynamically ?

2012 Citrix | Confidential Do Not Distribute

Citrix Globalization Services

2012 Citrix | Confidential Do Not Distribute

Groups key challenge

2012 Citrix | Confidential Do Not Distribute

Traditional G11N Functional testing process

G11N Testing Cycle

2012 Citrix | Confidential Do Not Distribute

Situations where Traditional process didnt scale

-up art ts St jec pro
Typical Shortcomings

Main tena proje nce cts

Less Base Language test cases or No Base Language test cases at all The Rapid product changes syndrome Difficulty in seamlessly keeping-up with Product teams milestones Difficulty in dealing with situations when there are no

2012 Citrix | Confidential Do Not Distribute second chance of finding bugs

The basis of Express functional testing approach

2012 Citrix | Confidential Do Not Distribute

What kind of change we were looking to see ?

Something that can help us deal Something that can help us find

with the rapid product changes bugs soon and early testing more scientific

Something that can help us make Regression Something that can help us deal

language) test cases

with reducing dependency on EN(base merge with product teams

Something that can help us seamlessly

milestones

2012 Citrix | Confidential Do Not Distribute

We had

We wanted to

Less documentation Lesser base test cases Less time at hand Less chances of timely bugs Less preparedness to deal with chances

Find more bugs Become more responsive to project changes Enhances our chances of meeting internal milestones. Not increase human resources infinitely

In short, we wanted to do more with less

9
2012 Citrix | Confidential Do Not Distribute

The basis of Express functional testing approach

Test case

10

2012 Citrix | Confidential Do Not Distribute

The essence of Express functional testing

SBTM: Session Based Test Management

1111 2012 Citrix | Confidential Do Not Distribute

The Three Core ideas

Basic source of tests-

a code check-in

Refined approach to testing-

express test ideas generation

Automating functional tests

reuse the existing code

12
12

2012 Citrix | Confidential Do Not Distribute

Explaining the basic source of tests- a code check-in

2012 Citrix | Confidential Do Not Distribute

Elements of the Express functional testing

Analysi s Verificati on

Track Daily check-ins

Assess the functional impact

Generate test ideas at runtime

Results are updated in the Tracker

Perform regression tests

Report bug if any issue

Note the build, results

2012 Citrix | Confidential Do Not Distribute

Citrix Confidential - Do Not Distribute

Citrix Confidential - Do

Components of a check-in

15

2012 Citrix | Confidential Do Not Distribute

check-in Summary
No.of Lines in the Code that were affected by the Actual Source Code change

Changes

2012 Citrix | Confidential Do Not Distribute 8/15/12

check-in types

check-ins related to Feature Changes check-ins related to Bug Fixes check-ins related to Document Changes check-ins related to Changes in Automation script

17

2012 Citrix | Confidential Do Not Distribute

Check-in types
check-in for feature change

Using check-in Summary and the other details in the source code we could derive ideas for tests

18

2012 Citrix | Confidential Do Not Distribute

 2012 Citrix | Confidential Do Not Distribute

Citrix Confidential - Do Not Distribute

Some challenges

Evolving the skill level of the team. Enhance the Developer skills and mix it appropriately with tester mindset to create an irresistible engineering combination. Build rapport/communication with the development team. Building the buy-in for the change in the approach. Imbibing the necessary Sense of urgency within the team.

20

2012 Citrix | Confidential Do Not Distribute

Express test ideas generation/testing

2012 Citrix | Confidential Do Not Distribute

Elements of the Express functional testing

Analysi s Verificati on

Track Daily check-ins

Assess the G11n impact

Generate test ideas at runtime

Results are updated in the Tracker

Perform regression tests

Report bug if any issue

Note the build, results

2012 Citrix | Confidential Do Not Distribute

Citrix Confidential - Do Not Distribute

Citrix Confidential - Do

Challenges in absence of test cases

Building accountability in testing. Establishing the focus into specific areas. Knowing how much is tested and how much more needs to be tested. Designing the tests and running them almost at the same time.

2012 Citrix | Confidential Do Not Distribute

Session based testing

What is a session?

A session is an uninterrupted block of reviewable, chartered test effort

Typical sessions last for 90 minutes

2012 Citrix | Confidential Do Not Distribute

Session

Test Design/Execution

Bugs Investigation/Reporting

Session Setup

Test design and execution means scanning the product and looking for problems

Bug investigation and reporting is what happens once the tester stumbles into behavior that looks like it might be a problem

2012 Citrix | Confidential Do Not Distribute

Anything else testers do that makes the first two tasks possible, including configuring equipment, reading manuals, or writing a session report

Session Report

2012 Citrix | Confidential Do Not Distribute

Automating tests - Reusing the existing code

2012 Citrix | Confidential Do Not Distribute

How the Automation Process Evolved. Understanding Analysis

Support For Non-En platforms

Execute Auto BVT Enable Unicode Support on JA, FR, DE, ES, SC, RU languages

Architecture Execution on EN Setup

Optimization

2012 Citrix | Confidential Do Not Distribute

Input non-ASCII characters as Input Data (Multi Language Data) Scheduler Automatic analysis of test results Citrix Confidential - Do Not Distributetest Result Auto e-Mail Citrix Confidential - Do

Enable the scripts to input non-ascii characters

Make the scripts UTF enabled # -*- coding: utf-8 -*-

2012 Citrix | Confidential Do Not Distribute

Citrix Confidential - Do Not Distribute

Citrix Confidential - Do

Multi Language data

@!23

30

2012 Citrix | Confidential Do Not Distribute

Results summary

2012 Citrix | Confidential Do Not Distribute

More Functional bugs per release

12 10 8 6 4 2 0
If further colors are needed, please contact Creative Services

32

2012 Citrix | Confidential Do Not Distribute

Early, Timely detection/fixing of Functional bugs

12 10 8 6 4 2 0
If further colors are needed, please contact Creative Services

33

2012 Citrix | Confidential Do Not Distribute

The crux of Express functional testing approach It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change

2012 Citrix | Confidential Do Not Distribute

 2012 Citrix | Confidential Do Not Distribute

References
Images Sources:

Other Credits:
Collgues: Rubina Gonsalves and Shivaraj Shet
http ://www.satisfice.com/sbtm/ http:// satisfice.com/articles/sbtm.pdf

http://www.cleapublicrelations.com http://www.c3-ssi.com http://www.linkbuildr.com http://www.torwars.com http://www.branchpastor.com http://blog.workinsports.com http://www.121box.com http://www.march-hare.com http://www.larter.ca

2012 Citrix | Confidential Do Not Distribute

Citrix Confidential - Do Not Distribute

Work better. Live better.

Anuj Magazine
anuj.magazine@citrix.com
http://anujmagazine.blogspot.com Twitter: anujmagazine

2012 Citrix | Confidential Do Not Distribute