Ebook

eBook
The Ideal Testing Solution:

Key Capabilities You
Are Probably Missing
What’s
Inside?
3 The State of Functional Testing
4 Business Process Centricity
8 Visibility and Collaboration
11 Workflow and Process Automation
13 Testing That Mirrors Production
17 Agile Test Management & Acceleration
2 The Ideal Testing Solution: Key Capabilities You Are Probably Missing
Experience Risk-Free Change Schedule My Demo »

The State of
Functional Testing
The testing industry is continuously changing. Enterprises’ journey
towards a continuous delivery of customer experience innovation is in
full swing, and Quality Assurance and Testing organizations are very
much in the eye of the storm.
A recent report based on Spiceworks‘ research* shows that while

priorities, methodology and strategy are aligned to support shorter
release cycles – QA and testing solutions fall short.
Five important trends that are currently underpinning the testing

solutions industry were also uncovered in the report. This eBook
applies insights from the report to address the challenges faced by
test managers today – such as the need for increased visibility, control,
standardization, and automation of manual processes - which most
testing solutions fail to answer. What they need is a test acceleration
solution to organize, measure and manage testing cycles without
having to compromise on scope or quality.
Choosing the right testing solution can stop the stress in testing
and help your organization stay ahead of the curve. Follow these
recommendations for a testing methodology that enables rapid
change without compromising on scope or quality.
*All numbered graphs are part of the 2017 State of Functional Testing - Survey Report

1 Business Process
Centricity
With change delivery to enterprise applications becoming
instrumental in the competitive race to win, serve, and retain
customers, survey results suggest that testing activities are also
becoming more pivotal, as business process quality is deemed
more important than time-to-market (Graph no. 1).
The infrastructure for technical and functional testing must

be present and connected to every phase. This starts with the
business-driven change request, through to project scoping, and
on to realization in development.
‘The Age of the Customer’ is impacting testing practices as

improving customer experience (eliminating defects) is now the
highest priority for testers.
Waterfall vs. Agile Application Delivery
Waterfall Delivery
Analyze Plan Design Test Build Deploy
Agile Delivery gn
si
De
Tes
Analyze Plan Deploy

t
Bu
il d

1 Business Process Quality Is More Important Than Time-To-Market
58% 55% 51%

62%
76% Critical
Nice to have
Not at all important
40% 42% 44%

35%
22%
Removing/ Releasing Validating use Minimizing Increasing
eliminating on time cases/ testing costs level of test
defects/ requirements automation
incidents
How to Meet the Demands of Enterprise Agility

The 2017 World Quality Report predicts that Testing will represent
40% of the IT spend in 2018.
This is certainly due to the increasing number of organizations

becoming agile, releasing change faster without compromising on
quality, as well as on the increasing complexity of regulations.
Ensure your testing practices support your organization’s agility

strategy by choosing a testing solution that is business process-
centric.
Introducing Panaya’s Business Process-Centric

Test Management
While a new code change may work well within its own element, a
business process test qualifies the change to the daily business.
Pursuing business process-centricity means that test cycles must
be validated with business process awareness, leveraging the
knowledge of subject-matter experts, who might be cross-functional
and geographically dispersed.

For example, a new purchase order format should function correctly
throughout the entire Purchase-to-Pay business process. As of its
initial creation, through to approval, goods receipt, invoice creation,
and finally to accounting - whether or not it interfaces with satellite
systems such as Salesforce.com or Supply Chain Management.
A Globally Distributed
Purchase-to-Pay Business Process

To ensure that any change to a business application will function
correctly at go-live, Panaya Test Center supports business process-
centric management and monitoring throughout the test lifecycle:
planning, execution and evaluation. This reflects into the creation
of test scripts, planning of workflows, assignment of testers,
sequencing of execution and reporting on the quality and integrity
of business processes.
With Panaya's business process-centric approach, the test manager

and business owners get full visibility into the overall quality and
status of business processes. Testers confirm that no defects or
risks are found when the new code change is applied to a specified
business process. While validation may depend on multiple users,
steps, functions and geographies, it is indispensable in ensuring
that no bugs have slipped into production to negatively impact
the customer experience.

2 Visibility and
Collaboration
In line with the Business-IT convergence trend, the survey found that
test managers need visibility into test readiness. In other words, they
lack the confidence that tests reflect real-life business scenarios. A
Business-IT disconnect hinders testers' ability to support a continuous
delivery of change that is truly complementary of business goals. For
example, the majority (53%) of testing leaders believe making
their organization more agile is critical to defining their test
methodology.
The lack of progress visibility and production data available for test
case creation leads testing executives to take an extra cautious
approach and as a result, test more and delay delivery. 31% of
testers admit to always test more than needed to ensure quality
(Graph no. 2), causing delays and other forms of waste.
In addition, the survey Test Scoping Challenges:

uncovered major collaboration 2 Testing More to Ensure Quality
pitfalls in relation to specific
testing activities. Going back
to the fact that eliminating Always
defects was found to be the top
priority for testing, an inability 31%
to recreate or duplicate
defects was identified as
the most common pain
point with regards to defect
management.
How to Collaborate with Visibility

With customer experience in the forefront of all businesses today,
testing must support business process validation. Testing is no longer
just about verifying that technical aspects of application changes
work correctly (e.g., load and performance) but also, the validation of
customer experience against business requirements.

Therefore, in the context of test acceleration, visibility and
collaboration are one and the same. A testing solution must provide
real-time visibility and tracking of all testing stages, the results, and
the defects, as well as workflow management. Multiple stakeholders
should not only see but also be able to act on real-time information.
To gain the confidence to meet both time and quality standards,

before embarking on a change project, a testing solution must provide
full visibility into the upcoming project. That is, a holistic overview of
system interfaces as well as information on which transactions are
used for each business process, how often and the significance of
those transactions.
To meet business requirements with speed without compromising

quality, invest in a testing solution that supports seamless
communication and collaboration between different and globally
dispersed stakeholders, such as business users, testers and
developers. Features like system-wide defect tracking that enables
communication with developers, testers, users and even third-party
applications partners are a plus.
Introducing Business Process-Centric Planning

and System-Wide Defect Tracking
With Panaya Test Center, the uncertainty that was previously
inherent to test planning, for example the UAT cycle, is now
completely removed. Visibility is conducive to time saving as
the progress dashboard is visible to all. A focus on business
processes - as opposed to test steps - empowers test managers
with the certainty that all business processes are indeed tested.
System-Wide Defect Tracking with Panaya Test Center

Panaya Test Center customers attest to using the Panaya dashboard
for UAT planning – all the way down to the module level - at steering
committees. For example, if an X number of test scripts must be run
and only Y days are left to complete UAT, test managers knew that on
the next day they must have Z% of those tests completed to be on
target. Thanks to Panaya Test Center, unused or outdated business
processes are taken out of this equation.
To eliminate waste of resources and time-to-market delays due to

defect inflation, Panaya Test Center alerts and identifies potential
system-wide defects, meaning that they may affect other tests.
Following the alert, the platform suggests several courses of action.
A testing manager can for instance, define an automatic notification

for all testers who are assigned to test a component associated with
a system-wide defect.
A testing project manager has full visibility into system-wide

defects and their influence on other tests, with Panaya Test Center
immediately highlighting the testing bottlenecks. Panaya Test Center
is designed to relieve the stress of test management, so that you can
organize, measure and execute testing cycles without compromising
on scope or quality.

3 Workflow and
Process Automation
According to the Spiceworks survey, test automation is coveted by all
but still too costly (even for regression testing) to implement without risk,
perhaps due to the lack of dedicated developers. However, automation
is a top priority for testing executives - the lack of which identified
as the second largest test execution challenge.
The survey also found that Percentage of Functional

while 43% of organizations 3 Testing Done Manually
have purchased a dedicated
testing solution (Graph no. 6),
54% of business process
testing is still done manually
(Graph no. 3). This fact speaks 54%
volumes to the automation
solutions market gap.
How to Go Beyond Automation

A testing tool must support holistic test automation. Test automation
needs to be an integral part of the entire execution process - not
just regression cycles, where most of the functional test automation
currently occurs.
Moreover, there is a greater, unfulfilled scope for test automation,

neglected by most testing solutions. Automating test design, allotment,
workflow management and other process orchestration activities
can reduce global test efforts by 30%. Features like the automatic
sequencing of manual tests, automatic capturing of test results, auto-
creation of test scripts and on-demand availability of the above via
a central platform can make test automation an integral part of the
entire execution process.

Beyond Test Execution Automation
To truly reduce testing time, test automation needs to go beyond
test execution. Panaya Test Center tackles this need with a holistic
view of automation - from smart defect management, through to
automated test case upload and on to auto-handover of test steps
and test evidence automation. Almost every counterproductive
element of manual work and coordination is automated - freeing
testers to focus on testing rather than logistics. Here are some of the
more unique elements automated:
›› Smart defect management. Panaya Test Center accelerates

defect lifecycle management with a full suite of powerful features
to increase productivity, encompassing industry best practices.
Features include automatic alerts for defects, real-time automatic
documentation of defects, retest notification to testers upon
defect resolution and close notification to developers upon retest
success, for all impacted business processes.
›› Test evidence automation. With no additional effort required by

testers, Panaya Test Center automatically documents every test
execution - delivering granular test evidence. This reduces the back
and forth between developers and testers, providing a record of
steps for test reproduction. Additionally, documentation is always
audit-ready, ensuring all company quality standards are met.
›› Effortless test case migration. With Panaya Test Center there is

no need to recreate or migrate existing test cases and business
processes. Whether they are formatted as Word, Excel or PDF
documents - just upload them to Panaya Test Box and Panaya
handles the migration from start to finish.

4 Testing That
Mirrors Production
The survey identified a growing demand for synergies between IT
and Business. This includes more collaboration, ongoing business
requirements awareness, joint planning and transparency and an overall
business process-oriented approach to application lifecycle management.
For example, the survey shows that testers’ key performance indicators
(KPIs) have shifted, with understanding business requirements seen as
most critical to defining test methodology; followed closely by improving
customer experience (Graph no. 4).
This trend is expected to intensify in the future, as over one third of testing
leaders believe that business domain expertise will become a requirement
for testers with the growing effects of agile and continuous delivery.
However, the new business-

4 Top Test Methodology Priorities
driven approach to
testing exposes gaps in IT Understanding the business
organizations’ readiness 74% requirement
for continuous delivery
of change, as almost half Creating a bettercustomer
(47%) of IT leaders surveyed
65% experience
believe their tests do not

accurately reflect real-
life production scenarios
55% Enabling continuous
delivery of change
(Graph no. 5). In fact, creating

testing scenarios that 53% Making my organization
more agile
accurately reflect business
processes is currently the top
test planning challenge.

5 Regression Testing Challenges
47% Tests do not accurately reﬂect real-life production scenarios
42% Regression suite is too hard to create and maintain
38% Not enough developers for test automation
35% Automation is too costly
*Percent according to challenges faced by respondents
How to Keep Test Cases Real

To meet delivery with quality, IT and Business become inherently
linked from defining the change through delivery and into production.
All the while, the two must work together to enable the right changes
without compromising business process quality. This partnership
between IT and Business is where Quality Assurance (QA) extends
from technical development and testing to business testing and
validation.
To accommodate the Business-IT convergence trend, choose a

testing solution that mirrors the way business is conducted in the
real world.
Introducing Panaya Autonomous TestingSM

Panaya Autonomous Testing℠ is a proprietary solution that generates
business process tests automatically. As testers go about their jobs,
machine learning captures user behavior, determining top usage
patterns as well as system level interactions not typically observed in
automation tools.

Panaya Autonomous Testing℠
Autonomous Testing℠ enables zero-touch test case creation

and maintenance. By automatically capturing transactions during
production and converting them into test cases that represent
domain expertise, it eliminates the pains associated with business
knowledge capture and the process of creating and maintaining
manually engineered scripts. As a result, quality teams test just the
right things.
Once the Autonomous Testing℠ converter is enabled on production

user workstations, it automatically and continuously gathers user
activities during production. Panaya machine learning, ensures an
always up-to-date, accurate and maintenance-free regression suite.
Omega, the machine learning engine at the core of Panaya’s

Autonomous Testing℠ is a groundbreaking technology that enables
the conversion of business user activities into test cases, without
user intervention.

While continually capturing and analyzing transactions during
production, Omega algorithms synthesize an optimal, 100% accurate
regression suite by automatically converting user activities into test
scripts.
The outcome is a comprehensive set of use cases that reflect actual

business usage, instrumental in verifying how SAP updates would
impact critical business processes, ensuring error-proof testing that
mirrors actual user behavior.
Panaya Omega Machine Learning Algorithm
Raw data Clustered data Cluster optimizations Cluster representative
Weighted
dissimilarity to
Converter Clustering
Script Noise reduction optimal
algorithms
generator (meaningful matching Pick cluster
Modeling and silhouette
(raw data to actions steps, representative
optimization
scenario steps) recognizing) scenarios
dissimilarity-
matrix
Panaya Crowd Source Technology Machine Learning

(based on ~33M executed Steps) Based on Customer Processes
Panaya Omega - Machine Learning Algorithm

5 Agile Test Management
& Acceleration
The Spiceworks survey findings further demonstrate that while
testing decisions and processes are driven by the understanding
of business requirements and the need to improve customer
experience, execution is hindered by the lack of appropriate tools,
as 40% of respondents claim to be challenged by the lack of business
oriented testing reports. In fact, when asked about their top test
execution challenges, most testing leaders reported tools that
are unfriendly to non-technical testers and require intensive
training.
The need for better solutions can account for the low percentage
of organizations (12%) relying solely on existing centralized
testing solutions such as HP QC, in contrast with the median
percentage (43%) of organizations who have purchased such
solutions (Graph no 6).
6 Tools Used for Test Management

Don't know
Only using
COTS testing
tools such as
12% 3%
HP ALM
54% Using ad hoc solutions
31% such as Excel,

Word, etc.
Using a combination
of both*
*Combination of ad hoc solutions and testing tools such as HP ALM.

How to Get Test Management Working for You
and Stop Working for It
Capgemini’s World Quality Report recommends investing in
“intelligent self-learning QA and testing platforms for all areas of the
application landscape”.
Today’s legacy test management solutions like HP QC, meant to

support QA and testing, tend to deliver technically-oriented user
interface and experience, consistent with the IT side of testing,
rendering it useless to the Business side of testing. The result is a
test management tool that lacks key components - collaboration,
business process centricity, documentation, visibility and automated
process handling – all conducive to delivering application changes
faster.
Introducing Agile Test Management & Acceleration

with Requirements Traceability
As the complexity of platforms and devices increases,
testing your applications becomes more crucial and at
the same time, more difficult. Clearly, the pressure is on
testing managers to deliver production-ready applications
quickly, which means being agile while maintaining control.
Testing managers are now required to ensure that as business

requirements evolve throughout enterprise applications’
lifecycle, the application change they deliver never strays off those
requirements.
Panaya Test Center is built in with Requirements Traceability

- the systematic linkage that allows business requirements
and their subsequent fulfillment and validation to be followed
in both forwards and backwards directions (i.e. from their
origins, through to development and specification, to its
subsequent deployment and use, and through all periods of
on-going refinement and iteration in any of these phases).

Business Process Test Planning and Execution with Panaya Test Center
With Requirements Traceability, project and test managers can rest

assured that they are testing the right things and have full testing
coverage for the new change – keeping them quick and agile.
Panaya Test Center’s central control and actionable progress

insights, as well as automated reporting, help control the continual
develop-test-fix-and-release cycle. Panaya Test Center optimizes
test management through total visibility across projects, teams,
and tests.
With Panaya Test Center you always know how done you really
are – at every phase of the project. The solution provides real-
time reports and dashboards that convey testing status and track
overall team progress across test projects, cycles and business
processes. It assesses quality at every stage and enable stakeholders
to identify bottlenecks in the testing process, understand ‘where
they are’ and set expectations.

To ease the stress and stay on top of their game, test
managers gain full visibility into test plans, cases, runs and
results, as well as defects - in a single dynamic repository.
Based on their insights from this information, they can
centrally organize, measure and manage testing cycles
without compromising on scope or quality.
Visibility in monitoring test workflows – based on end-to-end

business processes - enables managers to easily identify and
remove workflow bottlenecks.
Panaya Test Center further monitors test velocity and testers’

individual workloads - so test managers can make sure
everyone meets their defined goals. Other features include
monitoring of planned vs. actual reporting by team or by
tester.

THANK YOU
www.panaya.com
About Panaya | Panaya, an Infosys company, enables organizations to accelerate

application change and continuously deliver innovation with its Enterprise Agile Delivery
Platform. Panaya provides cloud based application delivery, and test automation and
management solutions that ensure collaboration between Business and IT. Enabling
enterprise agility with faster release velocity and uncompromising quality, Panaya delivers
an optimized user experience with end to end visibility of the application lifecycle. Since
2008, 1,600 companies in 62 countries, including a third of the Fortune 500, have been
using Panaya to deliver quick quality change to enterprise applications.

Ebook - The Ideal Testing Solution - Key Capabilities You Are Probably Missing

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ebook - The Ideal Testing Solution - Key Capabilities You Are Probably Missing

Uploaded by

Copyright:

Available Formats

The Ideal Testing Solution:

4 Business Process Centricity

8 Visibility and Collaboration

11 Workflow and Process Automation

13 Testing That Mirrors Production

17 Agile Test Management & Acceleration

Experience Risk-Free Change Schedule My Demo »

A recent report based on Spiceworks‘ research* shows that while

Five important trends that are currently underpinning the testing

Experience Risk-Free Change Schedule My Demo »

The infrastructure for technical and functional testing must

‘The Age of the Customer’ is impacting testing practices as

Waterfall vs. Agile Application Delivery

Analyze Plan Design Test Build Deploy

Analyze Plan Deploy

Experience Risk-Free Change Schedule My Demo »

58% 55% 51%

40% 42% 44%

How to Meet the Demands of Enterprise Agility

This is certainly due to the increasing number of organizations

Ensure your testing practices support your organization’s agility

Introducing Panaya’s Business Process-Centric

Experience Risk-Free Change Schedule My Demo »

Experience Risk-Free Change Schedule My Demo »

With Panaya's business process-centric approach, the test manager

Experience Risk-Free Change Schedule My Demo »

In addition, the survey Test Scoping Challenges:

How to Collaborate with Visibility

Experience Risk-Free Change Schedule My Demo »

To gain the confidence to meet both time and quality standards,

To meet business requirements with speed without compromising

Introducing Business Process-Centric Planning

System-Wide Defect Tracking with Panaya Test Center

Experience Risk-Free Change Schedule My Demo »

To eliminate waste of resources and time-to-market delays due to

A testing manager can for instance, define an automatic notification

A testing project manager has full visibility into system-wide

Experience Risk-Free Change Schedule My Demo »

The survey also found that Percentage of Functional

How to Go Beyond Automation

Moreover, there is a greater, unfulfilled scope for test automation,

Experience Risk-Free Change Schedule My Demo »

›› Smart defect management. Panaya Test Center accelerates

›› Test evidence automation. With no additional effort required by

›› Effortless test case migration. With Panaya Test Center there is

Experience Risk-Free Change Schedule My Demo »

However, the new business-

believe their tests do not

(Graph no. 5). In fact, creating

Experience Risk-Free Change Schedule My Demo »

47% Tests do not accurately reﬂect real-life production scenarios

42% Regression suite is too hard to create and maintain

38% Not enough developers for test automation

35% Automation is too costly

*Percent according to challenges faced by respondents

How to Keep Test Cases Real

To accommodate the Business-IT convergence trend, choose a

Introducing Panaya Autonomous TestingSM

Experience Risk-Free Change Schedule My Demo »

Autonomous Testing℠ enables zero-touch test case creation

Once the Autonomous Testing℠ converter is enabled on production

Omega, the machine learning engine at the core of Panaya’s