You are on page 1of 24

Orange County Convention Center

Orlando, Florida | June 3-5, 2014

Lessons from SAP Retail loyalty testing


Mathi Natarajan, Cognizant

Tech Solutions

Ana Paula Pacheco, Loblaw Companies Ltd

LEARNING POINTS
Frequent Testing challenges in Todays SAP Projects
TSAP Methodology and Solution Accelerators

RITE Methodology for Continuous Improvement

Todays SAP Implementation Landscape


Implementation Model
ASAP Big Bang
ASAP Waves
Agile, Iterative
Continuous Improvement

Technology Model
Best Practice / Off the Shelf
Wireframe
SOA/Plug & Play
Conventional
Delivery Model
Software Factory
Managed Services
Packaged Services
Semi-Packaged Services
SAAS

Intelligent SAP loyalty program

Loyalty invite to Customer

Customer Registers Online

Customer calls call center


to activate
Customer walks
into a store and
makes purchases

Customer earns loyalty


points
Point transactions
aggregated and sent to
predictive analysis
Customer Segmentation

Personalized rewards

Personalized Rewards
sent to Customers
mobile

Promotional
Rewards

The Big Picture Integration with SAP Retail


Article is assorted
and Items/Prices
sent to POS

Master Data

Loyalty invite to Customer

Customer Registers Online

Customer calls call center


to activate

Point transactions
Posted
POS Sales Audit
ECC Revenue and
Inventory Posting

Customer earns loyalty


points

Store receives the order

Customer walks
into a store and
makes purchases

Point transactions
aggregated and sent to
predictive analysis
Personalized rewards
Store Orders to DC
DC Pick, Pack & Ships
the product
Personalized Rewards
sent to Customers
mobile

Promotional
Rewards

Execution Model - Case Study


Software Factory

Development, System Testing

Managed Services

Development, System Testing and Hosting

Packages Services

SoA Solution, System/Integration Testing, Hosting & Support

Semi-Packaged Services Custom SoA Solution, System/Integration Testing, Hosting & Support

SAAS Services

Plug & Play Systems, No Development, Testing, & Support requirements

SAP CRM Loyalty


SAP Call Center
SAP Retail
SAP BI
POS
Enterprise Integration
SAP Mobile
Web
Predictive Analytics
Email Management System
Bank Integration

Test Approach

TSAP

RITE

TSAP Suites
Tools

Strategy

Analytics

Process Model

PM Pro

Kanban Test Scheduling*

Causal Analysis

Modular Test
Architecture*

Universal Data Migrator

Spray Testing Strategy

Application Quality Index

12 Point Inspection
System

RISE

Simulation Test Strategy*

Requirement Quality
Index

RITE*

iAuto*

Business Value Testing

Change Impact Analysis*

BA - QA

cTDMS

Pattern Based Testing*

Trend Prediction*

cTAAS

Solgen / Sol Access

EVM Dashboard*

RAPIDO / iFrame*

QIS

ADPART*
* Used in the current case study

RITE - Rapid & Intelligent Test Engineering


RITE - comprises of lean and fast delivery strategy, just in-time scheduling and
QA Analytics resulting in greater value generation

PBS
Trend & Impact
Analysis

AGILITY
Innovation &
Automation

BVT
Quality & Change
Control

KANBAN
Just In Time
Scheduling

Pattern Analysis
180

180
160

160

SAP ECC - Merch - Articles

140

140

120

SAP ECC - Merch - Pricing

100

120

80

Jul

100

60

Aug

80

40

Sep

60

20

Oct

0
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb

Nov

40

Dec
20

Sales Audit

MDM

Real Estate

Project Systems

MM

MD-Vendor

MD-Site

Retail Pricing

MD-Customer

Purchase Pricing

Assortments

MD-Articles

SRM/IP

General Ledger

Accounts Recievable

Accounts Payable

Asset Accounting

BI-Sales Audit

BI-Merch

BI-FI

ABAP

Strategy realignment
based on trend analysis

KANBAN

Just in Time Scheduling


enabling flexible intake
process as well as transparent
tracking system

Case Study - Test Challenges & Mitigations

Scope, Schedule & Budget

Requirement & Design

Environment & Release Management

Test Data Management

Delivery Model

Technology Management

Change Management

Incident Management

Problem Management & Continuous


Improvement

Challenges & Mitigation - 1


Scope, Schedule & Budget
o Delay in initial scope finalization & Frequent changes
in scope
o Schedule over run due to delay in
design/development/unit testing
o Budget constraint for regression testing
Mitigation
o Introduce Entry Exit Guardrails to limit scope
changes
o Leverage Semi-Agile Test Approach to mitigate build
delays
o Leverage Change Impact Analysis & iAUTO to
mitigate Regression testing constraints
o Build Common Regression cycles

Challenges & Mitigation - 2


Requirement & Design
o Requirement creep and frequent changes in design
o Inconsistent and incomplete requirements, which led
to multiple design changes
o No documentation or little documentation on Non
functional requirements
Mitigation
o Measure Requirement Stability Index to improve
requirement changes in future releases
o Perform Requirement assessment for Functional and
Non-Functional information's as part of Entry Gating
criteria

Challenges & Mitigation - 3


Environment & Release Management
o Saas, Hosted Services, etc impacting In-house QA
environment planning
o Challenges in Holistic release management for an
Enterprise implementation
Mitigation
o Introduce Integrated Environment planning and
scheduling process
o Use SPRAY testing approach for early detection of
environment issues
o Build / FUT Scope/dates by applications needs to
align with Semi-Agile SIT Scope/dates
o Collaborative Release and Test scheduling with
adequate contingencies
o Build Simulators to mitigate application / build
availability challenges

Challenges & Mitigation - 4


Test Data Management
o Master data quality issues due to incorrect conversion
o Sharing of master data among Marketing,
Merchandise, Supply Chain, Store, Finance, Reporting
work streams lead to huge co-ordination effort
o Data dependency across multiple applications
Mitigation
o Execute Data Validation Macros/Queries as part of
SIT/RT Entry Criteria
o Completion of Mock Conversion cycles with target
quality % before each QA cycle
o Build E2E Data Management Plan as part of E2E
Scenario creation with specific step wise callouts on
data and system dependencies

Challenges & Mitigation - 5


Delivery Model
o Highly customized SAP solution with incomplete
documentation
o Frequent churn resulting in knowledge gaps especially
for customized modules
o Geographically separated delivery teams
o Packaged vs Factory vs Managed Services model
introduces varying degrees of problems like Demand
management, Contractually diverse responsibility
matrix, Defect turnaround KPIs, FUT Quality, etc
Mitigation
o E2E Architecture expanded to include test
requirements, dependencies and data flow variations
o Leverage technology tools, flexible schedules and
process templates to mitigate geographical
constraints.
o Centralized QA Entry Exit gating criteria and Process
Control measures

Challenges & Mitigation - 6


Technology Management
o Complex end to end testing involving diverse
technologies leading to testing tool challenges
o Convoluted Non-Functional capabilities across
applications/ interfaces
o Application, Device and Infrastructure testing in play
o High number of infrastructure variations leading to
QA coverage limitations and QA result extrapolation
challenges
Mitigation
o Leverage hybrid testing tools and accelerators to
mitigate test tool limitations.
o Build Visual E2E Non-functional matrix for
applications/interfaces to identify gaps
o Master Test strategies should cover device and
infrastructure parameters to enable holistic strategy
o QA result extrapolation models to be fine-tuned
based on Pilot Go-Lives

Challenges & Mitigation - 7


Change Management
o Multiple vendors and Diverse delivery models pose a
change management night mare.
o Application Support KPIs and varying business needs
influence process deviations
o Expedited CRQs bypass all change control processes
Mitigation
o Leverage QA change control (Poka Yoke technique) to
mitigate production risk due to frequent changes
deployed into production
o Publish Weekly QA KPIs measuring upstream and
downstream quality including Dev Quality/Schedule,
QA Quality/Schedule/Capacity, Environment
Availability, etc. This would increase stakeholder
awareness including schedule.

Challenges & Mitigation - 8


Incident Management
o Infrastructure and Application stability/ maintenance
activities would cause high impact production
incidents which require faster resolutions including
QA activities.
o Consumer focussed applications warrant emergency
responses due to Business sensitivity & qualitative
impact which in-turn requires faster testing timelines
o Security patches are governed by legal procedures
requiring faster resolution times including QA
Mitigation
o Build Automation Modules to execute on-demand
including test data prep and validation
o Build Simulators to expedite critical path testing
o Build automated regression suite specific for security
and infrastructure patches
o Leverage Impact Analysis tools for risk based testing

Challenges & Mitigation - 9


Problem Management & Continuous Improvement
o Consumer focussed applications warrant constant fine
tuning of functionalities leading to frequent KTLO
enhancements
o Heterogeneous application landscape requires
frequent patching / maintenance activities leading to
regular QA testing activities
o Frequent changes to production causes application /
infrastructure stability issues increasing incident
testing requests
Mitigation
o Leverage pattern based analysis to right size QA
readiness based on incident prediction
o Right size automation requirements to meet
predicted demand
o Cross Skill associates to expedite integration testing
o Increase innovation through Gamification
o Leverage Incident Build Synergy and QA Execution
synergies for handling more requests

KEY LEARNINGS

Learn technology related challenges and its impact to SAP testing


How to apply hybrid testing techniques to fit changing needs of SAP
projects
Understand the RITE techniques like Pattern analysis, Kanban, etc for
warranty & maintenance phases
QA Analytics for successful SAP program management

THANK YOU
THANK YOU FOR PARTICIPATING

Please provide feedback on this session by completing


a short survey via the event mobile application.
SESSION CODE: 2511
For ongoing education on this area of focus,
visit www.ASUG.com

FOLLOW US