DMT UI-testing automation

practices
DMT problems

§ Neither manual nor automated

test cases cover 100% of code
§ No instrument to measure test
case’s coverage


§
§
§
Deeper view on problem

§ System tests are not flexible

§ Testing whole functionality build to
built is inaccessible and
unnecessary
§ Time spent for manual testing of
regression tests could be aimed
only on testing new functionality
§ There is no information what exactly
could be affected by particular code
change
§ There is no guaranties we test all
required peaces of code
System Tests

§ Manual test cases that aimed to cover

whole functionality and its
dependencies
§ Problem
§ Coverage: is unknown and scattered over
whole system
§ Maintainability: very low due to linkage
with several requirements
§ Execution: hard to execute because tests
executed in “one way” and pauses could
cause lost of this “test way”
§ Dependency: every step is dependent
from data used in case before or from
dump data
§ Automation: not automatable (support of
those cases will be unacceptable
System Test Case
Functional Tests

§ Atomic test cases that aimed to verify

particular functionality
§ Benefits
§ Coverage: measurable!
§ Maintainability: extremely high maintainability!
Case covers atomic peace of requirement
§ Execution: easy to execute! Every case could be
executed separately even in different days by
different test engineers
§ Dependency: test cases are independent! Every
case prepares preconditions (application, data) for
it’s run
§ Automation: extremely easy to automate! Just walk
through cases and convert it to automation
framework API language
Document for automation
Traceability

§ Traceability matrix allows linking

§ Issue for new functionality with
specification document
§ Specification document with
functional test case
§ Functional test case with automated
test script
§ “Black box” coverage assuming by
manual / automated tests
Traceability matrix
Changeability

§ New improved test cases design

allows be flexible with
continuously improved
requirements:
§ Easily find exact place in test
case/test script that should be
updated
§ Create new test case/test script
for new functionality without
affecting existing test cases
§ Measure not only code coverage
but also requirements coverage
Maintainability
Summary 1

§ Atomic test cases

§ Traceability matrix binds all
artifacts
§ Easy to support test cases
§ Easy to handle test case’s execution
results
§ Test cases are ready to be
automated
§
How to automate?
Automated Test Case

§ Pre-requisites
creation
use DMT entities
creation API
DBUnit

§
Test Case body
use DMT specific
Activities
§ forms , widgets
wrappers
WebDriver

Verificati use
§ ons TestNG

§ being ran by

§ DMT automated testing

framework
§ WebUI interaction API
§ Page object pattern
§ Pre-requisites creation API
Why DBUnit?

§ Test case’s pre-requisites

creation is significant part of
either manual or automated
approach (time consuming)
§ Solutions
§ XML based DB fixtures
§ Data that should be already in database
to pass test case
§ XML based GUI fixtures
§ Data that should be entered into form to
pass test case

Why Selenium/WebDriver?

§ Open source
§ Java support
§ Developers can help to automate test
cases
§ Developers can write functional tests
on defects
§ Cross platform/browser

Build process integration

§ You need only maven + browser

to run test cases
§ WebDriver supports:
§ IExplorer
§ Chrome
§ Firefox

Performance

§ Individual test case takes about

90 sec.
§ About 480 test cases per night
§
Summary 2

§ Atomic test cases are easy to

automate and maintain
§ Pre-requisites creation takes
milliseconds
§ Test cases can be run against
any browser without changes
How to measure?

§ Problems mentioned in the very

beginning (just to remind):
§ No guaranties that code in production
is covered by test cases
§ No instruments to measure functional
test case coverage over source
code

Rich UI based apps test coverage
tool

Manual TC
execution Browser App DB

Auto-test-cases Coverage
tool

§Aut om at ed t est cases (using WebUIint eract ion API)

or QA m anually int eract wit h applicat ion wit hin
Coverage t ool t hrough browser
§Coverage t ool t racks t he source code lines affect ed
by t hose act ivit ies
 Coverage tool – collecting coverage

Step Action

1.1:1.2 User/WebDriver activities

2.1:2.2 Communication with server

3.1 Cobertura tracks the code execution

4. 1:4.2 Dev Plugin controls the code execution on

Code Server

5.1 Emma tracks the code execution

6.1 The stop-tracking command request

7.1:7.2 Get client side coverage

8.1:8.2 Get Emma runtime coverage

9.1:9.2 Get Cobertura runtime coverage

10.1:10.2 Persist combined coverage for an activity to

DB
Code Coverage by Test Ccases

Not tested code

Test Cases
to Execute
Summary 3

§ We can measure the code

covered by test cases being ran
either manually or by
automated tool
§ We can determine the code not
covered by test cases
§ We can reduce the testing time
by executing the only test
cases against affected source
code
Q&A
Issues & problems

§ How to build pre-requisites

(database?)
§ Test execution performance
optimization (initial
loging/loading)
§ Traceability matrix has to be
integrated with RTH or other
document tracking system
§
§