Seminars on Software Testing

Here, you find information about seminars I can offer.

Contact me at hans.schaefer@ieee.org.

Testing is the last step in the software life cycle. Time pressure is well known and increasing
because too many defects are found late and may have to be repaired. Early test planning and
the use of reviews achieve a high degree of preventive defect removal. Still, at the end you
have to execute the tests, in order to measure the final product quality. The ultimate aim,
however, is that defects show up during test preparation rather than test execution.

These seminars show you “state of best industry practice” testing. You can use the techniques
right away, and even when you use only part of the techniques, you will feel the
improvement.

You also get an overview of the latest testing tools, which may help you make test work
proceed smoother and cheaper.

Seminar 1 (1 or 2 days) Methods for test design

Seminar 2 (2 days) Integration, system and acceptance testing
Seminar 3 (1 or 2 days) Managing, organizing and optimizing testing, controlling risks
Seminar 4: Risk based testing
Seminar 5: Advanced and special test methods
Seminar 6: Acceptance testing and supplier follow-up
Seminar 7: (1 or 2 days) Testing real time and embedded systems

Finally, I am able to offer ISTQB Foundation and Advanced Level seminars. See
www.istqb.org for details. (In Norwegian see www.istqb.no ).

In seminar 1, you will learn:

The basic methods for all test design: How to test data variation, logic, state transitions, use
cases and data flows. You also learn about start and exit criteria and test coverage. In the two
day version, it is augmented by discussions how to test program modules and integrate them.
This seminar is useful for ANYBODY working with testing, and, in addition, programmers.

In seminar 2, you will learn:

 How to integrate software
 How to test functions in system and acceptance testing
 How to test interactions between functions
 How to test nonfunctional characteristics (performance etc.)
 How to build up a maintainable test system
 However, the details of how to design the actual test cases are topic of seminar 1.
In seminar 3, you will learn:
 The basic ideas of modern software testing. How you can describe testing to other
involved people.
 How software testing should be organized to be as effectively as possible.
 How to identify and control project risks.
 How to monitor test progress and manage testing resources and schedules.
 Start and exit criteria and how to use them
 How to interpret test results, and be sure about your release decisions.
 What kind of technology is available to increase testing efficiency and effectiveness?
 How to use defect data to improve your testing and other software work processes.
 Test documentation templates and how to use them.
 How to tailor the general model to your own needs.

In seminar 4, you will learn:

 What is risk and how can risk be used in testing
 The basics of risk management for projects
 Test prioritization using product risk
 Managing risks in the test project itself (project risks)
 Statistical usage profile based testing
 Risk and test coverage
 Risk with corrections and changes

In seminar 5, you will learn:

 How to choose the right test technique, dependent on the character of the program under
test.
 How to cope with complex logic.
 How to optimize testing by statistical reliability modeling.
 How to maintain automated test suites.
 How to test real time embedded software.
 Test Process Improvement.
 Team selection and organization.
 What test research is doing.

Seminar 6:

 Objectives of an acceptance test

 Trouble with acceptance test
 Supplier follow-up
 Requirements follow-up
 Checking supplier functional testing
 Testing real work scenarios
 Testing nonfunctional characteristics
 How to interpret defect data to predict product performance after acceptance.
Seminar 7: "Testing embedded systems"

This seminar covers the special, additional requirements for testing real time embedded
systems. Normal testing practice is NOT covered here. Such systems are ridden by a whole
lot of extra problems, in addition to the normal software problems which testing must
address. There are memory and resource corruption, race conditions, deadlocks, live locks,
timeouts, interrupts, scheduling and the generally difficult access to the system under test.
This seminar focuses on the special problems encountered here and ways to deal with them.
The topics discussed here are not normally part of any testing seminars.

We hope you will be an active participant. The seminars have time for discussion and
questions and answers. We would like you to prepare questions on
 What would you like to know about testing
 Where do you think you should organize testing better.

There are more topics I can cover, but not part of the "standard" seminars.
Contents of the seminars

Seminar 1: Methods for designing test cases

1. Introduction to testing
What is testing?
Terminology
Conflicting goals
V-model, test phases, agile and iterative models
Black Box versus white box testing, reviews, static analysis as methods

2. The work process

Making a test plan
Detailed analysis of specifications and design
Design of the test
Making test data and environment
Running the test, checking coverage, regression testing
Evaluation and exit from testing

3. General test case design techniques

Black box test
White box test´
Statistical and random test
Data variation test
State transition test
Logic test
Data flow test
Event based test
Transaction flow test

4. Other techniques than testing

Code review
Making checklists
Static analysis
Use of assertions
Extreme programming and testing

5. Testing algorithmic programs

Detailed analysis of input and output data
Class partition
Test design (class combination or not, test data bases)
Test case design, including boundary value analysis
Every step with an exercise

6. Testing state machines

Presentation of a state machine
Typical defects
 Checking the specification with exercise
How to make a test with exercise
Issues of controllability and observability
What does this testing not cover?

7. Testing data flows, integration

Analysis of specification
Making data flow test cases
Exercise for data flow test cases

8. Issues in test running

Checking coverage
Exercise in analyzing test coverage
Rerunning tests and automation
How to automate a test
Reference to local tools (if I get your input)

9. Summary
Issues that came up
Priorities

In a two-day version, the following extra topics may be included:

1. Testing real time software

2. Formal entry and exit criteria

3. Use of automated tools to generate test environments

There will be several exercises during this seminar.

Seminar 2: Integration, system and acceptance testing

1. Introduction to higher-level test

What should be achieved?
Test model
Typical trouble

2. The general methods for black-box test design

Function based testing
Data based testing
Data flow based testing
State based testing
Event based testing
Coverage based testing
Random testing
3. Integration test
How to find and integration order
What to test, what not to test
Test case design
What to do if there is trouble
Issues with respect to change control
Coverage criteria

4. Test of real time programs (optional)

5. Functional system test

Testing end user functions
Creating scenarios
Test data selection

6. Function interaction testing

Interaction between functions executed after each other
Interaction between parallel functions
Test design

7. Nonfunctional testing
Testing installation, configuration, conversion and compatibility
Testing performance, stress, and volume
Testing reliability
Testing usability
Testing security
Testing other attributes

8. How to verify that changes and error corrections work - regression testing
Retesting during maintenance and after error repair
Maintaining tests
Automating tests

9. Acceptance testing
Customer supplier interaction
The process

10. Testware architecture for manual and automated testing

11. Test automation tools

Seminar 3: Managing, organizing and optimizing testing

1. The basic ideas of defect removal. Introduction, basic facts, terminology, test
model
2. Strategic planning of the defect removal project

3. Defect data collection

How to use defect data in order to improve your way to develop systems
Root cause analysis

4. How to be sure the product is reliable enough - start and exit criteria for testing

5. How to manage testing - Some metrics and their use

6. Testware architecture and test documentation. How to keep order in your test
material. How to tailor the new IEEE 829 standard for test documentation

7. Testing technology - software tools for test automation

8. Specifying and preparing the test environment

9. Start and exit criteria

10. How to do risk management as well as risk and profit based testing: prioritizing

11. Test Process Improvement. The TPI model. How to introduce better testing
methods in your group

Seminar 4: Risk based testing

How to fight product risk

How to manage a test project under pressure

1. What is risk

2. How to manage risk: The risk management life cycle

o Risk identification
o Risk analysis
o Risk mitigation
o Risk follow up
o Risk based replanning
o
3. How to identify risk in software project: Main risks and possible reactions
o Cost
o Scope and Quality
o Time
o Resources

4. Risk based testing: How to prioritize based on product risk

o Overall priority scheme
 o Factors determining damage
o Factors determining probability
o The difficulty to find faults
o A simple method to determine and calculate risk
o Selection of test methods and coverage according to risk
o How to do the risk based testing if you know little or nothing

5. Project risk management in test projects: Risks before, during and after the test

6. Optimizing testing using reliability engineering

- Fault density and failure rate predictions
- Statistical usage testing
- How to do this
- Problems

7. Test coverage and risk

8. Changes and corrections and risk in test

9. Summary, References

Seminar 5: Advanced and special test methods

1. An overview of the advanced methods and their practical applicability

2. Methods for testing logical combinations

Cause effect graphing
Minimizing logical combinations
Dependency Island method
Category partition method
Classification tree method
Meaningful impact strategy
Pair wise combination testing

3. Testing for very high reliability or safety

Mutation testing

4. Reliability engineering and statistical test

5. Testing real time and embedded programs (this in itself can be presented as a
one day tutorial)

6. How to design maintainable automated test suites

Seminar 6: Planning and Managing Acceptance Testing

1. Introduction: Basic principles for software development and test

 The objective of acceptance testing and supplier testing
 How testing fits into development
 The different types of testing and what kinds of problems they should find
 The need for testing even after the first release

2. The customer’s role in software development

 How projects run wrong
 How the customer is turned into a hostage
 How to prevent being hostage of the project
 Following up the supplier’s test efforts
 Preparing acceptance tests

3. Risk based test planning: How to prioritize what to test

 Why and how to use risk to control testing
 Factors that determine how much damage some feature may cause
 Factors that determine the probability of trouble
 How to use a spreadsheet to control your testing

4. The types of testing that should be done

 Testing product functions.
 Testing business flows (“soap opera test”).
 Testing product properties (performance, security, …)

5. Detailed test work: How to make test cases and data

 The most primitive method: One right, one wrong
 Systematic methods for input checking: Equivalence Class Partition - Boundary value
analysis
 Method for business logic: Condition tables
 Method for business flows: State transition testing
 Other test methods
 Exercises

6. How to check that your test is good enough, how to check supplier test
 Coverage criteria and their limits
 What you ever wanted to ask your supplier about his test

7. How to interpret test results

 Problem statistics
 Exit criteria

8. Testing later
 What about maintenance?
 How to assure that defect correction and changes don’t introduce side-effects
 How to put together a regression test
 How the supplier may help you
9. Putting together it all
 A step-by-step procedure for the whole test.

10. Discussion, Summary

Seminar 7: Testing embedded and real time systems

1. Introduction

* What are embedded systems?

* The modified V-model
* Types of testing

2. The special problems with embedded systems and how to test for them

* Many problems are hard to find by testing, and prevention would be much better.
However, testing methods can be described or tailored.
Race conditions
Data corruption
Timing
Deadlocks
Priority
Rounding problems
Long term problems
Hardware related problems
Etc.

3. Testing state machines

Many embedded systems employ state machines. This is why reviewing state machine
models and testing state machines is especially important

* How to review state machine models

* How to test them
* Some tools

4. Discussion, Summary

Other topics not covered in the general seminar material,

but possible to include in in-house seminars

- ISTQB Foundation and advanced level training

- Test Process Improvement

- Combining testing and inspections, how inspections can save time and budget
- Evaluation of software to be bought

- Web application testing

- Usability testing

- How to recruit test people and how to keep them

- Defect and failure statistics and learning from defects

- Checking your supplier's test

- Test Documentation (IEEE Standard 829)

And there is more. Please ask!

Contact me at hans.schaefer@ieee.org.