Table of contents

 Why is the purpose and necessity of Software testing?

 Software systems context
 Why Software Defects Occur?
 Role of testing in software development, maintenance and
operations
 Testing and software quality
 Is there such a thing as enough testing?
 What is software testing?
 Basic software testing principles
 Whyis software testing important?
https://youtu.be/TDynSmrzpXw
Why Is Software Testing Necessary?

 Software Testing is necessary because the existence of

faults in software is inevitable. Beyond fault-detection,
the modern view of testing holds that fault-prevention
(e.g. early fault detection/removal from requirements,
designs etc. through static tests) is at least as
important as detecting faults in software by executing
dynamic tests.
 Software Systems Context
 Software Testing is context dependent:
 Business
 Education
 Medicine
 Military
 Science
 Gambling
 Entertainment
Why Software Defects Occur?
 All humans make mistakes(errors)
 Errors produce defects
 Defects are faults / bugs in the program code, or in a
document (requirement)
 Human error causes faults in any project deliverable. Faults
may be caused by requirements, design or coding errors.
 If a defect in code is executed, that might cause a
failure:
 Fail to do what it should do
 Do something it shouldn’t
Why Software Defects Occur?
 The human factor
 Humans make mistakes
 Poor training
 Time pressure
 Code complexity
 Complexity of infrastructure
 Changing technologies
Why Software Defects Occur?
 Organizational factors
 Inefficient communication
 Unclearly defined requirements
 Environmental conditions
 Radiation, Magnetism, Electronic fields, Pollution, Etc.
 These can change the hardware conditions
Definitions
 Anomaly
 Any condition that deviates from expectation based on
requirements specifications, design documents, user
documents, standards, etc. or from someone’s perception or
experience
 Error (mistake)
 A human action that produces an incorrect result
Definitions
 Bug/Defect/Fault/Problem
 A flaw in a component or system that can cause the component or
system to fail to perform its required function
 A defect, if encountered during execution, may cause a failure of
the component or system.
 Failure
 is a deviation of the software from its expected delivery or service.
 Defect/fault masking
 When one defect prevents the detection of another
What is Software Quality?
 Definition found in IEEE (Institute of Electrical and
Electronics Engineers) Standard Glossary Of Software
Engineering Terminology
 The degree to which a system, component, or process meets
specified requirements
 The degree to which a system, component, or process meets
customer or user needs or expectations
What is Software Quality?
 Software functional quality reflects how well it complies
with or conforms to a given design, based on functional
requirements or specifications. That attribute can also be
described as the fitness for purpose of a piece of software
or how it compares to competitors in the marketplace as a
worthwhile product
 Software structural quality refers to how it meets non-
functional requirements that support the delivery of the
functional requirements, such as robustness or
maintainability, the degree to which the software was
produced correctly.
The Role of Testing
 Role of testing in software development, maintenance
and operations
 Reduce the risk of problems
 Reduce long-term defect-related costs
 Contribute to the quality of the software
 Help meeting standards:
○ Contractual or legal requirements
○ Industry-specific standards
Testing and Software Quality
 Testing can give confidence in the quality of the
software if it finds few or no defects
 If defects are found, the quality increases when those
defects are fixed
 Lessons learnt from previous mistakes improve future
performance
How Much Testing is Enough?
 How much testing should be done is a matter of risk:
 Too much testing can delay the product
release and increase the product price
 Insufficient testing hides risks of errors in
the final product
So what is software testing all about?
What is Testing?
 What is testing?
 The process of exercising software
○ To verify that it satisfies specified requirements and to detect
errors
 The process of analyzing a software item
○ To detect the differences between existing and required
conditions (that is, bugs),
○ To evaluate the features of the software item
What is Testing?
 What is testing?
 The process of operating a system or component under
specified conditions
○ Observing or recording the results
○ Making an evaluation of some aspect of the system or
component
What is Testing?
 Software testing is a process used to identify the
correctness, completeness, and quality of developed
computer software.
 Software testing is an activity to check whether the
actual results match the expected results and to ensure
that the software system is defect free.
What is Testing?
 Testing is the process of establishing confidence that a program or system does what it is
supposed to (Hetzel, 1973).
 Testing is the process of executing a program or system with the intent of finding errors (Myers,
1979).
 Software Testing is the process of exercising or evaluating a system by manual or automatic
means to verify that it satisfies specified requirements or to identify differences between actual
and expected results (IEEE 610.12, 1990)
 Testing is any activity aimed at evaluating an attribute or capability of a program or system.
Testing is the measurement of software quality (Hetzel, 1984)
 Testing is demonstrating that a system is fit for purpose (Evans, et al, 1996)
 Testing is a process of planning, preparation and execution to establish the characteristics of a
software product and to determine the difference between the actual and required status (Pol and
Van Veenendaal, 1996)
 Software Testing is a process consisting of all life cycle activities concerned with checking
software and software-related work products (Gelperin and Hetzel, 1988)
 Testing is the process of exercising software to verify that it satisfies specified requirements and
to detect errors (BS7925-1, 1998)
 As the objective of a test should be to detect faults, a “successful” test is one that does detect a
fault (ISEB, 1999)
Main Test Activities
 Testing is not all about running tests, but also:
 Planning and control
 Choosing test conditions
 Designing and executing test cases
 Checking results
 Evaluating exit criteria
 Reporting on the testing process and system under test
 Completing closure activities
Main Focus in Software Testing
 Testing pursues several objectives:
 Finding defects
 Gaining confidence about the level of quality
 Providing information for decision-making
 Preventing defects
Different Viewpoints
Different Viewpoints
 Focus of software testing differ according to the point of
view:
 From Developers Point of view:
○ Are perceived as very creative (constructive) - they write code
without which there would be no system
○ Are rarely good communicators
○ Can often specialize in just one or two skills (e.g. VB, C++,
JAVA, SQL)
Different Viewpoints
 Objectives of testing differ according to the point of
view:
 From QA engineers point of view:
 Are perceived as destructive - only happy when they are finding faults!
 Usually require good communication skills, tact & diplomacy.
 Normally need to be multi-talented (technical, testing, team skills).
Debugging VS Testing
 Software Testing process
 The activity that initially finds failures in a software item
 Debugging
 The development activity that finds, analyses and removes
the cause of the failure
 Debugging is therefore an activity which supports testing, but
cannot replace testing.
 Subsequent re-testing by a tester
 Ensures that the fix does indeed resolve the failure
Verification and Validation
 Verification is the checking or testing of items, including
software, for conformance and consistency with an
associated specification. (Are we doing the job right?)

 Validation is the process of checking that what has

been specified is what the user actually wanted. ( Are
we doing the right job?)
 The Basic Seven Testing Principles
 Testing shows presence of defects
 Exhaustive testing is impossible
 Early Testing
 Defect Clustering
 Pesticide Paradox
 Testing is context dependent
 Absence of errors - fallacy
 Testing shows presence of defects

 Testing can show that defects are present

 Cannot prove that there are no defects

 Appropriate testing reduces the probability for defects

 Exhaustive testing is impossible

 All combinations of inputs and preconditions are usually

almost infinite number
 Testing everything is not feasible
○ Except for trivial cases
 Risk analysis and priorities
should be used to focus testing efforts
 Early testing

 Testing activities shall be started as early as possible

○ And shall be focused on defined objectives
 The later a bug is found – the more it costs!
 Defect clustering
4.Defect clustering (групиране)
 Testing effort shall be focused proportionally
○ To the expected and later observed defect density of modules
 A small number of modules usually contains most of the
defects discovered
○ Responsible for most of the operational failures
 Pesticide paradox

 Same tests repeated over and over again tend to loose their
effectiveness
○ Previously undetected defects remain undiscovered

 New and modified test cases should be developed

 Testing is context dependent

 Testing is context dependent

o Testing is done differently in different contexts.
o Example: Safety – critical software is tested differently from
an e-commerce site.
 Absence-of-errors fallacy

 Finding and fixing defects itself does not help in these cases:

○ The system built is unusable

○ Does not fulfill the users’ needs and expectations

Additional test principles
 Testing must be done by an independent party.
 Assign best personnel to the task.
 Test for invalid and unexpected input conditions as well
as valid conditions.
 Keep software static during test.
 Provide expected test results if possible.
How serious the QA take on their job 
 Interesting links
 http://www.devtopics.com/20-famous-software-
disasters/
 http://www.computerworlduk.com/galleries/infrastructur
e/top-10-software-failures-of-2014-3599618/#23
 No QA as a part of development process:
https://youtu.be/xtpyjPrpyX8?t=50