1.

1 Testing Philosophy

If you are a student now, or were in the past, you are quite familiar with the word test,
and probably hate it. Understanding the teachers point of view will help. The teacher sets a
domain of knowledge for testing, called the course syllabus. It may be the contents of a book,
class notes, lectures, or some (arbitrary!) combination of all those. Next, comes the testing
method. The teacher asks questions and analyses the response, perhaps by matching answers to
correct ones from the book. The quality of such a test system depends upon how well the test
questions cover the syllabus. In VLSI testing also, one should know the specification
(synonymous to the course syllabus) of the object being tested and then devise tests such that if
the object produces the expected response then its conformance to the specification can be
guaranteed.

Returning to our student analogy, since no one has infinite time, the number of questions
must be limited, and they should be cleverly devised. The teacher now makes certain
assumptions. Certain typical errors, ones that the student is likely to commit, are assumed.
Questions are devised especially to uncover those errors and, if the students answers are correct,
the teacher grants the benefit of doubt, showing confidence in the implicit error model.
Electronic testing also uses fault modeling and tests are generated for the assumed fault models.
In testing, successful experience with a fault model gives it credibility, and eventually people
expect reliability when a high percentage of the modelled faults is tested.

Finally, remember that, if you fail, you must repeat the course. This is similar to redesign
and remake in our Algorithm: Perfect. Of course, you can do better by asking your teacher,
right at the beginning, about (1) the course syllabus and (2) error models (i.e., what you will be
tested for), and then plan your studies to succeed. In VLSI, that is called design for testability.

Example 1.1 Testing of students. In a course on xyzeeology, 70% of the students deserve to pass.
We will call them pass quality students. Assuming that the number of students in the class is
large, we will study the test process using a statistical basis. For a randomly selected student
from the class, we define the following events:

PQ: student is pass quality P: student passes the test
FQ: student is fail quality F: student fails the test

In our example, assuming that only pass/fail grades are awarded, the remaining 30% students
are of fail quality, i.e., as we know, it is impossible to design a perfect test. However, our
teacher does quite well and 95% of pass quality students actually pass the test. This is
represented by conditional probabilities, and similarly, the test correctly fails 95% of the fail
quality students. A reader not familiar with basic concepts of the probability theory may wish to
consult any basic text on the subject [677]. The diagram of Figure 1.1 illustrates the state
transition caused by the test. The initial state, on the left, consists of all students in one group.
The test separates them into two groups shown on the right as passed and failed. Sizes


Figure 1.1: A pass/fail test.


of the passed and failed groups created by the test are given by the total probabilities of passing
and failing, respectively. The total probability of passing is,



Similarly, the total probability of failing is found to be Notice that the original group had 70%
pass quality students, and the test has passed only 68%. Obviously, some pass quality students
have been failed. But, are all passed students of pass quality?

We will examine the conditional probability Prob (FQ\P) of a student belonging to the
fail quality subgroup, given that he or she has passed. The joint probability of events FQ and P
is given by:



and, therefore:



where Prob(P) comes from Equation 1.1. Equation 1.3 is known as Bayes rule [677] and is
commonly used for drawing inferences from statistical data. We obtain That is, 2.2% of passed
students are of fail quality. We will call this the teachers risk. The teacher can reduce this
risk by making the test more difficult, decreasing P (P\FQ) closer to 0. However, that test can
potentially fail a few more pass quality students. So, let us examine the students risk.

Applying the Bayes rule, we obtain



This shows that 11% of failed students should have passed. We call this the students risk.
Obviously, a pass quality student would not like to end up in the failed group. To reduce the
students risk, the probability P (F\PQ) will have to be reduced. This can be done by making the
test easier. That will, however, also pass a few more fail quality students, worsening the quality
of the passing batch. Thus, teachers risk and students risk are opposing criteria, requiring
practical compromises. An ideal test, that minimizes both risks, should be so tuned that it fails
no pass quality student and passes no fail quality student. Devising such a test is no mean task
for our teacher.

Testing of electronic systems differs only slightly from the above scenario. A student
may pass by correctly answering most, but not necessarily all, questions on the test. If a small
number of answers is wrong, the teacher gives the student the benefit of doubt, for he or she may
be having a bad day, or else could even learn correct answers in the future. There is no such
benefit of doubt for a VLSI chip. Being inanimate, it cannot be having a bad day and certainly
cannot learn. So, even a single incorrect test response will fail a VLSI chip. However, electronic
tests are not perfect either. They may not cover certain faults and some bad chips will pass. We
may also use some non-functional tests to prevent those bad chips from passing. Non-
functional tests do not execute the specified function an example is the quiescent current test
discussed in Chapter 13. These tests can, in turn, fail some good chips. For VLSI, failing of good
chips by tests is known as yield loss, which increases the cost of manufacturing.

In electronic testing, the teachers risk is synonymous to the consumers risk. It is related
to bad chips being shipped to the consumer. The students risk in the above example is similar to
the manufacturers risk, since failing of good devices increases the cost. We will examine these
aspects of electronic testing in Chapter 3.

1.2 Role of Testing

If you design a product, fabricate and test it, and it fails the test, then there must be a cause
for the failure. Either (1) the test was wrong, or (2) the fabrication process was faulty, or (3) the
design was incorrect, or (4) the specification had a problem. Anything can go wrong. The role of
testing is to detect whether something went wrong and the role of diagnosis is to determine
exactly what went wrong, and where the process needs to be altered. Therefore, correctness and
effectiveness of testing is most important for quality products (another name for perfect
products.)

If the test procedure is good and the product fails, then we suspect the fabrication process,
the design, or the specification. If all students in a class fail then it is often considered the
teachers failure. If only some fail, we assume that the teacher is competent, but some students
are having difficulty. To select students likely to succeed, teachers may use prerequisites or
admission tests for screening. Distributed testing along a product realization process catches the
defect-producing causes as soon as they become active, and before they have done much
damage. A well thought out test strategy is crucial to economical realization of products.


Figure 1.2: VLSI realization process (a naive version.)

The benefits of testing are quality and economy. These two attributes are not independent
and neither can be defined without the other. Quality means satisfying the users needs at a
minimum cost. A good test process can weed out all bad products before they reach the user.
However, if too many bad items are being produced then the cost of those bad items will have to
be recovered from the price charged for the few good items that are produced. It will be
impossible for an engineer to de- sign a quality product without a profound understanding of the
physical principles underlying the processes of manufacturing and test.