Bllp04 Ij Author

SOFTWAREPRACTICE AND EXPERIENCE
Softw. Pract. Exper. 2003; 34:915948
Prepared using speauth.cls [Version: 2002/09/23 v2.2]
Generation of test sequences

from formal specifications:
GSM 11-11 standard
case-study
Eddy Bernard1 , Bruno Legeard2 , Xavier Luck1 , Fabien Peureux2
1
SchlumbergerSema Montrouge Product Center 50, avenue Jean Jaur`es - 92542 Montrouge
Cedex, France
2
Laboratoire dInformatique de lUniversite de Franche-Comte (LIFC) - CNRS 16, route de
Gray - 25030 Besancon cedex, France
SUMMARY
This paper presents results of a case-study on generating test cases for a fragment of
the Smart Card GSM 11-11 standard. The generation method is based on an original
approach using the B notation and techniques of Constraint Logic Programming with
sets. The GSM 11-11 technical specications were formalised with the B notation. From
this B specication, a system of constraints was derived, equivalent to this formal model.
Using a set constraint solver, boundary states were computed, and test cases were
obtained by traversing the constrained reachability graph of the specications.
The purpose of this project was to evaluate the contribution of this testing environment,
called B-TESTING-TOOLS, in an industrial process on a real-life size application, by
comparing the generated test sequences with the already used and high quality manuallydesigned tests. This comparison enabled us to validate our approach, and showed its
eectiveness in the validation process of critical applications: the case-study gives a wide
coverage (about 85%) of the generated tests compared to the pre-existing tests, and a
saving of 30% of test design time.
key words: specication-based testing, B notation, Smart Card GSM 11-11 standard, Constraint
Logic Programming with sets
Correspondence
to: Laboratoire dInformatique de lUniversit

e de Franche-Comt
e - CNRS 16, route de Gray
- 25030 Besancon cedex, France (tel:(33) 381 666 663 fax:(33) 381 666 450)
E-mail: bernard e@montrouge.sema.slb.com
E-mail: legeard@lifc.univ-fcomte.fr
E-mail: luck@montrouge.sema.slb.com
E-mail: peureux@lifc.univ-fcomte.fr
Contract/grant sponsor: SchlumbergerSema Montrouge Product Center
c 2003 John Wiley & Sons, Ltd.

Copyright
Received 14 October 2003

Revised 19 November 2003
916
1.
E. BERNARD ET AL.
Introduction
Today, functional black-box testing still remains a signicant stage of the software life cycle.
The necessity of testing a product is bound to the need for checking that the features
described in the specications are implemented exactly as expected. In application domains of
critical software, i.e. software, avionics, railway signalling, automotive control and management
systems, secure smart card applications..., more than 50% of the development is devoted to
the validation of the software product. Testing is performed by applying test cases to an
implementation under test, by making observations during the execution of the tests, and by
subsequently assigning a verdict on the correct execution of the implementation. The design
of black-box tests in industry is seldom supported by tools, and relies on the expertise of
validation engineers. Consequently, there is considerable subjectivity exercised in created test
sets. The manual design of functional black-box tests is generally based around an informal
review of the system specication.
The need to oer better methods and tools for functional black-box testing has given rise to
a large amount of research on generating tests from formal specications e.g. [1, 2, 3]. Formal
methods of specication, and particularly model-oriented notations such Z [4], VDM [5] and
B [6], allow a high-level abstract formalisation of the expected behaviour of the system under
test. These notations are well suited for test generation because the expressiveness of setoriented logic constructs and the denition of an explicit model help both test case generation
and oracle synthesis. Thus, these formal notations are the basis of various proposals to more
or less automatically generate tests from the formal model, see for example [7, 8, 9, 10, 11].
In [12, 13, 14], we presented a new method for automated test generation from B abstract
machines using Constraint Logic Programming [15], called b-testing-tools . From the B
formal model of the functional requirements of the system to be tested, an equivalent system
of constraints is derived, and then, each operation of the specication is partitioned through
a translation into Disjunctive Normal Form. Using this partition, boundary goals and then
boundary states are computed with a specic solver, called CLPS-B [16, 17, 18], that uses
Constraint Logic Programming with sets. The CLPS-B solver is then used to build test cases
by traversing the constrained reachability graph of the specications. This technique has the
advantage of avoiding as a preliminary the construction of the complete reachability graph.
Finally, the formal specications are used as oracles to determine the expected output for a
given input. This test generation approach is supported by an environment called b-testingtools [19]. The b-testing-tools environment provides automated support for the design
and construction of test sets. Also the rationale used to create specic test sets is made explicit.
The purpose of this paper is to show, in an industrial validation context, the practicability
and eectiveness of the b-testing-tools approach to generating functional test cases. The
subject of this industrial case study is the GSM 11-11 standard [20]. The GSM 11-11 standard
application had the advantages, on the one hand, that it was nontrivial and common in the
smart card industry today and, on the other hand, that SchlumbergerSema already had a
The
SchlumbergerSema company is the leading supplier of Smart Card technology.

Copyright
Prepared using speauth.cls
GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY
917
mature, high-quality manually generated test set. The comparison of tests shows that more
than 85% of the tests used by the validation team are covered by those generated using the
b-testing-tools environment, while nearly 50% of the generated test cases supplement these
pre-existing tests. Moreover, the test design time analysis reveals a saving of time about 30%.
Our presentation is organised as follows:
Section 2 introduces the B method and notation.
Section 3 gives a short overview of the test generation method with the b-testing-tools
environment.
Section 4 presents the informal specications and the relevant B formal model of the
GSM 11-11 standard.
Section 5 describes the computation of the boundary goals and the constitution of the
boundary states.
Section 6 presents the generated test sequences computed using the boundary states and
the B model.
Section 7 provides a summary of the results and discusses the main lessons learned from
the industrial case study.
Section 8 describes related work.
Section 9 summarises the contribution of the b-testing-tools environment and
suggests further work.
2.
B formal method and notation
This section presents the B Abstract Machine Notation, used to formally specify the systems
functional requirements.
2.1.
Presentation of the B method
Formal methods are mostly used in safety-critical domains to produce high quality software
or hardware. The B method was designed for this purpose by Abrial in the late 1980s [6]. Its
basic construct is the abstract machine which contains encapsulated data and operations to
manipulate these data. It extends earlier set-based specication notations such as V DM [5] and
Z [4] by placing more emphasis on composition of machines, renement [21], code generation
and tool support [22]. B has already been successfully used for large industrial projects [23].
Figure 1 shows the use of the B method in the software life cycle.
2.2.
The data model of an abstract machine
The data of an abstract machine are specied in a data model by means of a number of
mathematical concepts such as sets, relations, functions, sequences and trees. This model, also
called the static model, presents the various data of the B machine, which are used by the
operations, and the invariant properties that the state variables must follow. The data model
Copyright
918
E. BERNARD ET AL.
Incremental Modifications
Requirement Analysis
and
Safety Properties
Technical Specifications
B Method
Abstract Machine
Formal
Proofs
Refinement
Code Generation
Figure 1. B method in the software life cycle
is also formally described by means of a number of clauses. In the GSM 11-11 B machine (see
Appendix A), we nd eight clauses, which are the following:
The machine clause introduces the name of the machine.
The sets clause introduces given sets. These sets, be they enumerated or abstract, all
denote independent types.
The constants clause lists the various constants of the machine.
The definitions clause introduces some useful, possibly parameterised, aliases, as well
as useful denitions to facilitate the reading of the specications.
The properties clause introduces the properties of the machine, which take the form
of various conjoined predicates involving the constants and the given sets.
The variables clause introduces the state variables (components) of the machine.
The invariant clause introduces the invariant properties of the machine. The invariant
consists of a conjunction of a number of predicates.
The initialisation clause makes possible the assignment of initial values to the variables
of the machine. The assignment must be proved to uphold the invariant. The initialisation
takes the form of a substitution.
Copyright
2.3.
919
The operation model of an abstract machine
The dynamics of an abstract machine is expressed through its operations. The role of an
operation, which is executed by the computer, is to modify the state of the abstract machine
or to reveal part of that state to an observer. Each operation must maintain the state invariant.
The specications of the operations of an abstract machine are expressed in a last clause, called
operations, as non-executing pseudo-code that does not contain any sequencing or looping. In
this pseudo-code each operation has a pre-condition and an atomic action. The pre-condition is
a necessary condition for invoking an operation. The nal state of the variables after execution
of the operation are dened using the notion of substitution.
2.4.
Testing framework with B-TESTING-TOOLS
The next section introduces a short overview of the test generation method with the btesting-tools environment. This method is based on models formalised by means of the B
notation and the concept of abstract machine, but is dierent to the traditional process of the
B method. The b-testing-tools approach consists of formalising the technical specications
of requirements by using the most abstract level of B modelling. This abstract model is used
to generate functional tests for the validation of an implementation developed using a classical
(design and coding) rather than a B method life cycle. The abstract model expresses in a
precise and detailed way the functional requirements of the product to be tested. The model
is developed from informal specications with the help of the validation team. The model is
animated [17] to provide further condence that it captures what was intended. For simplicity,
the b-testing-tools environment do not deal with composition of B machines. This is not
too restrictive; in various case-studies (process scheduler [17], protocol T=1 [24] or GSM
application discussed in this paper), this practice was convenient and gave good results.
The objective of the case-study presented here was to evaluate the coverage and quality of
the test generation process with the b-testing-tools environment. For that, the GSM 11-11
application is very suitable because the SchlumbergerSema manually-designed tests constitute
a very mature and high-quality test suite.
The manually-designed test suite was developed iteratively by the validation team using
a classical design process based on the informal review of the requirements documents (in
our case, the GSM 11-11 Standard). As the project was the rst trial of formal model-based
test generation at SchlumbergerSema Smart Card R&D Division, we focused on the textual
comparison between the two test suites: manually-designed and automatically-generated.
Translating the test cases generated by the b-testing-tools approach into executable test
scripts is outside the scope of this paper.
3.
Short overview of the test generation method
The rst step of our test generation method is to elaborate a B formal model of the system to be
tested. The abstract machine, the basic element of the B specications, characterises a machine
which has a hidden state (consisting of several state variables) and a number of operations.
Copyright
920
E. BERNARD ET AL.
A user is able to activate these operations in order to modify the state of the machine. The
behaviour of a B abstract machine can be described in terms of a sequence of operations,
where the rst is activated from the initial state of the machine. This sequence of operations
is called a trace. However, if the pre-condition of an operation is false, substitution cannot
establish any post-condition. The traces in this case are of no interest, since it is impossible
to determine the state of the machine. We dene a test case to be any legal trace, i.e. trace
where all preconditions are true. A test case corresponds to a sequence of states presenting
the value of each state variable after each operation invocation. The submission of a legal
trace is a success if all the values returned by the concrete implementation during the trace
are equivalent (through a function of abstraction) to the values returned by its specication
during the simulation of the same trace (or included in the possible values if the specication
is non-deterministic).
In order to select the traces to be submitted, our approach consists of testing the operations
with boundary inputs when the system is in a boundary state. A boundary state is a state
where at least one state variable has a value at an extremum -minimum or maximum- of its
subdomains (see Section 3.1). The trace constituting the test case is also divided into four
parts. According to the ISO9646 standard [25], the four successive parts are dened as follows
(see Figure 2):
1. Preamble: the part of a test case taking the system from its initial state to a state in
which the test purpose can be achieved (i.e. from initial state to a boundary state).
2. Body: the part of a test case containing all the events essential to achieve some test
purpose (i.e. invocation of one operation that becomes the tested operation). In this
way, each operation is executed with input boundary values.
3. Identication: the part of a test case consisting of invocations whose aim is to determine
certain observable aspects of the system at the end of the test body. In addition to the
possible output data returned by the operation of the body, the output values returned
by this third part enable us to assign a verdict: the results obtained by simulation of
the specications are those expected during the execution of the same sequence on the
implementation.
4. Postamble: the part of a test case taking the system from the nal state of the
identication part to the nal state of the test case. This last part is used to reach
a specic state in order to link several new test cases together.
The operations are partitioned into update and observation operations: update operations
are used in the preamble, body and postamble, and observation operations in the identication
part. Update means that such an operation modies state variables. Observation operations
return information about the state variables (but do not modify them). The test purpose
is to test each update operation from each boundary state of the B abstract model. The
generation method is dened by the following algorithm, where {bound1 , bound2 , ..., boundn }
and {op1 , op2 , ..., opm } respectively dene the set of all boundary states and the set of all the
update operations of the B model:

Copyright
921
Preamble invocation
preamble
identification
Body invocation
body
Identification invocation
postamble
Postamble invocation
Figure 2. Test case constitution
for i=1 to n
% for each boundary state
preamble(boundi );
% invocations to reach the boundary state from the initial state
for j=1 to m
% for each update operation
body(opj );
% computation of the operation as tested operation
identification;
% invocations to observe the current state
postamble(boundi ); % invocations to return to the current boundary state
endfor
postamble(init);
% invocations to return to the initial state
endfor
Figure 3 shows how the state space is traversed during this generation method. Such a
collection of test cases, that are linked together, denes a test sequence. The next subsections
describe each step of the b-testing-tools generation method.
3.1.
Boundary state and preamble
A boundary goal is a predicate which describes a subset of the system state, where at least
one variable has an extremum value (maximum or minimum). A boundary state is a state
verifying one or several boundary goals. Since the b-testing-tools method aims to test every
operation at every boundary state, the boundary goals act as test objectives.
The state variables, that are not valued by the current boundary goal, take a value of their
domain during a constrained simulation of the specications. Indeed, the technique can be
seen as a traversal of the reachability graph whose nodes represent the constrained states
built during the simulation, and the transitions symbolise the activation of an operation. This
processing is performed with a single specic solver, devoted to the B notation and using
Constraint Logic Programming with sets, called CLPS-B [16, 17, 18]. Simulation is performed
from the initial state to a state satisfying the required boundary goal. In order to improve
convergence of the resolution procedure, a Best-First search [26, 27] is implemented.
Copyright
922
E. BERNARD ET AL.
INITIAL STATE
BOUNDARY
STATE 1
BOUNDARY
STATE 2
...
BOUNDARY
STATE n
preamble invocations
FINAL STATE
OF BODY 1.1
FINAL STATE
OF BODY 1.2
...
FINAL STATE
OF BODY 1.m
body invocation
identification invocations
FINAL STATE OF
IDENTIFICATION
FINAL STATE OF
IDENTIFICATION
FINAL STATE OF
IDENTIFICATION
postamble invocations
Figure 3. Traversal of the state space during the test sequence generation
3.2.
Body
The constrained simulation is also used to execute the operation to be tested at the boundary
state. A body is generated for each boundary value of input parameters. Thus, from the same
preamble and boundary state, several dierent bodies can be obtained with the same operation,
but with dierent input values.
3.3.
Identication
The operations selected to be called for the observation are then activated. This part is used
to verify the current value of state variables. The computed operations, that generally have
no decision points, already belong to the model and are available in the implemented system.
Copyright
3.4.
923
Postamble
When the test sequences are executed on the implementation, in order to avoid having to
manually initialise the system into a specic state (initial or boundary state) after each test
case, it is useful to compute a postamble to reach this kind of state. If no postamble is able to
reach the initial or boundary state, a complete reset is performed, and the preamble is used
to reach the boundary state. This part is computed using Best-First search again.
4.
GSM 11-11 standard: the B formal model
The GSM (Global System for Mobile communication) is the rst of the second generation
mobile phone systems: i.e. the rst system involving fully digital features. It was launched in
1979 by the World Administrative Radio Conference (WARC), and has evolved technically
for many years before the ocial opening of the rst commercial GSM networks in Europe
around 1993. From 1991 onwards, it has been adapted to create very similar new systems
(DCS 1800 and then DCS 1900 in which only the radio interface diers). Nowadays, the
GSM is a worldwide standard and the only one accepted in Europe. Its evolution is managed
by the European Telecommunication Standard Institute (ETSI). Although most of the GSM
operators are european, the GSM is used in 109 countries worldwide, including Africa, Asia and
Australia. Its main competitors are the North American and Japanese mobile communication
systems.
The main interest of mobile networks is of course the users mobility. To ensure security,
GSM features an optimised radio interface with digital communications over it being secured
by cryptographic algorithms. The SIM contains the ciphering keys, authentication algorithms
and also information about the subscribers identity, and services available to him. This is a
card belonging to and preset by the GSM operator, but remaining in the users mobile phone.
The commands performed by the mobile phone to the SIM and all the features available from
it are specied in the GSM 11-11 standard [20].
The GSM 11-11 standard also denes the interface between the SIM and the ME . During
the communication, the SIM is passive: it only answers the requests sent by the ME. The
applications contained in the ME access and modify the les of the SIM through dened
functions and by respecting access conditions of each le. The abstraction level of the tested
kernel concerns only the functions of the GSM 11-11 without formalising the upper layer (the
application layer), or the lower layer (the communication protocol between the SIM and the
ME). In the industrial case-study, we proceed with 11 operations and a subtree of 8 data
les. Due to the industrial condency, in this article, we only present a smaller fragment (6
operations and 4 data les).
The les of the SIM and the selected functions are now described. These functions make it
possible to select a le and insert a personal identication number (PIN). According to the
Subscriber
Mobile
Identication Module.
Equipment.

Copyright
924
E. BERNARD ET AL.
Figure 4. Tree Structure of considered SIM les
validity of this code and the relevant access of les, the user may read the currently selected
le. The rights corresponding to the PIN code may be blocked if the authentication process
fails three times, and can be unblocked if a corresponding personal unblocking key (PUK) is
entered.
4.1.
Selected les
We consider two kinds of le dened in the GSM 11-11 standard:

The transparent Elementary les (EF) are les which contain only data. An EF le with
a transparent structure consists of a sequence of bytes .
The Dedicated les (DF) are directories, which can contain other DF or EF. They dene
the tree structure of the les. A special DF le, called Master File (MF), denes the root
of the tree.
The tree structure and the les considered in this case-study were limited to two DFs (of
which MF is one) and four EFs with the tree structure shown in Figure 4. The current directory
of the system is dened by a DF while its current le may be dened by an EF (in this case,
the current directory is its DF father). Initially, the current directory of the system is MF and
no current le is selected. It should be noted that there is no current le after the selection of
a DF. The selection of a le (DF or EF) is performed according to the following rules:
The
Every le which is a child of the current directory can be selected.

Every DF le which is a sibling of the current directory can be selected.
The parent of the current directory can be selected.
The current directory can be selected.
The MF can always be selected.
GSM 11-11 standard also includes sequences of records and cyclic EF, which are not considered in this
study.

Copyright
925
Table I. Read access conditions
4.2.
Files
Read access conditions
ef iccid
NEVer
ef lp
ALWays
ef imsi
CHV
ef ad
ADM
Security aspects
Every EF has its own specic read access conditions (no le access condition is currently
assigned by GSM to the DF). The relevant access condition must be fullled before the
requested action can take place. These conditions are described in Table I. Four types of
access condition can be found in the standard:
ALW ays: the action can be performed without any restriction,
N EV er: the action cannot be performed over the SIM\ME interface (the SIM may
perform the action internally),
CHV : the action is possible only if one of the following conditions is fullled:
a correct PIN code has already been presented to the SIM during the current
session,
the PUK code has been successfully presented during the session.
The CHV level, once satised, remains valid until the end of the GSM session as long
as the corresponding PIN code remains unblocked, i.e. after three consecutive wrong
attempts, not necessarily in the same card session, the access rights previously granted
by the PIN or PUK code are lost immediately.
ADM : Allocation of these levels and respective requirements for their fullment are the
responsibility of the appropriate administrative authority. In the selected kernel of the
standard, ADMinistrator access is always denied.
4.3.
Selected commands
We present six operations among those dened in the GSM 11-11 standard. This selection was
motivated by the wish to generate test sequences to elucidate in more detail the security of
the read access conditions. The chosen operations were the following:
Card
Holder Verication.

Copyright
926
E. BERNARD ET AL.
RESET : this function consists in selecting MF as the current le, and to suppress the
access conditions granted since the beginning of the session. This function makes possible
the simulation of consecutive card sessions.
SELECT F ILE: this function selects a new le according to the conditions dened in
Section 4.1 above.
READ BIN ARY : this function reads the data from the current EF. This function can
be performed only if the read access condition for this EF is satised.
V ERIF Y CHV : this function veries the PIN code presented by the ME by comparing
it with the relevant one stored in the SIM. The verication process is allowed only if
CHV is not blocked.
If the presented PIN code is correct, the number of remaining CHV attempts shall be
reset to its initial value of 3 and the CHV access condition is satised.
If the presented PIN code is false, the number of remaining CHV attempts shall be
decremented. After 3 consecutive false PIN presentations, not necessarily in the same
card session, the CHV shall be blocked and the CHV access condition can never be
fullled until the UNBLOCK CHV function has been successfully performed.
U N BLOCK CHV : this function, by verifying the presented PUK code, unblocks CHV
which has been blocked by three consecutive wrong PIN code presentations. This function
may be performed whether or not CHV is blocked, but is not allowed if UNBLOCK CHV
is itself blocked.
If the presented PUK code is correct, the value of the PIN code, presented together with
the PUK code, replaces the currect stored CHV PIN value, the number of remaining
UNBLOCK CHV attempts is reset to its initial value of 10 and the number of remaining
CHV attempts is reset to its initial value of 3. After a successful unblocking attempt,
the CHV access condition level is satised.
If the presented PUK code is false, the number of remaining UNBLOCK CHV attempts
shall be decremented. After 10 consecutive false PUK code presentations, not necessarily
in the same card session, the UNBLOCK CHV shall be blocked. A false PUK code shall
have no eect on the status of the CHV itself.
ST AT U S: this function returns the value of the current le, the current directory, and
the current value of all the counters of remaining attempts.
Each function, except the ST AT U S operation, always returns a status word. This response,
which has the form of a hexadecimal code, takes one of the values described in Table II.
4.4.
The B formal model
The informal specications of the GSM 11-11 kernel, introduced in the previous subsections,
were used to write the B formal specications. The relevant B abstract model, presented in
Appendix A, is now briey described.
4.4.1.
Sets
Six sets are dened to model the simplied kernel:

Copyright
927
Table II. Potential response codes to commands

Responses
codes
Descriptions
9000
- Normal ending of the command
9400
- No EF selected
9404
- File not found
9804
- Access condition not fullled

- Unsuccessful CHV verication, at least one attempt left
- Unsuccessful UNBLOCK CHV verication, at least one attempt left
9840
Unsuccessful CHV verication, at least one attempt left

Unsuccessful UNBLOCK CHV verication, at least one attempt left
CHV blocked
UNBLOCK CHV blocked
P ERM ISSION : this represents the various read access rights.

V ALU E: this denes the two Boolean values true and f alse.
F ILES: this denes the set of SIM les and directories to be considered.
BLOCKED ST AT U S: this is used to model the various states of blocking.
CODE: this represents the potential codes, which can be used to verify an access
condition. The codes were limited to four possible values to facilitate the reading of
the generated test sequences.
DAT A: this abstract set denes the set of all the data, which can be written in the les.
4.4.2.
Constants
Five constants are dened in the B machine:

M AX CHV : this is the maximum number of consecutive unsuccessful PIN attempts
that will be tolerated.
M AX U N BLOCK: this is the maximum number of consecutive unsuccessful PUK
attempts that will be tolerated.
F ILES CHILDREN : this is used to dene the tree structure of les with parent\child
pairs.
P ERM ISSION READ: this describes the read access condition for each le with
le\condition pairs.
P U K: this denes the constant value of the PUK code.
Copyright
928
E. BERNARD ET AL.
4.4.3.
Denitions
Five denitions complete the sets and constants already dened:

COU N T ER CHV : a range of non-negative numbers. The number of further consecutive
unsuccessful PIN attempts that will be tolerated is in this range.
COU N T ER U N BLOCK: a range of non-negative numbers. The number of further
consecutive unsuccessful PUK attempts that will be tolerated is in this range.
M F : this set denes the master le.
DF : this contains the set of all the directories except the master le.
EF : this presents the set of all the transparent les.
4.4.4.
Variables
Ten state variables are used in the formal model:

current f ile: this indicates the current le.
current directory: this indicates the current directory.
counter chv: this denes the number of PIN attempts with a wrong code, which are
tolerated to verify CHV.
counter unblock: this denes the number of PUK attempts with a wrong code, which
are tolerated to verify UNBLOCK CHV.
pin: this denes the PIN code.
permission session: this determines the access rights currently granted with
condition\Boolean pairs.
blocked chv status: this denes the status of blocking of CHV.
blocked status: this denes the blocking status of UNBLOCK CHV.
data: this presents the data in each le with le\data pairs.
The invariant and initialisation clauses introduce respectively the invariant properties
of the state of the machine and the initialisation of each state variable. The six operations,
previously described, are presented in the operations clause. Table III shows, for each
operation, the description of its input and output parameters.
4.5.
Validation and verication of the B abstract model
During the formalisation process, we use the animation capabilities of the b-testing-tools
environment [17] to simulate the execution of the model. It allowed the interactive validation
of the model with regard to the requirement specications (i.e the informal specications of
the GSM 11-11 standard).
Moreover, the model was veried using the proof tools of the atelier b [22]. In the B
method, at the abstract level, proof obligations dene what must be proved to ensure that
operations preserve invariants (subject to operation pre-conditions being met). For the GSM
11-11 B abstract machine shown in Appendix A, 377 proof obligations were generated: 376
were automatically proved and 1 was satised using the interactive proof tool.
Copyright
929
Table III. Input and output parameters of operations

Operations
Input Parameters
Output Parameters
RESET
none
none
SELECT F ILE
f f : le to be reached
sw: status word
READ BIN ARY
none
dd: read data

sw: status word
V ERIF Y CHV
code: code of CHV
sw: status word
U N BLOCK CHV
code unblock:
code of UNBLOCK CHV
new code: new code of CHV
sw: status word
ST AT U S
none
cd: current directory

cf : current le
cc: counter of CHV
cuc: counter of UNBLOCK CHV
The two next sections describe the application of the test generation method on the
standard GSM 11-11 B specications: rstly the computation of boundary states, and then
the computation of the test sequences themselves.
5.
Boundary state computation
To compute boundary goals, rstly a partition analysis is performed on each operation, by

transforming its postcondition into disjunctive normal form (DNF), then conjoining it with
the precondition and the state invariant. This ensures that each eect of each operation is
tested independently.
Secondly, boundary goals are obtained by searching for maximum and minimum solutions
to each of these eect predicates. The maximization/minimization is performed using a default
ordering which is chosen according to the types of the state variables. A boundary goal is also
a predicate that describes a subset of the state space, represented by a set of constraints.
Finally, each boundary goal is instantiated into a boundary state obtained by traversal of
the state space during the preamble computation. A more detailed presentation is given in [13].
5.1.
Domain partitioning
For test generation purposes, we transform the postcondition of each operation of the B model
into DNF (each before/after predicate of the DNF is conjoined with the precondition and the
state invariant). After this transformation, each disjunct corresponds to one behaviour of the
Copyright
930
E. BERNARD ET AL.
Table IV. Eect predicates of the READ BIN ARY operation

N
Before predicates
After predicates
P1
current f ile EF current f ile =
sw := 9400
dd := 1
P2
dp dp P ERM ISSION current f ile EF

current f ile = permission session(dp) = true
P ERM ISSION READ[current f ile] = {dp}
sw := 9000
AN Y f f W HERE
f f current f ile T HEN
dd := data(f f ) EN D

current f ile = permission session(dp) = true
current f ile = permission session(dp) = f alse
current f ile = permission session(dp) = f alse
sw := 9804
dd := 1
P3
P4
P5
relevant operation. These disjuncts are called eect predicates. For each operation, the set of
all eect predicates corresponds to the set of all behaviours of the operation.
For example, if an operation contains a conditional substitution IF condx THEN S1 ELSE
S2 where condx is an atomic condition on the variable x, and S1 and S2 are two atomic
substitutions, two eect predicates are created: condx S1 and condx S2 .
Table IV summarises the eect predicates obtained from the partitioning into DNF
of the operation READ BIN ARY . The eect predicates computed from the operations
V ERIF Y CHV U N BLOCK CHV and SELECT F ILE are shown in Appendix B. The
two other operations (STATUS and RESET), presenting one simple (only substitutions) eect
predicate, are not shown. Although it does not appear in the various tables, the state invariant
is assumed to hold for all the identied eect predicates. The 22 generated eect predicates
are then used to generate boundary goals and hence boundary states.
5.2.
Boundary goal computation
On the basis of the partition analysis, one or more boundary goals are calculated from
the before-state of each eect predicate. The boundary goals are obtained by searching for
maximum and minimum solutions to each eect predicate. The maximization/minimization is
performed on the state variables of the predicates as follows:
Copyright
931
If the variable is a set of non-constant cardinality, two types of boundary values are
computed: the rst one is equal to the smallest cardinality set, and the other to the
largest cardinality set of each subdomain.
If the variable is a set of constant cardinality, all the potential values are boundary values.
If the variable is equal to a pair or a set of pairs, the boundary values are recursively
computed on the antecedent and on the image of each pair.
If the variable is a numeric atom, the maximum and minimum values of each subdomain
are boundary values.
If the variable is an non-numeric atom, by default, all the potential values are boundary
values (a validation engineer can force the choice of a single value).
The generated boundary goals may be unreachable. Within this framework, a partial test of
reachability is used. On the one hand, it is based on checking the invariant properties of the
specications: each boundary goal not satisfying the conditions of the invariant clause of the
specications is not taken into account. On the other hand, during the preamble computation,
the paths to reach a boundary state from the initial state are bounded by length.
The Boundary goals, generated from the eect predicates of the operation READ BIN ARY
are shown in Table V. The other boundary goals (obtained from SELECT F ILE,
V ERIF Y CHV and U N BLOCK CHV eect predicates) can be found in Appendix C.
Some boundary goals are found to be redundant and are withdrawn from the generation
by the validation engineer (for example, boundary goals BG7 , BG8 and BG9 are respectively
subsumed by BG10 , BG11 and BG12 ). Finally, 17 boundary goals are used to generate test
cases (the selected goals are identied in the various tables by heavy type).
6.
Test case computation
The test cases, introduced by the concept of traces, are built according to the constrained
reachability graph of the specications and the collection of boundary goals. A trace is
represented by a sequence of invocations. Each invocation is dened by the activation of
an operation of the formal specication. The values of input and output parameters of the
operation, and the values of state variables following its activation, are always indicated.
We now illustrate the various stages of the approach with a specic example. The particular
boundary goal is current f ile = ef imsi (chv, true) permission session (BG3 in the
Table V). The approach for other boundary goals follows a similar pattern.
6.1.
Computation of the preamble
The rst stage of the method is to generate a path from the initial state to each boundary
state of the specication. As shown previously, the computation of the invocations, making it
possible to reach a boundary state from the initialisation, is performed during the computation
of the boundary state itself. This technique has the advantage of avoiding, as a preliminary,
the construction of the complete reachability graph: the preamble is computed during the
construction of the boundary state. Thus, for each boundary goal, a path from the initial state
Copyright
932
E. BERNARD ET AL.
Table V. Boundary goals from the READ BIN ARY operation

Predicates
Boundary Goals
P1
current f ile =
BG1
P2
current f ile = ef lp (always, true) permission session

current f ile = ef imsi (chv, true) permission session
BG2
BG3
P3
current f ile = ef iccid (never, f alse) permission session

(chv, true) permission session
(always, true) permission session
current f ile = ef imsi (chv, f alse) permission session
current f ile = ef ad (adm, f alse) permission session
(chv, true) permission session
P4

P5

(chv, f alse) permission session
(adm, f alse) permission session
(never, f alse) permission session
(chv, f alse) permission session
BG4
BG5
BG6
BG7
BG8
BG9
BG10
BG11
BG12
to the relevant boundary state is performed. A consequence of this preamble computation

is that state variables which are not already assigned a value by the boundary goal, are
assigned a reachable value of their domain. Only one instance is generated. A kind of uniformity
hypothesis over the preamble domain is therefore assumed. As argued below, this may be too
strong and require revision in future work.
Table VI shows the path generated by CLPS-B to reach, from the initial state, a state of
the specication that satises the boundary goal current f ile = ef imsi (chv, true)
permission session. Thus, from the initial state, the three consecutive invocations of the
operations V ERIF Y CHV (a1 ), SELECT F ILE(df gsm) and SELECT F ILE(ef imsi)
lead the machine into a state where the variables current f ile and permission session have
the expected values.
Copyright
933
Table VI. Path from the initial state into the boundary state
Operations
6.2.
Input
data
Output
data
Modied data
from the previous state
V ERIF Y CHV
a1
9000
permission session = [..., (chv, true)]
SELECT F ILE
df gsm
9000
current directory = df gsm
SELECT F ILE
ef imsi
9000
current f ile = ef imsi
Computation of the body
The purpose of the body is to test, for a given boundary state, all the update operations,
with all boundary values of their input variables. The process of boundary analysis for input
variables is similar to that for state variables. The maximization/minimization is performed
on the input variables of the predicates in a similar way to that for the state variables (see
Section 5.2).
This selection of input boundary values makes possible the exploration of all the decision
points of the operation (according to their possible dierent input boundary values, extracted
from the eect predicate already generated and used to compute the boundary goals). In this
way, as many invocations were computed with the same operation as its input parameters
present dierent boundary values.
From the boundary state generated from the boundary goal current f ile = ef imsi
(chv, true) permission session, each operation is executed as many times as it has dierent
possible input boundary values, and each execution represents one test. Table VII lists all the
operations with their respective input boundary values to be activated from the boundary
state.
Table VIII shows the computation of one possible test : the execution of the operation
V ERIF Y CHV with a wrong code as input parameter.
6.3.
Computation of the identication
We have chosen the ST AT U S and READ BIN ARY operations to be executed for the
observation (no operation is added to the B model to perform observations). The identication
part can consist of a single operation or a sequence of operations.
In the GSM 11-11 standard case-study, observations are computed with the ST AT U S
and READ BIN ARY operations. In the example (see Table IX), the computation of the
ST AT U S operation shows that the counter of attempts to verify the CHV access condition,
was decremented, while the computation of the READ BIN ARY operation ensures that the
permission on the CHV access is always accepted.
Copyright
934
E. BERNARD ET AL.
Table VII. Presentation of the operations to be computed in the body

References
Computed operations
Input parameters
Values
reset
none
verif y chv
code
correct code
verif y chv
code
wrong code
unblock chv
code unblock
new code
correct code
any code
unblock chv
code unblock
new code
wrong code
any code
select f ile
ff
master le
select f ile
ff
current directory
select f ile
ff
DF sibling
select f ile
ff
EF child
Table VIII. Example of body

Reference
Operations
V ERIF Y CHV
Input
data
Output
data
Modied data
from the boundary state
a2
9804
counter chv = 2
Table IX. Invocations of the test case identication

Operation STATUS
Output variables
Values
Operation READ BINARY
current directory
df gsm
Output variables
current f ile
ef imsi
sw
9000
counter chv
dd
d3
counter unblock chv
10

Copyright
Values
935
Table X. Return path to the boundary state

Operations
V ERIF Y CHV
Input
data
Output
data
Modied data
from the last state
a1
9000
counter chv = 3
Table XI. Return path from the boundary state into the initial state
6.4.
Operations
Input
data
Output
data
RESET
none
none
Modied data
from the boundary state
current f ile =
current directory = mf
permission session = [..., (chv, f alse)]
Computation of the postamble
When the test sequences are executed on the concrete system, in order to avoid having to
initialise manually the system into the initial or boundary state after each test case (the
testing process would be seriously slowed down), it is useful to compute a path that returns
to this kind of state. So, after each identication part, a sequence of operations is computed
to return to the relevant boundary state. Table X shows the sequence obtained on the current
example to return to the boundary state from the current state.
In the same way, when all the bodies have been computed, a path to return to the initial
state from the current boundary state is performed. Table XI shows the postamble used in
this case on the current example.
7.
Comparison results
This section deals with the comparison between the test cases generated with b-testingtools and the existing manually-designed test cases. The comparison results were obtained
on a larger part of the GSM 11-11 standard than presented in this paper: the model contained
11 operations and 8 data les with many access conditions of CHV type and CHV update
rights. This model gave rise to the generation of 58 boundary goals and 42 dierent boundary
Copyright
936
E. BERNARD ET AL.
states . From each boundary state, about 24 dierent invocations were performed in the body,
and about 1000 test cases were generated [28]. Computing time was about two hours (on Sun
Ultra Sparc - Processor 200MHz - Memory 256MB).
7.1.
Test coverage
We denote by Man-T the set of all SchlumbergerSema manually-designed test cases for the
GSM 11-11 standard considered kernel, and by Gen-T the test cases generated with btesting-tools for the same subset of GSM 11-11 standard. Man-T is a very mature and highquality test suite, used for several years by the SchlumbergerSema company, and continuously
improved. Each test of Man-T is, in general, multi-purpose, with at least twenty operations.
Man-T includes about 100 dierent tests. A test case from Gen-T contains less operation
invocations than a test case from Man-T. Indeed, the length of Gen-T tests lies between 2
and 13 operation executions (more generally, the Gen-T tests are built with an average of 7
operations).
7.1.1.
Comparison method
Three comparisons can be done between manually-designed (Man-T) and automaticallygenerated (Gen-T) tests:
The rst comparison is lead by the ne analysis of the GSM 11.11 technical specication
by people. This study gave out a number of functional basic requirements, i.e. sentences
or small sentence sets dening the expected behaviour of the system in given situations.
The entire reading of the specication then results in a list of basic requirements, which
can be used as coverage reference. The reading of the Man-T test plan enables one to tag
each of theses requirements as manually tested (this is the classical method for manual
construction of functional coverage matrices). Then, keeping in mind that requirements
can be formalised as (input/expected output) pairs, the reading of Gen-T tests enables
one to tag the requirements as automatically tested. At the end, the requirement list
contains four requirement types: never tested, manually only tested, automatically only
tested and tested by both methods. Coverage statistics are then computed with the basic
requirement as their unit.
The second comparison is quite simple. It consists in listing all the automaticallygenerated behaviours and checking within the list for the ones also covered by the Man-T
tests. Statistics are then computed with the behaviour as their unit.
The third comparison is the contrary of the second one: it consists in listing the manuallydesigned tests and checking within the list for the ones also covered by the Gen-T tests.
Statistics are then computed with the behaviour as their unit.
For example, six basic requirements can be extracted from the section 8.9 (Verify CHV) of
the GSM 11.11 version 7.2.0 standard [20]. They are all covered by 20 Man-T tests and by
single boundary state may contain several boundary goals.

Copyright
937
112 Gen-T tests. Although the number of tests is dierent, we can assert that the functional
coverage is identical according to the GSM 11.11 reference. But among the 112 generated
behaviours, only 28 (25%) are also covered by the Man-T tests. Among the 20 Man-T tests,
16 (80%) are also covered by the Gen-T tests (the missing four tests are repetitions of other
tests with command order changes).
More generally, the comparison of tests on the overall fragment of the GSM 11.11 standard
shows that Man-T and Gen-T both cover the GSM 11.11 basic requirements, Gen-T covers
85 % of Man-T tests and 50 % of Gen-T supplements Man-T. These comparison results are
now presented and discussed as follows:
the automatically-generated test cases in common with manually-designed test cases,
the automatically-generated test cases not in the manually-designed test cases,
the manually-designed test cases not automatically-generated.
7.1.2.
Gen-T test cases in common with Man-T
Each manually-designed test case that belongs to this class usually corresponds to several
automatically-generated test cases. Performing only one operation after the boundary state
makes the generated test cases more modular and atomic than the manually-designed test
cases, which are already optimised. So, several generated test cases correspond to one manuallydesigned test. This explains in particular why the automatic procedure can generate a number
of tests ten times as great as those manually-designed, and why about 500 of the (short) Gen-T
tests correspond to 87 of the (long) Man-T tests.
7.1.3.
Gen-T test cases not in Man-T
The generation of test cases which do not appear in Man-T (about 500) has two main causes:
Use of boundary values from a set with non-numeric values: when the boundary values
of a variable are computed, if it is not a numeric atom, all these potential values are
considered as boundary values, and each of these values is also used to generate boundary
states. Similar test cases are also performed, and, in most cases, only one appears in the
Man-T tests. The typical example is the pin variable, which has each possible code
from the B model as boundary value. Thus, each value is tested while, at rst sight, it
seems that they are all equivalent, and so only one test of the code would be sucient.
Nevertheless, this type of test makes it possible to eliminate any anomaly concerning the
common properties of the values of the same variable.
Execution of all the update operations from each boundary state: the operations that
do not impact and use, directly or indirectly, the variables of the relevant boundary
goal are executed from the corresponding boundary state. In this case, the activation
of these operations may be not needed. An obvious example is the execution of the
U N BLOCK CHV operation after each boundary state, even if the corresponding
boundary goal does not concern CHV or UNBLOCK CHV attempts. However, there
Copyright
938
E. BERNARD ET AL.
Table XII. Comparison of the design time

Automatic
design
Human
design
Test plan redaction
6 days
Test implementation
24 days
Total
30 days
B specications writing
12 days
Test generation
and coding
6 days
Total
18 days
is a need to verify the real independence of the processed variables and to check any side
eects.
7.1.4.
Man-T test cases not in Gen-T
15 test cases from the Man-T were not generated with the automatic procedure. It is mainly
because test cases involving consecutive sessions were not generated. On the one hand, the
search heuristics, used to reach the boundary state from the initial state, gave only one path
and the RESET operation did not appear in this path. This operation is the only one making
it possible to simulate consecutive card sessions. As the RESET operation did not appear in
the generated test preamble, this kind of test case was not generated. On the other hand, in
this study, the number of operations executed after a boundary state was limited to one. If
more operations were executed, these kinds of test case could be generated by activating the
RESET operation before the last operation of the body. More generally, the choice to take
into account only one path to reach a boundary state and to perform only one operation from
the boundary state, prevents us from obtaining some judicious test cases.
7.2.
Design cost
Table XII presents an estimation of design time of both Man-T and Gen-T test suites. A
comparison between our method and a human design is shown.
The evaluation of the work duration to dedicate for the design of manual tests has been done
as for industrial feasibility study: statistics are computed based on actual projects management
cases using internal criteria. Then, based on it, it is possible to evaluate the workload induced
by the validation of the dened scope of the GSM 11.11 (a few man.days per command, for
this kind of simple command and for an experienced validation engineer).
This test design time analysis shows that the specication-based generation method saves
about 30% of time, including the formalization time.
Copyright
7.3.
939
Method assessment
The comparison between the Man-T and the Gen-T tests shows that a large coverage rate,
about 85%, of Man-T are also generated by b-testing-tools, while nearly 50 % of GenT test cases supplement Man-T. Although the obtained results are very encouraging, it is
possible to increase the quality of the generated test sets. The reasons explaining the lack
of automatically-generated test cases in relation to those manually-designed, show the main
improvements:
to handle the boundary goals: it would be necessary to be able to eliminate certain
boundary goals, which prove in fact to have only little interest. The reverse is
also necessary: to have the possibility of adding boundary goals to those generated
automatically.
to propose several preambles: the choice of the path to reach a boundary state from
the initial state is currently the rst path discovered by the CLPS-B solver. However,
according to the path, the direction of the test can be dierent. The assumption of
uniformity on the domain of path may be too strong. This proposition involves the
creation of testing objectives according to particular needs required by the system to be
tested.
to test several operations activated consecutively: once the boundary state is reached,
the body is built using only one update operation. However, certain test cases of Man-T
relate to the consecutive activation of two update operations.
7.4.
Position in the industrial validation process
The B formal model of the GSM 11.11 standard was a collaborative design between validation
engineers from SchlumbergerSema and researchers from the LIFC laboratory. This B formal
model was validated using interactive model review and animation techniques with the btesting-tools environment. In fact, the B notation appears easily understood by engineers
and the use of the b-testing-tools simulation capabilities to improve the formal model
validation was practical. We believe that validation engineers should always be involved in
the generation process, and should be able to interact with the environment in the ways
summarised above.
8.
Related work and discussion
The b-testing-tools approach is related to the emerging Model-Based Testing

techniques [29, 30]. Model based test automation is a very active research area. These tools
use as input a formal model of the system under test and allow the validation engineer to drive
the test generation automated process.
Formal description languages based on Labelled Transition Systems (LTS), Extended Finite
State Machines (EFSM) or Abstract State Machines (ASM) have been widely considered [31].
Copyright
940
E. BERNARD ET AL.
In these approaches, the test generator may support a variety of existing transition based
strategies like state coverage, path coverage, constrained path coverage, all transition pairs,
etc. The main limitation of these approaches comes from the state explosion problem and the
test case explosion problem [32]. Several proposals address these limitations, for example on
the basis of the formalisation of reachable properties guiding the test generation or by reducing
the test suites using individual requirements selection [33].
On the other hand, set-oriented model based formal notations, like VDM, Z or B, have been
extensively studied for test generation purpose. Most approaches [7, 34, 10, 9] use a partition
analysis of the operation to build a Finite State Automaton - FSA - corresponding to an
abstraction of the reachability graph denoted by the specication. Test cases are then generated
using the same kinds of coverage criteria as used by the LTS/EFSM/ASM approaches, with the
same limitations (test case explosion). Moreover, the transformation of the formal model into
an abstract FSA introduces several fundamental problems, such as the non-discovery problem
and again the state explosion problem [31].
The b-testing-tools approach proposes another way, which is also based on the partition
analysis of the operation, but avoids the a priori construction of the FSA. The test generation
is then conducted by Boundary Goal calculation, guiding the computation of the preambles.
Moreover, the Constraint Logic Programming technology [18], used as a basis for animation
purposes, allows reasoning on so called constraint states, represented by constraint stores,
which denote sets of valued states. This reduces the combinatorial explosion.
Another unique feature of b-testing-tools is its strong boundary-testing orientation.
Despite the enormous popularity of boundary-value testing strategy for black-box testing in
the software practitioner guides, currently this approach has not been widely investigated as an
automated specication-based test generation approach. In the b-testing-tools approach,
boundaries are computed both for state variables for each eect predicate and for the input
variables, in the computation of the body part of the test. In this, our approach is related to
domain testing [35, 36].
In the GSM 11-11 case study, two kinds of variable make it possible to compute boundaries:
integer variables and set variables on which we compute the set cardinality (the integer
variables are counter chv, counter unblock and a set variable is current f ile). Actually, the
boundary analysis process generalises the boundary computation and applies it to all kinds
of variable: enumerated by taking a value, and boolean by considering 0-1 as bounds of an
interval.
Finally, the b-testing-tools approach is based on specic CLP techniques. There is
an increasing use of constraint solving in software verication techniques as well in modelchecking [37] and in test generation. For code-based test generation, Gotlieb et. al. [38] present
a framework where a system of constraints is built from a Static Single Assignment form of
the source code (for C programs). Solving the generated constraints allows to identify code
elements (statements or decisions) to be exercised in particular ways. Meudec [39] uses CLP
for symbolic execution of Ada code in order to generate tests. In specication-based test
generation, Marre et. al. [40] interpret LUSTRE specications in terms of constraints over
boolean and integer variables and solve them to generate test sequences. Pretschner et. al. [41]
translate System Structure Diagrams and State Transition Diagrams into Prolog rules and
constraints to allow symbolic execution of the specications and thus test generation. They
Copyright
941
applied also their approach on smart card software [42]. Van Aertryck et. al. [10] use DNF
and constraint solving to generate an FSA and test sequences from a B-like specication, but
this is not fully automated. Mostly, these techniques use existing constraint solvers (Boolean
and nite domains in general). In b-testing-tools, due to the particular features of the setoriented B notation, we developed our own solver to handle constraints over sets, relations
and mappings. This solver augments the capabilities of and co-operates with the integer nite
domain solver of SICStus Prolog [43].
9.
Conclusions and future work
This paper summarises the results of a case-study. The purpose of this project consisted in
generating test cases for the Smart Card GSM 11-11 standard, and was to evaluate, in an
industrial process, the contribution of the generation method on a real life size application.
The test case generation technique is dierent from the traditional renement process of the B
method. It is based on an extraction of boundary goals from the abstract model by automated
partition analysis. A test case is composed of four parts:
the
the
the
the
the
preamble used to reach a boundary state (a state verifying a boundary goal) from
initial state of the B machine,
body composed of the invocation of the operation to be tested,
identication part used to assign a verdict,
postamble used to reach a specic state in order to link test cases together.
A specic solver of constraints, CLPS-B, reduces the B abstract machine to a constrained

system, calculates boundary goals and then boundary states, and nally computes the
generation of test cases by constrained animation. The principal advantages and originality of
our approach lie in the B-specic constraint logic programming technology used in CLPS-B,
in the computation of traces during the constrained simulation (not requiring the construction
of a reachability graph or nite state automaton of the specications), and in the technique of
indirect observation of the states using the output values returned by the operations invoked
in the identication part of the test case.
A thorough comparison of the generated test collection with the manually-designed tests
enabled us to validate our approach, and showed its eectiveness in the validation process
of critical applications. In the same way, this case-study shows possible improvements. It
demonstrated that our procedure should not be completely automatic - human input is needed.
The testing engineer should interact with the tool, in particular in the choice of the boundary
goals to be considered and in the choice of preamble.
Thus, our test generation method makes possible the improvement and partial automation
of the generation of functional tests. The testing engineer is not excluded from the testing
process; he writes the B formal model, and drives the test generation (choice of boundary
goals, choice of preambles, etc). This approach helps validation engineers eciently generate
better functional tests. Complete automation is not the aim. A project currently underway is
investigating the generation of functional test sequences for a Java Card Virtual Machine [44].
This should enable us to study the proposed improvements and evaluate their impact. A new
Copyright
942
E. BERNARD ET AL.
feature is being considered in this new case-study: the automatic production of executable test
applets from generated test sequences. This new experiment will contribute to the development
of a prototype for distribution [45].
ACKNOWLEDGEMENTS
The authors wish to acknowledge Mark Utting from The University of Waikato (New-Zealand) and
Tim Miller from The University of Queensland (Australia) for their detailed reading and helpful
suggestions in the preparation of this paper. We sincerely appreciate the clarifying comments of the
referees on the earlier version of this paper.
REFERENCES
1. M.J. Balcer, W.M. Hasling, and T.J. Ostrand. Automatic generation of test scripts from formal
test specications. In Proceedings of 3rd Symposium on Software Testings, Analysis and Verification
(TAV89), pages 210218, Key West, December 1989. ACM SIGSOFT.
2. D.A. Carrington and P. Stocks. A tale of two paradigms: formal methods and software testing. In
Proceedings of the 8th Z User Meeting (ZUM94), pages 5168, Cambridge, June 1994. Springer Verlag.
3. M-C. Gaudel. Testing can be formal too. In Proceedings of the 6t h Joint Conference on Theory and
Practice of Software Development (TAPSOFT95), volume 915 of LNCS, pages 8296, Aarhus, Denmark,
May 1995. Springer Verlag.
4. J.M. Spivey. The Z notation: A Reference Manual. Prentice-Hall, 2nd edition, 1992. ISBN 0 13 978529
9.
5. C.B. Jones. Systematic Software Development Using VDM. Prentice-Hall, 2nd edition, 1990.
6. J-R. Abrial. The B-BOOK: Assigning Programs to Meanings. Cambridge University Press, 1996. ISBN
0 521 49619 5.
7. J. Dick and A. Faivre. Automating the generation and sequencing of test cases from model-based
specications. In Proceedings of the International Conference on Formal Methods Europe (FME93),
volume 670 of LNCS, pages 268284. Springer Verlag, April 1993.
8. P. Stocks and D.A. Carrington. Test templates: a specication-based testing framework. In Proceedings
of the 15th International Conference on Software Engineering (ICSE93), pages 405414, Baltimore,
Maryland, May 1993. IEEE Computer Society Press.
9. R. Hierons. Testing from a Z specication. The Journal of Software Testing, Verification and Reliability,
7:1933, 1997.
10. L. Van Aertryck, M. Benveniste, and D. Le Metayer. CASTING: a formally based software test generation
method. In 1st IEEE International Conference on Formal Engineering Methods (ICFEM97), pages 99
112, 1997.
11. S. Behnia and H. Waeselynck. Test criteria denition for B models. In Proceedings of the World Congress
on Formal Methods (FM99), volume 1708 of LNCS, pages 509529, Toulouse, France, 1999. Springer
Verlag.
12. B. Legeard and F. Peureux. Generation of functional test sequences from B formal specications Presentation and industrial case-study. In Proceedings of the 16th International Conference on Automated
Software Engineering (ASE01), pages 377381, San Diego, USA, November 2001. IEEE Computer Society
Press.
13. B. Legeard, F. Peureux, and M. Utting. Automated Boundary Testing from Z and B. In Proceedings of
the International Conference on Formal Methods Europe (FME02), volume 2391 of LNCS, pages 2140,
Copenhagen, Denmark, July 2002. Springer Verlag.
14. B. Legeard, F. Peureux, and M. Utting. A comparison of the BTT and TTF test-generation methods.
In Proceedings of the International Conference on Formal Specification and Development in Z and B
(ZB02), volume 2272 of LNCS, pages 309329, Grenoble, France, January 2002. Springer Verlag.
15. J. Jaar and J-L. Lassez. Constraint Logic Programming. In Proceedings of the 14th Symposium on
Principles of programming Languages (POPL87), pages 111119, Munich, Germany, January 1987. ACM
Press.

Copyright
943
16. F. Ambert, B. Legeard, and E. Legros. Constraint Logic Programming on Sets and Multisets. In
Proceedings of the ILPS94 Workshop on Constraint Languages/Systems and their Use in Problem
Modeling, pages 151165, Ithaca, New York, November 1994.
17. F. Bouquet, B. Legeard, and F. Peureux. Constraint Logic Programming with sets for animation of B
formal specications. In Proceedings of the CL00 Workshop on (Constraint) Logic Programming and
Software Engineering (LPSE00), London, UK, July 2000.
18. F. Bouquet, B. Legeard, and F. Peureux. CLPS-B A constraint solver for B. In Proceedings of
the ETAPS02 International Conference on Tools and Algorithms for the Construction and Analysis
of Systems (TACAS02), volume 2280 of LNCS, pages 188204, Grenoble, France, April 2002. Springer
Verlag.
19. F. Ambert, F. Bouquet, S. Chemin, S. Guenaud, B. Legeard, F. Peureux, N. Vacelet, and M. Utting.
BZ-TT: A tool-set for test generation from Z and B using constraint logic programming. In Proceedings
of the CONCUR02 Workshop on Formal Approaches to Testing of Software (FATES02), pages 105120,
Brn
o, Czech Republic, August 2002. INRIA Technical Report.
20. European Telecommunications Standards Institute, F-06921 Sophia Antipolis cedex - France. GSM 11-11
V7.2.0 Technical Specifications, 1999.
21. E. Sekerinski and K. Sere. Program development by refinement - Case studies using the B method.
Springer Verlag, 1999. ISBN 1 85233 053 8.
22. Clearsy, Europarc de Pichaury 13856 Aix-en-Provence Cedex 3 - France. Atelier B Technical Support
version 3, May 2001. http://www.atelierb.societe.com.
23. P. Behm, P. Desforges, and J.M. Meynadier. METEOR : An Industrial Success in Formal Development.
In Proceedings of the 2nd International Conference on the B method (B98), volume 1393 of LNCS, pages
2945, Montpellier, France, April 1998. Springer Verlag.
24. J. Julliand, B. Legeard, T. Machicoane, B.Parreaux, and B.Tatibouet. Specication of an Integrated
Circuit Card Protocol Applications using the B method and Linear Temporal Logic. In Proceedings of the
2nd International Conference on the B method (B98), volume 1393 of LNCS, pages 273292, Montpellier,
France, 1998. Springer Verlag.
25. ISO. Information Processing Systems, Open Systems Interconnection. OSI Conformance Testing
Methodology and Framework ISO 9646, 1998.
26. A. Pretschner. Classical search strategies for test case generation with Constraint Logic Programming.
In Proceedings of the CONCUR01 Workshop on Formal Approaches to Testing of Software (FATES01),
pages 4760, Aalborg, Denmark, August 2001. BRICS.
27. S. Colin, B. Legeard, and F. Peureux. Preamble computation in automated test generation using
Constraint Logic Programming. In Proceedings of UK-Test Workshop, York, UK, September 2003.
28. B. Legeard, F. Peureux, and J. Vincent. Automatic generation of functional test patterns from a formalized
smart card model - application to the GSM 11-11 specication (condential). Technical Report TR-01/01,
LIFC - University of Franche-Comt
e and Schlumberger Montrouge Product Center, 2001.
29. A. Hartman. AGEDIS - Model Based Test Generation Tools. http://www.agedis.de/documents/
ModelBasedTestGenerationTools\_cs.pdf, 2002.
30. I.K. El-Far and J.A. Whittaker. Model-based software testing. Encyclopedia of Software Engineering,
1:825837, 2002.
31. W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Generating Finite State machines from Abstract
State Machines. In Proceedings of the International Symposium on Software Testing and Analysis
(ISSTA02), volume 27, pages 112122, Rome, Italy, July 2002. ACM SIGSOFT.
32. G. Friedman, A. Hartman, K. Nagin, and T. Shiran. Projected State machine Coverage for Software
Testing. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA02),
volume 27, pages 134143, Rome, Italy, July 2002. ACM SIGSOFT.
33. B. Vaysburg, L.Tahat, and B. Korel. Dependence Analysis In Reduction of Requirement Based Test
Suites. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA02),
volume 27, pages 107111, Rome, Italy, July 2002. ACM SIGSOFT.
34. H.M. H
orcher and J. Peleska. Using formal specications to support software testing. Software Quality
Journal, 4(4):309327, 1995.
35. E.J. Weyuker and T.J. Ostrand. Theories of program testing and the application of revealing subdomains.
The journal of IEEE Transactions in Software Engineering, 6(3):236246, May 1980.
36. B. Beizer. Black-Box Testing: Techniques for Functional Testing of Software and Systems. John Wiley
& Sons, New York, USA, 1995.
37. Y. Choi, S. Rayadurgam, and M.P.E. Heimdahl. Automatic abstraction for model checking software
systems with interrelated numeric constraints. Software Engineering Notes, 26(5):164174, September

Copyright
944
E. BERNARD ET AL.
2001.
38. A. Gotlieb, B. Botella, and M. Rueher. A CLP framework for computing structural test data. In
Proceedings of the First International Conference on Computational Logic (CL00), pages 399413,
London, UK, July 2000. Springer Verlag.
39. C. Meudec. ATGEN: Automatic test data generation using constraint logic programming and symbolic
execution. The Journal of Software Testing, Verification and Reliability, 11(2):8196, 2001.
40. B. Marre and A. Arnould. Test Sequence generation from Lustre descriptions: GATEL. In Proceedings
of the 15th International Conference on Automated Software Engineering (ASE00), pages 229237,
Grenoble, France, 2000. IEEE Computer Society Press.
41. A. Pretschner and H. L
otzbeyer. Model-Based Testing with Constraint Logic programming : First Results
and Challenges. In Proceedings of the 2nd ICSE Workshop on Automated program Analysis, Testing and
Verification (WAPATV01), pages 19, Toronto, May 2001.
42. J. Philipps, A. Pretschner, O. Slotosch, E. Aiglstorfer, S. Kriebel, and K. Scholl. Model-Based Test Case
Generation for Smart Cards. In Proceedings of the 8th International Workshop on Formal Methods for
Industrial Critical Systems (FMICS03), pages 168182, Trondheim, Norway, June 2003. Also Electronic
Notes in Theoretical Computer Science Volume 80, Elsevier Science.
43. Swedish Institute of Computer Sciences. SICStus Prolog 3.8.7 manual documents, October 2001.
http://www.sics.se/sicstus.html.
44. Sun microsystems. Java Card 2.1.1 Virtual Machine Specification, May 2000. http://java.sun.com/
products/javacard/javacard21.html\#specification.
45. The BZ-TT web site. http://lifc.univ-fcomte.fr/~bztt, 2005.

Copyright
945
APPENDIX A. The B abstract model of a fragment of GSM 11-11 standard

MACHINE GSM
SETS
F ILES = {mf, df gsm, ef iccid, ef lp, ef imsi, ef ad};
P ERM ISSION = {always, chv, never, adm};
V ALU E = {true, f alse};
BLOCKED ST AT U S = {blocked, unblocked};
CODE = {a1 , a2 , a3 , a4 };
DAT A = {d1 , d2 , d3 , d4 }
CONSTANTS
F ILES CHILDREN ,
P ERM ISSION READ,
M AX CHV ,
M AX U N BLOCK,
P UK
DEFINITIONS
M F == {mf };
DF == {df gsm};
EF == {ef iccid, ef lp, ef imsi, ef ad};
COU N T ER CHV == 0..M AX CHV ;
COU N T ER U N BLOCK CHV == 0..M AX U N BLOCK
PROPERTIES
F ILES CHILDREN (M F DF ) F ILES
F ILES CHILDREN = {(mf, df gsm), (mf, ef iccid), (df gsm, ef lp), (df gsm, ef imsi),
(df gsm, ef ad)}
P ERM ISSION READ EF P ERM ISSION
P ERM ISSION READ = {(ef iccid, never), (ef lp, always), (ef imsi, chv), (ef ad, adm)}
M AX CHV = 3
M AX U N BLOCK = 10
P U K CODE
P U K = a3
VARIABLES
current f ile,
current directory,
counter chv,
counter unblock chv,
blocked chv status,
blocked status,
permission session,
pin,
data
INVARIANT
current f ile EF
card(current f ile) 1
current directory DF M F
counter chv COU N T ER CHV
counter unblock chv COU N T ER U N BLOCK CHV
pin CODE
permission session P ERM ISSION V ALU E
blocked chv status BLOCKED ST AT U S
blocked status BLOCKED ST AT U S
data EF DAT A
(blocked chv status = blocked) = (chv, f alse) permission session
(counter chv = 0) (blocked chv status = blocked)
(counter unblock chv = 0) (blocked status = blocked)
(current f ile = (dom(F ILES CHILDREN current f ile) = {current directory}))

Copyright
946
E. BERNARD ET AL.
INITIALISATION
current f ile :=
current directory := mf
counter chv := M AX CHV
counter unblock chv := M AX U N BLOCK
blocked chv status := unblocked
blocked status := unblocked
permission session := {(always, true), (chv, f alse), (adm, f alse), (never, f alse)}
pin := a1
data := {(ef iccid, d1 ), (ef lp, d2 ), (ef imsi, d3 ), (ef ad, d4 )}
OPERATIONS
sw SELECT FILE(f f ) =
PRE
f f F ILES
THEN
IF (f f (DF M F ))
T HEN
IF (((f f, current directory) F ILES CHILDREN )
((current directory, f f ) F ILES CHILDREN )
( dp (DF M F ) ((dp, current directory) F ILES CHILDREN
(dp, f f ) F ILES CHILDREN ))
(f f = mf )
T HEN
sw := 9000
current directory := f f
current f ile :=
ELSE
sw := 9404
EN D
ELSE
IF ((current directory, f f ) F ILES CHILDREN )
T HEN
sw := 9000
current f ile := f f
ELSE
sw := 9404
EN D
EN D
END;
sw, dd READ BINARY =
PRE
current f ile EF
THEN
IF (current f ile = )
T HEN
sw := 9400
dd := 1
ELSE
IF ( dp P ERM ISSION (P ERM ISSION READ[current f ile] = {dp}
permission session(dp) = true))
T HEN
sw := 9000
AN Y f f W HERE (f f current f ile)
T HEN
dd := data(f f )
EN D
ELSE
sw := 9804
dd := 1
EN D
EN D
END;
cd, cf, cc, cuc STATUS =
BEGIN
cd := current directory
cf := current f ile
cc := counter chv
cuc := counter unblock chv
END;

Copyright
947
sw VERIFY CHV(code) =
PRE
code CODE
THEN
IF (blocked chv status = blocked)
T HEN
sw := 9840
ELSE
IF (pin = code)
T HEN
permission session(chv) := true
sw := 9000
ELSE
IF (counter chv = 1)
T HEN
counter chv := 0
blocked chv status := blocked
permission session(chv) := f alse
sw := 9840
ELSE
counter chv := counter chv 1
sw := 9804
EN D
EN D
EN D
END;
sw UNBLOCK CHV(code unblock, new code) =
PRE
code unblock CODE
new code CODE
THEN
IF (blocked status = blocked)
T HEN
sw := 9840
ELSE
IF (P U K = code unblock)
T HEN
pin := new code
counter unblock chv := M AX U N BLOCK
sw := 9000
ELSE
IF (counter unblock chv = 1)
T HEN
counter unblock chv := 0
blocked status := blocked
sw := 9840
ELSE
counter unblock chv := counter unblock chv 1
sw := 9804
EN D
EN D
EN D
END;
RESET =
BEGIN
current f ile :=
current directory := mf
permission session := {(always, true), (chv, f alse), (adm, f alse), (never, f alse)}
END;
END;

Copyright
948
E. BERNARD ET AL.
APPENDIX B. Eect predicates of GSM 11.11 formal model

N
Before predicates
After predicates
SELECT FILE
f f f f F ILES f f (DF M F )
(f f, current directory) F ILES CHILDREN
(current directory, f f ) F ILES CHILDREN
f f, dp f f F ILES f f (DF M F )
dp (DF M F ) (dp, f f ) F ILES CHILDREN
(dp, current directory) F ILES CHILDREN
f f f f F ILES f f (DF M F ) f f = mf
sw := 9000
current directory := f f
current f ile :=
f f dp f f F ILES f f (DF M F )
dp (DF M F ) f f = mf
(dp, f f ) F ILES CHILDREN
sw := 9404
P13
(current directory, f f ) F ILES CHILDREN )
sw := 9000
current f ile := f f
P14
(current directory, f f ) F ILES CHILDREN )
sw := 9404
P15
blocked chv status = blocked
sw := 9840
P16
code code CODE blocked chv status = blocked

pin = code

sw := 9000
P17
code code CODE blocked chv status = blocked

pin = code counter chv = 1
counter chv := 0
blocked chv status := blocked
permission session(chv) := f alse
sw := 9840
P18
code code CODE blocked chv status =

blocked pin = code counter chv =
1
counter chv := counter chv 1

sw := 9804
P19
blocked status = blocked
P20
pin := new code

code unblock, new code code unblock CODE
counter unblock chv :=
new code CODE blocked status = blocked P U K = code unblock
M AX U N BLOCK
sw := 9000
P21

new code CODE blocked status = blocked
P U K = code unblock counter unblock chv = 1
counter unblock chv := 0

blocked status := blocked
sw := 9840
P22

new code CODE blocked status = blocked
P U K = code unblock counter unblock chv = 1
counter unblock chv :=

counter unblock chv 1
sw := 9804
P6
P7
P8
P9
P10
P11
P12
VERIFY CHV
UNBLOCK CHV

Copyright
sw := 9840
949
APPENDIX C. Boundary goals of GSM 11.11 formal model
Predicates
Boundary Goals
SELECT FILE
P6
current directory = dfgsm
BG13
P7
BG14
BG13
P8
no solution
P9
P10
no solution
P11
no solution
P12
no solution
P13
BG14
BG13
P14
BG14
BG13
BG14
BG13
VERIFY CHV
P15
blocked chv status = blocked counter chv = 0
BG15
P16
blocked chv status = unblocked
BG16
P17
blocked chv status = unblocked counter chv = 1
BG17
P18

BG18
BG19
P19
blocked status = blocked counter unblock chv = 0
BG20
P20
blocked status = unblocked
BG21
P21
blocked status = unblocked counter unblock chv = 1
BG22
P22

BG23
BG24
UNBLOCK CHV

Copyright

Bllp04 Ij Author

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bllp04 Ij Author

Uploaded by

Copyright:

Available Formats

SOFTWAREPRACTICE AND EXPERIENCE

Softw. Pract. Exper. 2003; 34:915948

Prepared using speauth.cls [Version: 2002/09/23 v2.2]

Generation of test sequences

to: Laboratoire dInformatique de lUniversit

c 2003 John Wiley & Sons, Ltd.

Received 14 October 2003

SchlumbergerSema company is the leading supplier of Smart Card technology.

c 2003 John Wiley & Sons, Ltd.

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

B formal method and notation

Presentation of the B method

The data model of an abstract machine

Softw. Pract. Exper. 2003; 34:915948

Figure 1. B method in the software life cycle

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

The operation model of an abstract machine

Testing framework with B-TESTING-TOOLS

Short overview of the test generation method

Softw. Pract. Exper. 2003; 34:915948

c 2003 John Wiley & Sons, Ltd.

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

Figure 2. Test case constitution

Boundary state and preamble

Softw. Pract. Exper. 2003; 34:915948

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

GSM 11-11 standard: the B formal model

c 2003 John Wiley & Sons, Ltd.

Softw. Pract. Exper. 2003; 34:915948

Figure 4. Tree Structure of considered SIM les

We consider two kinds of le dened in the GSM 11-11 standard:

Every le which is a child of the current directory can be selected.

c 2003 John Wiley & Sons, Ltd.

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

Table I. Read access conditions

Read access conditions

c 2003 John Wiley & Sons, Ltd.

Softw. Pract. Exper. 2003; 34:915948

The B formal model

Six sets are dened to model the simplied kernel:

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

Table II. Potential response codes to commands

- Normal ending of the command

- File not found

- Access condition not fullled

Unsuccessful CHV verication, at least one attempt left

P ERM ISSION : this represents the various read access rights.

Five constants are dened in the B machine:

Softw. Pract. Exper. 2003; 34:915948

Five denitions complete the sets and constants already dened:

Ten state variables are used in the formal model:

Validation and verication of the B abstract model

Softw. Pract. Exper. 2003; 34:915948

GENERATION OF TEST SEQUENCES: GSM 11-11 STANDARD CASE-STUDY

Table III. Input and output parameters of operations

sw: status word

READ BIN ARY