An Interactive and Secure User Authentication Scheme for
Mobile Devices*
Qibin Sun1, Zhi Li2, Xudong Jiang3 and Alex Kot3
1
Institute for Infocomm Research, 119613, Singapore
2
Dept of EE, Stanford University, CA 94305-9515, USA
3
School of EEE, Nanyang Technological University, 639798, Singapore
ABSTRACT
Graphical password (i.e., image based authentication) is
considered as a promising alternative to traditional textual
password for mobile devices, to achieve better tradeoff
between usability and security. However, previous
proposals of graphical password have the limitation of
limited entropy. In this paper, we propose a new scheme
incorporating user face based authentication into the
association-based graphical password solution we
proposed before, aiming at achieving higher security
without compromising user-friendliness for mobile
application scenarios. System performance analysis and
comparisons with other schemes are presented to validate
our scheme.
1. INTRODUCTION
Today, the prosperity of e-business via mobile terminals
(e.g. PDA, smart phone and etc.) has boosted the
development of secure and convenient user authentication
solutions for touch screen devices. Traditional textual
password or PIN, however, relies on keyboard as the input
device. Many researchers thereby look at an alternative
approach - graphical password, or image-based
authentication (IBA) in a broader sense. Besides the
convenience of password input, it is deemed more user-
friendly in terms of memorability and recallability. The
basic hypothesis is that human brain is more capable of
storing graphical information than numbers or alphabets;
in addition, IBA utilizes an easier and more human-
friendly memorization strategy - recognition-based
memory, instead of recall-based memory for textual
password.
We classify state-of-the-art IBA approaches into two
categories: click-based approach [1, 2] and image-
selection-based approach [3, 4, 5, 6]. The former is based
on sequential clicks of some points on an image, in which
the location and order of the clicks are used as the
password. In the latter approach, the user selects some
“recognizable” secret images from a given image list. The
whole authentication process consists of several rounds of
such selections. In [7], inspired by a classic mnemonics –
Method of Loci, we presented a novel graphical password
design called association-based graphical password. The
*This work is supported by the A-STAR SERC Mobile Media TSRP Grant No 062 130 0056.
978-1-4244-1684-4/08/$25.00 ©2008 IEEE
principal idea rests on the human cognitive ability of
association-based memory. By creating “bounds” between
the password elements, the mnemonic effect is enhanced.
It is analogous to splitting a telephone number into chunks
to aid memorization. Based on the principle of zero-
knowledge proof protocol, we further improved our
primary design to overcome the shoulder-surfing attack
issue without adding any extra complexity into the
authentication procedure. However, one common problem
with above approaches is that the password entropy is
relatively small (see the evaluation in Section 4) which
may be easy for attackers to guess the password.
In this paper, motivated by the fact that most of today’s
mobile phones have been equipped with a digital camera,
we present a new solution by incorporating human face
into graphic password authentication process. The
proposed scheme is naturally resistant to shoulder-surfing
attack (i.e., the attacker could get the password by standing
behind the user and observing the whole key-in process
over the user’s shoulder)---even though the attackers guess
out the graphic password, he or she still cannot get into the
system because of the uniqueness of human face. The
password entropy is also significantly increased to the
level which is comparable to other cryptographic modules
like crypto hash and digital signatures which are widely
employed in today’s e-business.
This paper is organized as follows. Section 2 briefly
introduces two related prior work: face hashing and
association based graphic password. In Section 3, we
present our authentication scheme. Section 4 compares our
designs with some prior related work. Section 5 addresses
our future work and concludes the paper.
2. RELATED WORK
2.1 Face hashing
Biometrics could become a complementary means of
cryptography based user authentication because of its
uniqueness---everyone has only one face. To make
biometrics and cryptography combined seamlessly, the
latest research on biometrics intends to generate a robust
bit-string from human face [8]. The basic idea could be
summarized as follows. Firstly the system is trained on a
large of face data set, one example is by PCA (Principle
2973
Component Analysis), a set of Eigen matrix is then
obtained. The input face is, after some preprocessing and
face normalization, then projected onto this set of Eigen
matrix to obtain an Eigen vector F pertaining to this given
face. On the other hand, the system generates a random
number (RDN) then a random matrix from user’s login
request. This random matrix R is then orthogonalized by
the Gram-Schmidt process. The output of the dot product
of F and R is finally quantized by presetting thresholds to
generate the robust bit-string which could be served as the
user’s password. The idea is illustrated in Fig. 1. A more
detailed description about robust face hashing is given in
[8].
In [8], it has shown that a stable 40 bits could be
extracted from input face data with a nearly zero false
alarm. In Section 3, we will ride on this observation for
our system design and further improve the system security
(password entropy and shoulder-surfing attack) by
introducing an interactive authentication protocol between
server and mobile devices.
Fig.1 The diagram of robust face hashing
2.2 Graphic password based on association
Our authentication scenario involves three parties – Alice,
Bob and the machine verifier (MV). Alice’s objective is to
authenticate herself to the MV via some input devices,
such as the touch screen on a PDA. The MV – either
server or client-side – is to verify whether the person
trying to authenticate herself is Alice or another
impersonator. Bob – the impersonator or shoulder-surfer –
is to obtain the password shared between Alice and the
MV such that he could impersonate Alice by either blindly
guessing the password out (due to low password entropy)
2974
or observing the full interactions between Alice and the
MV.
In [7], inspired by a classic mnemonics - Method of
Loci, we proposed a novel association-based IBA scheme.
The principal idea rests on the human cognitive ability of
association-based memory. The mnemonic efficacy is
enforced by creating “bounds” between the password
elements, which is analogous to splitting a telephone
number into chunks to aid memorization.
In the user registration phase, Alice is required to pick a
desirable background image. The image is partitioned into
some small regions, each partition being a locus. Define
the locus alphabet as the set of all the loci L={l1, l2, …,
l|L|}. Also define an object alphabet O={o1, o2, …, o|o|} and
a color alphabet C={c1, c2, …, c|c|}. The object alphabet
consists of clip-arts images of objects, such as images of a
cup, a bike, a cats etc. The color alphabet consists of
colors like red, blue, green, cyan etc. To create the
password profile, Alice is then required to create N
triplets, each triplet with one element chosen from each
alphabet ijn={ln’, on’, cn’}, for 1”n”N. Note that Alice
usually tends to choose some “salient points” as the pass
loci, therefore, in practice, ln’ is selected from a subset
L’⊂L .
A schematic diagram of the authentication procedure is
shown in Fig. 2. The authentication phase consists of N
rounds. Triplet ijn serves as the “pass triplet” for round n,
with ln’, on’ and cn’ being the pass locus, pass object and
pass color, respectively. In round n, Alice needs to click
on the region of the background image associated with the
pass locus ln’. After the click, a window pops up, showing
a list of object elements O1⊂O, including the pass object
on’∈O1. The remaining subset O2=O1\{on’} is called the
decoy object set. Alice needs to select the pass object on’
from the list. After the selection, another window pops up,
showing a list of color elements C1⊂C, including the pass
color cn’∈C1. Similarly, the remaining subset C2=C1\{cn’}
is called the decoy color set. Alice needs to correctly select
the pass color on’. This procedure repeats for N rounds.
Alice is verified as authentic only when all the pass loci
are correctly clicked, and all the pass objects and pass
colors are correctly selected.
Fig. 2 Association based graphic password authentication
 In the authentication procedure, two levels of
association are created – association between the locus and
the object, and association between the object and its
color. By using mnemonics technique similar to the
Method of Loci, Alice could remember the associated
locus, object and color as a whole, rather than separately.
To enhance the security, Alice is encouraged to create
“bizarre scenes” (e.g. a blue banana in the bath) to enhance
the mnemonics effect.
We argue that this association-based approach is
superior compared to the recall-based and recognition-
based approach. Firstly, in the recall-based approach, the
problem is that Alice does not know how or where to
search in memory for the item. However, in association-
based approach, the item is hooked to the cues that are
available to her, thus Alice has no difficulty to retrieve
them. Secondly, since recognition-based approach only
leaves Alice limited actions to take (e.g. merely selecting
the pass images), this approach provides very limited
password entropy. In the association-based approach, the
user is given much more choices to act, and thus the
password entropy is much larger than in the recognition-
based approach.
3. PROPOSED AUTHENTICATION
SCHEME
In this section, we shall describe the basic idea of our new
solution, based on the combination between face hashing
and association based graphic password for further system
security enhancement.
3.1 Description
In the user registration phase, Alice uses her phone
camera to capture 2-3 face images and send to the
server. The server then registers her face into the
secure face database associated with her computed
unique Eigenface vector. The remaining registration is
the same process as the standard association based
graphic password described in previous section.
The authentication process is shown in Fig.3. Here we
summarize it as follows.
Step 1: Alice sends her ID and login request to the server.
Step 2: The server firstly generates a one-time random
number (RDN) particularly for Alice’s this time login.
Based on this RDN, the server computes a one-time face
hash bit-string for Alice. Note that every time, the server
will generate a different hash bit-string for Alice for
security consideration which will be explained later. Note
that the server keeps this temporary hash bit-string secretly
for verification purpose. Server then sends this RDN
together with the Alice’s registered background image
(BG) to Alice.
2975
Step 3: After Alice receives the RDN and BG, she
captures her face by her phone camera. A 40 bits one-time
face hash bit-string could then be extracted from her face
image by the RDN.
Step 4: A standard association based graphic password
authentication then starts. Alice needs to correctly pick up
all her selected objects associated with correct colors she
registered to the server before. The one-time RDN is again
used here to decide the display order of all objects and
colors. Crypto hash the selected Loci / Objects / Colors to
obtain another hash bit-string.
Step 5: Concatenate and randomize the generated face
hash bit-string and the graphic password. Send to the
server as Alice’s login password.
Step 6: Server compares Alice’s temporary face hash with
the received face hash and the stored hash of her graphic
password with the received one to decide whether the
server grants / denies Alice’s access to the server.
The reason why the server every time generates a different
face hash for Alice is because biometric data is very
critical ----once it is revealed, you cannot get it revoked.
Incorporating one-time RDN will make Alice’s face hash
bit-string different every time so that even Bob intercepts
one or a few of Alice’s previous face hashes, he still has
no idea about the one she is currently using for
authentication.
3.2 Analysis
Password entropy is usually used to measure the security
of generated password, which conceptually means how
hard to blindly guess out the password.
For simplicity, assume all passwords are evenly
distributed, the password entropy of graphic password can
then be calculated as follows [7].
H ( X std ) = N log 2 ( L ' O1 C1 )
(1)
For a typical application, suppose the size of the salient
point set of an image |L’| is 30, |O1| and |C1| are both 4,
and the number of rounds is 4, the entropy is therefore
35.6 bits, which is equivalent to the entropy of a 6-digit
textual password.
For a 40 bits generated face hash, its entropy is 40 bits
assuming again its uniform distribution. Therefore the
entropy of the final generated password is about 75.6 bits
which is comparable to other crypto modules. Note that in
design of a security related system, the system security
only depends on the weakest module in whole system.
Another advantage to incorporate face into user
authentication is that we could naturally avoid the threats
from shoulder-surf attacks because everyone’s face is
different. Therefore even Bob figures out Alice’s graphic
password, he still cannot impersonate Alice’s access. Note
that in [7], to avoid shoulder-surfing attack, we have to security (both password entropy and shoulder-surfing
pay the price by reducing the password entropy. attacks). Our future work includes conducting the studies
and experiments on the robustness of face hash and to
4. COMPARISONS WITH PRIOR WORK examine the effectiveness of our methods.

In this section, we compare our proposed schemes with

some prior related work in literature. The calculation of
REFERENCES
password entropy for various methods is in TABLE I. We [1] G. Blonder, Graphical Passwords, United States
can see that the password entropy of the proposed solution patent 5559961, 1996.
is significantly increased while the user-friendliness of [2] C. Perra and D. D. Giusto, A framework for image
graphic password is still maintained. based authentication, International Conference on
Acoustic, Speech, and Signal Processing (ICASSP),
TABLE I
2005.
COMPARISON OF PASSWORD ENTROPY
[3] The science behind Passfaces, Real User Corporation
Password Method & Descriptions Password Entropy (Sept. 2001) http://www.realuser.com
(bits) [4] R. Dhamija, A. Perrig, Déjà Vu: User study using
images for authentication, 9th USENIX Security
Textual. Small or capital letters or 6*log262 = 35.7
Symposium, 2000.
digits, length is 6.
[5] Sorensen, V.: PassPic (formerly ADS Security
Image-selection-based. 5 runs, in 5*log29 = 15.8 Wizard) – http://www.authord.com/PassPic/
each run select 1 from 9 images [3, [6] Takada, T., Koike, H.: Awase-E: Image-based
4, 5, 6] Authentication for Mobile Phones Using User's
Favorite Images. Int. Symposium on Human-
Click-based. 4 loci (Assuming 30 4*log230 = 20.0 Computer Interaction with Mobile Devices and
salient points) [1, 2] Services (Mobile HCI 2003). In: Lecture Notes in
Standard Authentication. 4 loci, 4 4* log2(30*4*4) = Computer Science, Vol. 2795, Springer-Verlag (2003)
objects, 4 colors (Assuming 30 35.6 347-351.
salient points) [7] [7] Z. Li1, Q. Sun, Y. Lian and D. D. Giusto, An
association-based graphic password design resistant to
SS-resistant Authentication. 4 loci, 4* log2(30*4*2) = shoulder-surfing attack, International Conference on
4 objects, 4 colors, K=2 (Assuming 31.6 Multimedia and Expo (ICME), 2005.
30 salient points) [7] [8] D.C.L. Ngo, A.B.J. Teoh and A. Goh, “Eigenspace-
The proposed solution 40 + 35.6 = 75.6 based face hashing”, in Proc. of International
bits Conference on Biometric Authentication (ICBA), pp.
195-199, 2004.

5. CONCLUSION
In this paper, we proposed a novel interactive and secure
authentication scheme for mobile applications. By
incorporating human face into the graphical password, we
obtained a significant improvement in terms of system

Fig.3. Interactive and secure authentication for mobile devices

2976