An Approach to

Counter Replay
Attack on Face
Recognition Systems

PREETHIKA ANAND
S8 CSE
B2E041

INTRODUCTION
Current authentication systems are based on two
factors:
What we have(say an object) and
What we know(a piece of knowledge)

But both of these factors together fails to prevent a person

masquerading as a valid user.Eg:ATM card+pin

Password protect us from illegitimate access,but can

be easily hacked.
An attractive soultion to this problem is
Biometric Recognition.

BIOMETRIC SYSTEMS
Verifies or identifies an individual from their biological
characteristics.
Function in one of 2 modes:
Verification
Authentication

Biometric systems work as follows:

Extracts biometric sample-defined by as set of parameters.
Comparator compares the sample to trained data
(previously acquired data).

Identification yiels result as confidence.

Verification as :acceptance or rejectance.

SPOOFING ATTACKS
Occurs when someone tries to bypass system using
fake biometrics.
Two forms of attack:
Direct attacks
Indirect attacks

Figure shows possible attacks on a biometric system.

1.Spoofing attacks.
2.Replay atacks.
3.Attacks after the
biometric is digitally
captured.

A POSSIBLE SOLUTIONLIVENESS DETECTION

System tries to detect signs of life.
Observing features like:
Blinking of eyes.
Change of facial expressions.
Skin texture ananlysis.
Mouth movements
Motion analysis.
Depth information.

REPLAY ATTACKS
Digital communication or action is captured
and replayed later.
For a face recognition system,a photo or
video of a live person can bypass the system
Opens possiblity to use someone else's
biometric to masquerade as that person.
Fraudster could even automate the collection
of fake biometrics -affecting large no of users.

A SOLUTION-DIGITAL
WATERMARKING
The biometric input needs to be validated.
Ensuring the signal was captured now and is not a
replay from past.
Digital watermarking is a technique to authenicate
biomertic samples.
Augments additional authentication data within
signal(called in-band) or as a seperate signal(out-band).
By the time outband marking is done-replay attack
would have ocuured.
In -band techniques are thus used.

CHALLENGE RESPONSE
TECHNIQUE
The user is required to perform a
ranomised specific action.
The correct response to challenge
indicates liveness .

THE PROPOSED SOLUTION

Color images are displayed on the screen.
The face reflects a signification portion of illumination
due to these colors.
The reflections are captured by a camera.
The reflected color digitally watermarks the captured
images.
The sequence of displayed colors form the challenge
If the calculated response matches the challenge -it
is less likely for the input to be a replay attack.

FRAME DIFFERENCING
Initial frame of captured video contains image of face
without illumination-W0 (of size p*q)
Subsequent frames are named as Wn
Subtract each frame Wn from W0 in RGB color space
to analyse reflected color.
Wn =min(max(Wn W0,0),), (1)
where represents the maximum value for each colour channel pixel (in our
case, =255; Wn [ 0,])

This helps to eliminate effect of factors like lighting

background images and individual camera properties.

IMPGE CAPTURE
ALGORITHM
1:Start camera and allow camera settings to settle
2: Display Black
3: Capture and save four frames
4: for four cycles do {Four cycles is approx. one second of video}
5: Display random colour and wait 100 msec
6: Capture and discard one frame
7: Capture and save three frames
8: Display Black and wait 100 msec
9: Capture and discard one frame
10: Capture and save three frames
11: end for
12: Capture and save three frames
13: return Captured frames; List of displayed colours

DIFFERENCE FRAME
ANALYSIS ALGORITHM
1: W0 firstframe
2: for all Wn captured frames do
3: Wn apply Eqn 1
4: Convert Wn from RGB to HSV colour space
5: xn[0]apply Eqns 2 and 3
6: xn[ck]apply Eqns 4 and 5
7: Classify colourn SVM(xn)
8: end for
9: return Predicted frame colours (i.e., colour n)

Each Wn is converted from the RGB colour space to the

HSV colour space.
Seven dimensional feature vector x { 0.0100.0}7 is
extracted.
Lowercase bold symbols represent a vector (e.g., xn).
Uppercase bold symbols represent a matrix (e.g., M).
Each element in a vector is referenced as x[].
The ith row and jth column element in a matrix is referenced as
M(i, j).
The colour space channel for each element in an image matrix
is represented as M(i, j)[] (e.g., Wn(i, j)[V] represents the
brightness channel for the element at row i and column j of Wn).

The rst dimension in xn (xn[1]) for Wn pq represents the percentage number of

dark pixels.
Dark pixels have a low value in the V (brightness) channel.
Dark pixels were designated as any pixel value less than or equal to two (2) in the
V channel.
p

xn[1]= M(i,j) *100/1

(2)

i=1 j=1
________________
|Wn|
where
M(i, j)={1 if 0 Wn(i, j)[V]2
0 otherwise

(3)

The remaining dimensions xn[ck] represent the percentage distribution of the H values where
the pixel is not designated as a dark pixel.
For each colour in Table , xn[ck] is described as:
p

xn[ck]= Nk(i,j)

(4)

i=1 j=1
where
Nk(I,j)={1 if Wn(i,j)[V]>2 and Wn(i,j)[H]= k

(5)

0 otherwise
and k is dened in Table . xn[ck] is then normalised as a percentage of the number of pixels in
all colours in Table
7
(i.e., xn[ck]/ xn[ck]100/1 ).
K=2
The differenceframefeature vector xn is classied using the Multi-class SupportVector Machine
(SVM) in OpenCV, to predict the reected colour.

COLOUR CLASSIFICATION
ALGORITHM
The color sequences predicted by the SVM are analysed to
determine the reected colour.
The 3 captured color frames are separated using a black
frame.
The colour is determined by choosing the majority colour
classied in the set of three frames .
If there is no clear majority, the nal colour in the set of three
frames is chosen (reasoning that this is the most likely colour
after the camera has performed any adjustments).
The use of three frames adds redundancy to the
classication process, improving the correct colour
classication.

EXPERIMENTAL SETUP
Reflection from three different objects were analysed:
1.Blank white paper
2. Printed photo of a face
3.Live person.
Four colours in a 150 long sequence was used and
reflection was captured.
Experiment 2 used a life sized printed face mounted in front
of camera.The face has been cut to remove background
surfaces.
Experiment 3 used 10 live people from different ethinic
background(asian,Caucasian,middle eastern etc.

RESULTS

First table details the Classication Matrix of colour classications in experiment 1 for the internal
camera. The ith row shows the colour that was displayed,and the jth column shows the colour that was
classied. The element at (i, j) is the percentage rate that the displayed colour was identied in the reection
as the classied colour.Vast majority of errors are due to the misclassied colours migrating towards Blue.
Next table details the Classication Matrix of colour classications in experiment 1 for the external
camera. Using the external camera, most colours classied correctly, but Magenta often was misclassied
as either Green or Blue .

ADVANTAGES
1.Frame difference analysis helps to eliminate effects in face
recognition due to factors like Automatic white
balance,background images,ambient light and individual
camera responses by capturing responses interleaved by
duration for camera to adjust after illuminations.
2.A large random sequence of colors can be used to create
harder challenges,difficult to predict.
3.Pseudo random no generator can be used create an even
distribution of sequence of color,strengthening the algorithm
by improving entopy of algorithm.
4.This technique can be applied in even uncontrollable
environment like consumer devices.

DISADVANTAGES
1.Sensitive to movement of face-causing
color misclassification.
2.Attacker can try to replicate the
watermarking(replicating requires
sophisticated image processing
mechanisms).

CONCLUSION
The technique for countering replay attacks by
analyzing refections from face when coloured images are
used showed excellent results.
This technique can be applied in even uncontrollable
environment and overcomes errors due to factors like
illumination,saturation etc.
Future work include incorporating techniques to
improve entropy of the system by creating harder
challenges using more colors.
This technique opens the door for further studies in
video analysis for preventing spoofing attacks.