Face Video Based Touchless Blood Pressure and

Heart Rate Estimation

Monika Jain, Sujay Deb, A V Subramanyam
Department of Electronics and
Communication Engineering
Indraprastha Institute of Information Technology
Delhi, India 110020
Email: monikaj@iiitd.ac.in, sdeb@iiitd.ac.in, subramanyam@iiitd.ac.in

Abstract—Hypertension (high blood pressure) is the leading
cause for increasing number of premature deaths due to cardio-
vascular diseases. Continuous hypertension screening seems to
be a promising approach in order to take appropriate steps to
alleviate hypertension-related diseases. Many studies have shown
that physiological signal like Photoplethysmogram (PPG) can be
reliably used for predicting the Blood Pressure (BP) and Heart
Rate (HR). However, the existing approaches use a transmission
or reflective type wearable sensor to collect the PPG signal. These
sensors are bulky and mostly require an assistance of a trained
medical practitioner; which preclude these approaches from
continuous BP monitoring outside the medical centers. In this
paper, we propose a novel touchless approach that predicts BP
and HR using the face video based PPG. Since the facial video can
easily be captured using a consumer grade camera, this approach
is a convenient way for continuous hypertension monitoring
outside the medical centers. The approach is validated using
the face video data collected in our lab, with the ground truth
BP and HR measured using a clinically approved BP monitor
OMRON HBP1300. Accuracy of the method is measured in terms
of normalized mean square error, mean absolute error and error
standard deviation; which complies with the standards mentioned
by Association for the Advancement of Medical Instrumentation.
Two-tailed dependent sample t-test is also conducted to verify that
there is no statistically significant difference between the BP and
HR predicted using the proposed approach and the BP and HR
measured using OMRON.
I. I NTRODUCTION
Hypertension (High Blood Pressure) is one of the major
causes of cardiovascular disease. It is a symptom-less condi-
tion that mostly goes unnoticed and untreated, especially due
to the lack of medical facilities or due to the busy lifestyle of
potentially hypertensive population. An efficient solution to
this is an ubiquitous hypertension monitoring system that can
be used continuously for in-situ health tracking. Unfortunately,
the prevalent ways for measuring accurate Blood Pressure (BP)
are cuff-based, bulky and costly, which cannot be used fre-
quently without doctor’s supervision. In pursuit of finding the
compact and cuffless ways for monitoring the BP continuously,
researchers have rigorously analyzed the Photoplethysmogram
(PPG) signal. Recent studies have proved that PPG can be
reliably used in BP and Heart Rate (HR) estimation.
Fukushima et al. in [1] use the accelerated PPG waveform,
Heart Rate Variability (HRV) and rate of PPG signal drift to
estimate the BP. Suzuki et al. in [2] show how orthogonal array
and the Signal to Noise ratio from Taguchi method is used to
reject the noise and select the robust features from the PPG
signal. The feature selection method is applied to multiple
regression analysis for Systolic BP estimation. Choudhury
et al. in [3] use a 2-element Windkessel model to estimate
BP based on the PPG signal. Visvanathan et al. in [4] and
Kurylyak et al. in [5] use a large number of parameters
extracted from PPG for estimating the BP.
Many researchers also introduce special sensor-based sys-
tem to make the PPG collection procedure easier. Samria et al.
in [6] introduce a finger PPG sensor designed using an infra-
red LED. The algorithm estimates BP using the PPG based
features, such as, systolic upstroke volume, diastolic time and
time delay. Ahmed et al. in [7] present a prototype that records
PPG from the user’s head region (temple). The recorded PPG
is used for estimating the BP. All these studies depend upon
a wearable sensor and a dedicated processing and acquisition
unit, resulting in a bulky system that can induce discomfort to
the patient during continuous monitoring. These approaches
are suitable only for in-house or clinical health monitoring.
To make continuous hypertension monitoring a more com-
mon practice, a completely touchless and cuffless solution is
required, that does not intrude much into day to day activity of
the user. Such solution should be less time consuming and easy
to use. Many state-of-the-art approaches suggest the extraction
of PPG signal distantly using the face or palm video. Some
of these approaches are discussed below.
Verkruysse et al. in [8] propose a method for remotely
measuring the PPG signal using the face video. Based on
this PPG, the Respiratory Rate (RR) and HR is quantified
up to several harmonics. Poh et al. in [9] propose a low-cost
method for extracting the PPG signal based on face video
recorded from the webcam. They use the PPG signal for
quantifying HR, RR and HRV. Kumar et al. in [10], propose
a new method, DistantPPG, that extracts a good quality PPG
from the face video by overcoming the constraints like skin
color and motion artifacts. Sun et al. in [11] present a motion
artifact reduction technique that gives a good quality imaging
PPG signal using the face or palm video. This technique is
suitable for measuring HR and RR during exercise.
Unfortunately, these approaches could only quantify HR,
RR or HRV using the remotely recorded PPG. In a latest
 Input Video Extraction of PPG signal Peak detection
Principal component and
Pre-processing Parameter extraction
analysis
Crop the face
Reconstruction
Video stabilization BP and HR estimation
Reconstruction Error

Fig. 1. Process flow of the proposed approach

study by Jeong et al in [12], a high correlation of image
based Pulse Transit Time (iPTT) is demonstrated with systolic
BP. This study uses a high speed camera for recording the
image based PPG (iPPG) and iPTT (using the face and palm
video). However, this study is limited to finding the range of
correlations only.
This paper draws motivation from all such previous studies.
We present an efficient novel approach for recording the PPG
touchlessly using only the face video. Moreover, we use this
PPG to estimate the HR as well as systolic and diastolic BP,
unlike other studies. A face video of the subject is processed
to extract the corresponding PPG signal. Time [4][5] and
frequency domain parameters are obtained from this PPG
signal. Finally, BP and HR are estimated based on these
extracted parameters. The BP and HR estimation accuracy of
the proposed approach shows that it can be safely used for
continuous and non-invasive hypertension screening.
The remaining paper is organized as follows: Section II
discusses how PPG signal can be extracted using the proposed
approach. It shows the estimation of BP and HR based on the
extracted PPG signal. Section III describes the experimental
set-up used to verify the proposed approach. It shows the
accuracy of the approach by testing it on a database collected
in our lab. The paper concludes with Section IV, which
discusses the advantages and future scope of the proposed
approach.
II. M ETHODOLOGY
This section explains the method used for extracting the
PPG signal from the face video. It also shows how BP and
HR are estimated based on the extracted PPG. A complete
process flow of the proposed approach is shown in fig. 1 and
is subsequently explained as follows:
A. Video Collection and Pre-processing
A face video of the subject is recorded (in ambient lighting
conditions) by keeping the camera still at about half meter
distance from the face. The video is recorded for the duration
of one minute, while the subject is asked to sit still with the
Considering that the subject or the camera might have
moved during the video recording, all the frames are stabilized
using a video stabilization technique [14]. Although the green
channel features the strongest plethysmographic signal [8],
the red and blue channels also contain plethysmographic
information [15]. In this study, we use only the red channel
component of the RGB frames. Once we obtain the stabilized
T
video, we generate the data matrix A = [a1 , a2 , · · · , aL ] ,
where ai represents the column vector obtained by vectored
representation of red channel component of ith frame. A
∈ RL×M ; where L is the total number of frames in the video
and M is M1 .M2 .
B. Extraction of PPG Signal
Assuming that the lighting conditions and camera settings
were constant throughout the recording, any intensity variation
observed in the red channel should be due to the variation
in the blood that flows beneath the face skin. To extract
these variations from the video, we use Principal Component
Analysis (PCA) [16]. Once we have the matrix A that carries
intensity of the red channel of the video frames, PCA is
performed on A. In order to extract PPG signal, we compute
0
the error between A and the reconstructed matrix A . Here,
0
A is determined by using only n principal components of A,
where n is decided experimentally. The reconstruction error
represents the minute changes in the pixel intensity that occur
due to the variation in the blood flowing beneath the face skin.
This error is proportional to the PPG signal and is given as:
APPG = A0 − A (1)
Each row in matrix APPG is reshaped back in its original
size (that is, original face cropped frame size, M1 × M2 ).
This gives us a sequence of L error video frames, B i , where
i represents the ith frame. From each frame, the PPG signal
content, P P Gi , is obtained using the following expression:
1 XM1 XM2
i
P P Gi = Bjk ∀ i = 1, 2, · · · L (2)
(M1 M2 ) j=0 k=0

eyes closed. The PPG signal is extracted using only the face
region of the video, so to clip off any extra portion in the video
frames, KLT (Kanade-Lucas-Tomasi) face detection algorithm
is used [13]. A new video is created that contains only the
face of the subject, having M1 × M2 frame size.
Since frequency range of the PPG signal is 0.5–5 Hz [17],
high frequency noise is removed from the obtained PPG signal
(P P G1:L ) using a bandpass filter with cut-off frequency [0.5,
5] Hz [11]. In the next section, we describe the peak detection
and parameter extraction performed on the filtered PPG signal.
 Fig. 2. Peak detection in the PPG signal extracted from face video Fig. 4. Calculating dominant frequency in the PPG signal

TABLE I
PARAMETERS E XTRACTED F ROM THE PPG S IGNAL
_ _ ---------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------

PPG Pulse
PeakN+2
AP _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ PeakN
_____________________________________
<----------------------------PPG HeightN------------------------------>

PeakN+1 Domain Parameter Description

___________________________________

Time [4][5] T 2N +1 − T 2N = PP
T 1N +1 − T 1N = FF
T 2N − T 1N = PF
Amplitude

Dicrotic NotchN
AD _ _ _ _ Dicrotic
________________
Notch Dicrotic NotchN+1
N-1
T 1N +1 − T 2N = FP
_________________________

_______________________

PP/FF
PP/FP
FP/FF
FP/PF
PPG Height(AP )
AF and AD Heights
FootN+2 (T 2N − T IN ) = Crest time
FootN FootN+1
AF_ _ _ _ _ _ _ _ _ _ _ _ <--------PPG WidthN------->
P T 3N
___

T 2 (P P G pulse)= Systolic area (S)

__

o T1N+1 T2N+1 T3N+1

P T 1NN +1
T1N T2N T3N T1N+2 (P P G pulse)= Diastolic area (D)
T3N
Time (s) S / D = Ratio area
Fig. 3. Time domain parameters in the PPG signal S + D = Total area
(T 3N − T 2N ) = Delta time
(AD − AF )/(AP − AF ) = Augmentation Index (AI)
C. Peak Detection and Parameter Extraction 1-AI = Reflection Index
Frequency Dominant frequency of PPG
Even after a very controlled video recording and processing
Power Spectral Density of PPG
procedure, the quality of PPG signal may get degraded due to
the constraints like weak peripheral perfusion of the subject,
time taken by the camera to adjust the focus etc. To avoid this,
of dominant frequency from the bandpass filtered PPG using
the first and last 5 seconds of PPG signal is discarded. Only
its Fourier transform (P P G(f )). All the parameters are listed
the best 10 second portion of the signal is selected and used
in Table I (where N is the N th cardiac cycle and N + 1 is
for further processing; that is, the portion which has visible
N + 1th cardiac cycle).
significant features like peaks, foots and dicrotic notches. A
good quality PPG signal will ensure a more accurate BP and D. Blood Pressure and Heart Rate Estimation
HR estimation.
In the selected PPG signal, peaks, foots and dicrotic notches To predict the BP, we use the basic regression framework,
are detected, as show in fig. 2. These detected peaks are used y = Px (3)
to extract the time and frequency domain parameters based on
which BP will be estimated. Fig. 3 shows the extraction of where y represents the vector of expected BP (systolic or
time domain parameters [4][5] and fig.4 shows the extraction diastolic), P is the matrix of parameter vectors – the rows
of P correspond to different individuals and the columns TABLE II
correspond to the various parameters (which is equal to 21 BP E STIMATION E RROR W ITH R ESPECT TO OMRON HBP1300
here, as shown in Table I), and x is the regression weights (to
– Systolic Blood Pressure Diastolic Blood Pressure
be estimated). The regression framework is simple to interpret;
Fold NMSE* MAE* ESD* NMSE MAE ESD
the regression weights tell us the relative importance of various
I 3.99 4.07 4.97 6.03 3.10 5.02
parameters in BP prediction. In this study, we will solve (3)
II 4.00 2.71 4.12 6.45 4.47 2.38
using Polynomial kernel regression [18]. To extract the HR (in
III 4.21 3.56 4.07 7.74 4.67 6.33
beats per minute), a simple relation given below is used [19]:
IV 6.55 4.00 7.80 10.20 4.39 7.53
V 3.80 4.19 5.00 4.22 2.96 3.17
number of peaks
HR = ∗ 60 (4) VI 5.28 2.95 6.37 6.07 3.50 5.01
duration of the signal VII 4.15 4.28 5.55 8.54 4.19 6.89
VIII 5.06 4.45 4.45 7.84 3.48 6.25
III. E XPERIMENTAL S ET- UP AND R ESULTS IX 4.60 4.92 5.97 3.88 2.71 3.11
Mean 4.63 3.90 5.37 6.77 3.72 5.08
The face videos of 45 subjects are collected and pre- *NMSE = Normalized mean square error in %,
processed as described in Section II (A). Most of the par- MAE= Mean absolute error in mmHg
ticipating subjects are students at our research institute, aging ESD= Error standard deviation in mmHg
between 20 - 40 years, and had BP in the normal range, that
is, Systolic BP in the range of 95-130 mmHg and Diastolic
TABLE III
in the range of 60-90 mmHg. The videos are captured using HR E STIMATION E RROR W ITH R ESPECT TO OMRON HBP1300
Sony HDR-CX405 9.2 megapixel camcorder at 50 frames per
second with resolution 1920×1080. PPG signals are extracted Fold NMSE MAE ESD
from each video and were further processed for peak detection I 3.05 2.40 2.88
and parameter extraction, as described in Section II (B and C). II 2.42 2.00 2.19
Here, n is taken as 10% of the principal components of A. III 2.21 1.80 1.51

A database of these 45 subjects is created, that consists of IV 2.32 1.80 2.16

the extracted parameters and the corresponding ground truth V 2.33 1.60 2.30
BP and HR (recorded using a clinically approved OMRON VI 2.34 1.40 2.04
HBP1300 digital BP monitor). VII 2.55 2.00 2.34
VIII 1.85 1.40 1.78
In order to perform the 9 fold cross validation, entire
IX 1.92 1.20 1.64
database is divided uniformly into 9 parts (5 subjects per
Mean 2.33 1.73 2.09
part), such that, each part is used as the test data once
when the remaining 8 parts are used as the training data.
Accuracy of the proposed approach is measured in terms of
Normalized Mean Square Error (NMSE in %), Mean Absolute IV. DISCUSSION
Error (MAE in mmHg) and Error Standard Deviation (ESD in Video based PPG has gained a lot of popularity in the recent
mmHg), as shown in Table II and III. The calculated errors for past. Researchers have rigorously analyzed it to quantify the
BP estimation fall under the standard allowable error limits HR, HRV and RR. However, BP extraction based on video
mentioned by Association for the Advancement of Medical PPG signal has not been explored yet. This paper proposes
Instrumentation; (MAE < 5mmHg and ESD < 8mmHg) [20]. a novel approach that extracts the PPG using face video
Two-tailed dependent samples t-test [21] is also conducted and estimates the BP and HR based on it. The preliminary
to verify that there is no statistically significant difference experiments implies that the approach can be reliably used
between BP and HR predicted using the proposed method and for hypertension monitoring. This approach can be easily
the measured BP and HR. To conduct the t-test, the differences modified into a mobile application and software, that can be
between the predicted and measured BP and HR values should used with android mobiles and laptop/computers. In future,
be normally distributed. This condition was found satisfied as the proposed approach can be used for stress management
the skew and kurtosis levels for the difference were found and hypertension monitoring at home, office, school, college,
lesser than the maximum acceptable range for conducting the meditation and medical centers. It is an inexpensive, compact
t-test (skew ≤ 2.0 and kurtosis ≤ 9.0 ), as suggested by [22]. and time efficient approach that can be readily used whilst
The null hypothesis: There is no statistically significant involved in your day to day activities.
difference between the BP and HR estimations made by Factors like skin complexion has been overlooked in this
the proposed approach and the measured BP and HR, was study; it is challenging to extract PPG from the face video
accepted; as the calculated t-value was lesser than the critical with a darker skin-tone [10]. Also, at the current stage, the
value of t and p > 0.05 with degree of freedom equal to 4 approach is tested over a very small database that mostly had
(number of test subjects - 1). young and healthy volunteers from our institute. Since, BP
estimation using the video based PPG has not been explored [13] B. D. Lucas and T. Kanade, “An iterative image registration
yet, there is no existing suitable online database on which technique with an application to stereo vision,” in Proceedings
of the 7th International Joint Conference on Artificial Intelligence
the experiments can be conducted. In future, we look forward - Volume 2, ser. IJCAI’81. San Francisco, CA, USA: Morgan
to collect and work with a bigger and diverse database, Kaufmann Publishers Inc., 1981, pp. 674–679. [Online]. Available:
collected at government dispensary. This database will include http://dl.acm.org/citation.cfm?id=1623264.1623280
[14] K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhyoung, “Video
old and young subjects with known cardiovascular diseases stabilization using robust feature trajectories,” in Computer Vision, 2009
and a variety of skin-tone. Our future work will be based IEEE 12th International Conference on. IEEE, 2009, pp. 1397–1404.
on improving the robustness and accuracy of the proposed [15] T. Tamura, Y. Maeda, M. Sekine, and M. Yoshida, “Wearable photo-
plethysmographic sensorspast and present,” Electronics, vol. 3, no. 2,
approach. pp. 282–302, 2014.
[16] H. Abdi and L. J. Williams, “Principal component analysis,” Wiley
ACKNOWLEDGMENT Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, pp.
433–459, 2010. [Online]. Available: http://dx.doi.org/10.1002/wics.101
This work is partially supported by Indo-US Grand Chal- [17] N. Shetty, N. Prasad, and N. Nalini, Emerging Research in Computing,
lenge Initiative-Affordable BP Measurement Technologies Information, Communication and Applications. Springer, 2015.
for Low Resource Setting. We would also like to ac- [18] Y. Goldberg and M. Elhadad, “splitsvm: fast, space-efficient, non-
heuristic, polynomial kernel computation for nlp applications,” in Pro-
knowledge the support of ITRA project, funded by DE- ceedings of the 46th Annual Meeting of the Association for Compu-
ITy, Government of India, under a grant with Ref. no. tational Linguistics on Human Language Technologies: Short Papers.
ITRA/15(57)/Mobile/HumanSense/01. Association for Computational Linguistics, 2008, pp. 237–240.
[19] A. K. Kanva, C. J. Sharma, and S. Deb, “Determination of spo2
and heart-rate using smartphone camera,” in Control, Instrumentation,
R EFERENCES Energy and Communication (CIEC), 2014 International Conference on,
[1] H. Fukushima, H. Kawanaka, M. S. Bhuiyan, and K. Oguri, “Cuffless Jan 2014, pp. 237–241.
blood pressure estimation using only photoplethysmography based on [20] A. Coleman, S. Steel, P. Freeman, A. de Greeff, and A. Shennan,
cardiovascular parameters,” in 2013 35th Annual International Confer- “Validation of the omron m7 (hem-780-e) oscillometric blood pressure
ence of the IEEE Engineering in Medicine and Biology Society (EMBC), monitoring device according to the british hypertension society proto-
July 2013, pp. 2132–2135. col,” Blood pressure monitoring, vol. 13, no. 1, pp. 49–54, 2008.
[2] A. Suzuki and K. Ryu, “Feature selection method for estimating sys- [21] M. R. King and N. A. Mody, Numerical and statistical methods for
tolic blood pressure using the taguchi method,” IEEE Transactions on bioengineering: applications in MATLAB. Cambridge University Press,
Industrial Informatics, vol. 10, no. 2, pp. 1077–1085, May 2014. 2010.
[3] A. D. Choudhury, R. Banerjee, A. Sinha, and S. Kundu, “Estimating [22] H. O. Posten, “Robustness of the two-sample t-test,” in Robustness of
blood pressure using windkessel model on photoplethysmogram,” in statistical methods and nonparametric statistics. Springer, 1984, pp.
2014 36th Annual International Conference of the IEEE Engineering 92–99.
in Medicine and Biology Society, Aug 2014, pp. 4567–4570.
[4] A. Visvanathan, A. Sinha, and A. Pal, “Estimation of blood pressure
levels from reflective photoplethysmograph using smart phones,” in
Bioinformatics and Bioengineering (BIBE), 2013 IEEE 13th Interna-
tional Conference on, Nov 2013, pp. 1–5.
[5] Y. Kurylyak, F. Lamonaca, and D. Grimaldi, “A neural network-based
method for continuous blood pressure estimation from a ppg signal,” in
2013 IEEE International Instrumentation and Measurement Technology
Conference (I2MTC), May 2013, pp. 280–283.
[6] R. Samria, R. Jain, A. Jha, S. Saini, and S. R. Chowdhury, “Noninvasive
cuff’less estimation of blood pressure using photoplethysmography
without electrocardiograph measurement,” in Region 10 Symposium,
2014 IEEE, April 2014, pp. 254–257.
[7] N. Ahmed, R. Banerjee, A. Ghose, and A. Sinharay, “Feasibility analysis
for estimation of blood pressure and heart rate using a smart eye
wear,” in Proceedings of the 2015 workshop on Wearable Systems and
Applications. ACM, 2015, pp. 9–14.
[8] W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote
plethysmographic imaging using ambient light.” Opt. Express,
vol. 16, no. 26, pp. 21 434–21 445, Dec 2008. [Online]. Available:
http://www.opticsexpress.org/abstract.cfm?URI=oe-16-26-21434
[9] M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in non-
contact, multiparameter physiological measurements using a webcam,”
IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, pp. 7–11,
Jan 2011.
[10] M. Kumar, A. Veeraraghavan, and A. Sabharwal, “Distan-
ceppg: Robust non-contact vital signs monitoring using a
camera,” CoRR, vol. abs/1502.08040, 2015. [Online]. Available:
http://arxiv.org/abs/1502.08040
[11] Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu,
“Motion-compensated noncontact imaging photoplethysmography to
monitor cardiorespiratory status during exercise,” Journal of Biomedical
Optics, vol. 16, no. 7, pp. 077 010–077 010–9, 2011. [Online].
Available: http://dx.doi.org/10.1117/1.3602852
[12] I. C. Jeong and J. Finkelstein, “Introducing contactless blood
pressure assessment using a high speed video camera,” J. Med.
Syst., vol. 40, no. 4, pp. 1–10, Apr. 2016. [Online]. Available:
http://dx.doi.org/10.1007/s10916-016-0439-z