You are on page 1of 13

Activity Recognition using Wrist worn Accelerometer

Name : Sreenivasa Murthy Venkateshan


SS 2012 : Activity Recognition using Wrist worn Accelerometer

Institut fr Datentechnik Forschungsgruppe Mikroelektronische Systeme Prof. Dr. Dr. h.c. mult. Manfred Glesner Betreuer : Dipl-Ing. Franois Philipp

1. Abstract
Activity recognition is one of the most important and deeply researched topics at various universities across the world. This paper deals with one such activity recognition where the activity data is obtained and processed with the help of EZ430-Chronos from Texas Instruments and is recognised using machine learning algorithms such as KNN classifier to classify the activity. Here the classification has been done offline (after the data has been collected). These algorithms can be implemented in real time situations and processed to classify the activity of a person. The paper also examines two variations in KNN classification algorithm and tries to give a conclusion on the final result. In the end, the paper tells about the drawbacks in the algorithm used and also suggests other machine learning algorithms that are used globally.

2. Outline
1. 2. 3. 4. 5. Abstract Outline Introduction Related work Experimental setup 5.1 Hardware 5.2 Feature extraction 5.3 Classification Evaluation Conclusion and Future Work Acknowledgements References 2 3 4 5 6 6 7 8 9 11 12 13

6. 7. 8. 9.

3. Introduction
Activity/gesture recognition is widely dicussed research topics in top universities across the world. It is one of the fast growing research areas in the field of healthcare, sports and entertainment. Activity recognition is one of the parameters which define Personal health Informatics and it tells about its applications in healthcare and fitness guidance. Researchs have been performed using wearable sensors and systems to measure and track the activities of people.Activity recognition can also be used in elderly care where people are more prone to injuries and accidents.These systems can be used in emergency cases of patients and also home monitoring of elderly people to identify their activities in case of an accidental fall and emergencies. Earlier gesture recognition was performed using computer vision techniques. However such kind of techniques strongly depend on the lighting condition and camera facing angles.Therefore these techniques cannot be used in dark environments. Then later gesture recognition carried out based on the acceleration data for a particular gesture/activity obtained from the accelerometer sensors. For example TUB-Sensor Glove[fig:1] can collect hand orientation and acceleration, and finger joint angles. Now with the use of 3-axis accelerometer, we are able to extract the data needed for processing and then implement the algorithms for classification. Recent advances in MEMS and embedded systems have given rise to commercial products such as Apple Iphone [fig: 3] and Nintendo Wiimote [fig: 2] which contain embedded accelerometers. These devices can be used for wireless interaction and can be used for personal applications such as home appliances and health monitoring.

Figure 1 : TUB-Sensor Glove. Source [9]

Figure 2: Nintendo Wiimote. Source [11]

Figure 3: Apple Iphone. Source [10]

4. Related Work
Activity recognition has been researched at various universities and the following literature and references throws some light on the techniques and methods used [3],[4],[5]. Accelerometer has been used as wearable sensors to do activity recognition as it gives most appropriate values for an activity. The data obtained is also accurate for classification of an activity. Embedded sensing systems group at TU-Darmstadt have been doing wide research on activity recognition using accelerometer data [2]. There have been many research works on activity recognition systems which have used many accelerometers on different locations in the body such as chest, waist, legs, thighs etc [1]. These usages of accelerometers will reduce the movement as there would be many accelerometer sensors obstructing the daily activities. Many researchers have carried out experiments to measure body temperature and measure heart rate of the patients and elderly along with the activity recognition which would form the part of Personal health monitoring systems. However in this paper, I deal with wrist worn accelerometer to classify simple day to day activities such as sitting, walking and running. The best location for activity location using an accelerometer would be the waist as it is the centre of the body and can give accurate data for an activity. Since we are classifying simple activities we are using a TI EZ-430 Chronos wrist watch to obtain the accelerometer data. Varieties of features have been extracted from accelerometer data and have been used in training machine learning algorithms and classifiers. Features such as mean, variance, standard deviation, energy and entropy have been extracted and used to classify the activities. In this paper I have made use of popular features such as mean and variance to train the classifiers. Many papers have implemented various classification algorithms such as k-Nearest Neighbour (KNN), Support Vector Machine (SVM), Nave Bayes Classifier and Hidden Markov Models (HMM) making use of the above mentioned features for classifying the activities. More features would definitely increase the accuracy of these classifiers thereby giving a better and more activity recognising system. Here in the paper, I have used k-Nearest Neighbour algorithm to classify the activity. Processing of data can be done in real time as well as offline. Real time processing would be better in the field of healthcare as patients who are at home or hospitals can be monitored constantly and continuously. Offline data processing can be done to collect the 3-axis accelerometer data and then use the classifying algorithms for activity recognition. Nokia 95 is a phone with data sensing capabilities and some of the algorithms having online classifying capabilities have been implemented. Nike+iPod Sports Kit is used to collect data which imports collected data periodically to iPhone via Bluetooth connection [6].

Figure4: Sitting posture

Figure5: Walking posture

5. Experimental Setup
5.1 Hardware
EZ430 Chronos from Texas Instruments (TI) is a development system where designers and developers can create wireless smart watch applications. EZ Chronos consists of a built in MSP430 microcontroller and also integrated sensors such as tri-axial accelerometer, pressure and a temperature sensor. It also comes with a CC1111 USB RF access point through which the data can be transmitted wirelessly from the smart watch. Many application related projects have been developed by disassembling the watch and programming the microcontroller using the TI softwares provided such as the code composer studio through USB development programming interface.

Figure 6: EZ430 Chronos with USB development stick (top) and CC1111 RF USB access point (bottom).Source [7]

It also comes with a control center tool developed by Texas Instruments which is user friendly. Customers and people can use this tool and plot the accelerometer data. The temperature and heart rate can be monitored by connecting the watch wirelessly to the USB access point. The data can be transferred wirelessly from the computer to the watch and vice versa. Data logged can also be downloaded to the computer and updated.

Figure 7: EZ430 Chronos control center. Source [7]

5.2 Feature Extraction


Accelerometer data is obtained from the single tri-axial accelerometer using the TI EZ430 Chronos. The tri-axial data is obtained to do the training of the classes (A,B,C).Each class has tri-axial data as explained in the equations 1,2,3 below. Parameter T has been defined in these equations as it is the function of the 3 axes(x, y, z) of the accelerometer. Each class is assigned to an activity. Here in the report, I am assigning 3 basic activities such as running, walking and sitting to the 3 classes defined. The amplitude of the accelerator data is obtained using the equations given in 4, 5 and 6.These amplitudes can be used to obtain popular time domain features such as mean, variance, standard deviation, energy etc.

A (ax, ay, az )..............................1 B (bx, by, bz )..............................2 C (cx, cy , cz )...............................3


where T=f (x, y, z)

AT BT CT

a b c

2 x ,i

2 a y ,i a z2,i .......................4

2 x ,i

2 b y ,i bz2,i .......................5

2 x ,i

2 c y ,i c z2,i ........................6

where i = 1, 2 N

N is the number of segments of accelerometer data obtained.


The test data obtained is divided into window frames of length W and also a overlapping of 50%. The data is processed in the similar way to that of training samples.

5.3 Classification
Once the features are available, they can be used to train the classes and be used to label each class to an activity as given in the figure [8].
Walking class Running class Sitting class

Variance

Mean

Figure 8: Pre-Processing

After training the classes i.e. activities, test data features are obtained in similar way to that of the training class. We use K Nearest Neighbour classifier to do the classification by obtaining the test data and comparing it with the K nearest neighbours of the training class data. Euclidean distances are calculated between the training class and test data and nearest distance to a particular class are considered to label the corresponding class/activity to the test data. Also as the K value increases the accuracy also increases. Figure [9] gives the graphical implementation of the algorithm.
Walking class Running class Sitting class

Variance

Test Sample??

Mean
Figure 9: Classification Process

6. Evaluation
Using the above mentioned formulae and procedures, we carry out the experiment. In this report, I have implemented 2 variations of KNN algorithms. One is the normal KNN algorithm, the other being the Clustered KNN algorithm [6].Fig [10] and fig [11] gives the flow of the algorithms implemented for classification. The variations have been done to optimize the accuracy and output of the classifier while implementing the 2 different algorithms and also finding the performance of the outputs obtained. Offline monitoring and classification has its own advantages and benefits, but if a doctor has to monitor his patients health regularly and if a coach has to improve the fitness of his team every day such that he could bring new and optimal methods to his training, he has to use online and real time monitoring and classification. Here evaluation is carried out offline for both the algorithms and the results have been documented.

Pre-Processing

Read the accelerometer data for training classes

Calculate mean and variance of all the accelerometer data Calculate the mean and variance of all the accelerometer data

Train the values obtained according to respective classes

Proceed to classification

Classification

Obtain the test accelerometer data

Classify the test data to the class with the K nearest neighbours

Test data Class

Figure 10: KNN algorithm

Pre-Processing and classification process flow for the KNN algorithms has been shown for activity recognition taking into account the K nearest points.

Pre-Processing

Read the accelerometer data for training classes

Calculate mean and variance of all the accelerometer data Calculate the mean and variance of all the accelerometer data

Take K nearest points to the mean of accelerometer values Classify the test data to the K nearest points obtained from the pre-processing step

Plot them on the graph and Proceed to classification

Classification

Obtain the test accelerometer data

Select K nearest neighbours to the K nearest training points

Label the class

Figure 11: Clustered KNN algorithm

In the basic KNN algorithm, 15 training samples were collected with 5 samples corresponding to each class or activity. Some of the samples were collected from people of different age and used to train data. Preprocessing and classification were done offline based on the above mentioned flowcharts for both the variations of KNN algorithms. The graphs in the fig [12], fig [13], fig [14], fig [15] which depicts the

training classes and also includes pre-processing and classification steps based on KNN algorithms. From the graphs it is visible that the training samples obtained from Clustered KNN are concentrated closely. Clustered KNN algorithm also uses less data to train samples as compared to basic KNN algorithm.

Figure 12: Trained Classes/Activities

Figure 13: Classification and recognition

Experiments have been carried out with other machine learning algorithms such as Decision trees, Support Vector Machine and Hidden Markov models which have produced better results and accuracy for a variety of activities and a large number of training and test samples[3][4][5].

Figure 14: Clustered KNN training

Figure 15: Clustered KNN Activity Recognition

Table: 1

Class/Activity Run Sit Walk

Basic KNN Algorithm 0 18 9

Clustered KNN Algorithm 0 10 17

7. Conclusion and Future work


The aim of the paper was to realize activity recognition using wearable sensors. In this paper, I have implemented KNN classification algorithm for recognition of 3 basic activities of a human which include sitting, walking and running. However the recognition was done offline. It can be implemented and monitored real time using other classifier algorithms as KNN algorithm consumes more memory/resources and causes computational problems. Since it was only recognition of 3 activities and also there was less number of samples for training the classes, output obtained was accurate. Clustered KNN results were not matching to that of basic KNN algorithm. From the table [1] and fig[15]it can be seen that it classified some of the sitting class as walking class. However the accuracy decreases and the activity classification complexity increases with the increase of number of activities and samples. Other machine learning algorithms have been proposed which give a better and accurate result for activity recognition. Activity recognition can be done by implementing algorithms monitoring in real time [6].

8. Acknowledgements
I would like to thank my tutor Dipl-Ing. Franois Philipp, TU-Darmstadt for guiding me throughout the seminar and also providing me TI EZ-430 Chronos watch to carry out my experiment. I would also like to thank my friends Ganapati Hegde and Prashanth Venkatesha for being part of my experiment in obtaining the training samples for activity recognition.

9. Bibliography
1. Hong, Y-J., Kim, I-J., Sang Chul Ahn and Kim, H-G. Activity Recognition using Wearable Sensors for Elder Care, 2008 Second International Conference on Future Generation Communication and Networking. pp. 302-305, Dec 13-15, 2008. 2. http://www.ess.tu-darmstadt.de/ 3. Activity classification using a single wrist-worn accelerometer,Saisakul Chernbumroong1, 2, Anthony S. Atkins2, and Hongnian Yu2 1College of Arts, Media and Technology, Chiang Mai University, Chiang Mai, Thailand 2Faculty of Computing, Engineering and Technology, Staffordshire University, Stafford, UK 4. A Feature Set Evaluation for Activity Recognition with Body-Worn Inertial Sensors Syed Agha Muhammad1, Bernd Niklas Klein2, Kristof Van Laerhoven1, and Klaus David2 1 TU Darmstadt, Darmstadt, Germany {muhammad,kristof}@ess.tu-darmstadt.de 2 University of Kassel, Kassel, Germany {niklas.klein,klaus.david}@comtec.eecs.uni-kassel.de

5. Gesture Recognition with a 3-D Accelerometer,Jiahui Wu1, Gang Pan1, Daqing Zhang2, Guande Qi1, and Shijian Li1 1 Department of Computer Science Zhejiang University, Hangzhou, 310027, China {cat_ng,gpan,shijianli}@zju.edu.cn 2 Handicom Lab, Institut TELECOM SudParis, France daqing.zhang@it-sudparis.eu 6. Online Human Activity Recognition on Smart Phones,Mustafa Kose, Ozlem Durmaz Incel, Cem Ersoy Computer Engineering Department Bogazici University, Istanbul, Turkey {mustafa.kose, ozlem.durmaz, ersoy}@boun.edu.tr

7. http://processors.wiki.ti.com/index.php/EZ430Chronos?DCMP=Chronos&HQS=Other+OT+chronoswiki 8. Wireless Sensor Networks Lab-Lecture 5: Sensing in WSNs,Kristof Van Laerhoven,kristof@ess.tudarmstadt.de, http://www.ess.tu-darmstadt.de 9. http://www.sciencedirect.com/science/article/pii/S1753193407004384 10. http://www.thewikimag.com/Cell_phone_reviews-detail/apple-iphone-4-8gb/ 11. http://en.wikipedia.org/wiki/File:Wiimote.png