Machine learning methods for fully automatic recognition of facial expressions and facial a.pdf

Machine Learning Methods for Fully Automatic Recognition of Facial Expressions and Facial Actions Marian Stewart Bartlett, Gwen Littlewort, Claudia Lainscsek, Ian Fasel, Javier Movellan Institute for Neural Computation University of California, San Diego San Diego, CA 92093-0523 Abstract We present a systematic comparison of machine learning methods applied to the problem of fully automatic recogni- tion of facial expressions. We explored recognition of facial actions from the Facial Action Coding System (FACS), as well as recognition of full facial expressions. Each video- frame is first scanned in real-time to detect approximately upright-frontal faces. The faces found are scaled into im- age patches of equal size, convolved with a bank of Gabor energy filters, and then passed to a recognition engine that codes facial expressions into 7 dimensions in real time: neu- tral, anger, disgust, fear, joy, sadness, surprise. We report results on a series of experiments comparing recognition engines, including AdaBoost, support vector machines, lin- ear discriminant analysis, as well as feature selection tech- niques. Best results were obtained by selecting a subset of Gabor filters using AdaBoost and then training Support Vector Machines on the outputs of the filters selected by AdaBoost. The generalization performance to new subjects for recognition of full facial expressions in a 7-way forced choice was 93% correct, the best performance reported so far on the DFAT-504 dataset. We also applied the system to fully automated facial action coding. The present sys- tem classifies 18 action units, whether they occur singly or in combination with other actions. The system obtained a mean agreement rate of 94.5% on a FACS-coded dataset of posed expressions (DFAT-504). The outputs of the classi- fiers change smoothly as a function of time and thus can be used to measure facial expression dynamics. 1. Introduction We present results on a user independent fully automatic system fo