An approach to automatic recognition of spontaneous facial actions.pdf

An Approach to Automatic Recognition of Spontaneous Facial Actions B. Braathen, M.S. Bartlett, G. Littlewort, E. Smith, and J.R. Movellan Institute for Neural Computation University of California, San Diego Email: bjorn, marni, gwen, evan, javier @inc.ucsd.edu Abstract We present ongoing work on a project for automatic recognition of spontaneous facial actions. Spontaneous fa- cial expressions differ substantially from posed expressions, similar to how spontaneous speech differs from directed speech. Previous methods for automatic facial expression recognition assumed images were collected in controlled environments in which the subjects deliberately faced the camera. Since people often nod or turn their heads, auto- matic recognition of spontaneous facial behavior requires methods for handling out-of-image-plane head rotations. There are many promising approaches to address the prob- lem of out-of-image plane rotations. In this paper we ex- plore an approach based on 3-D warping of images into canonical views. A front-end system was developed that jointly estimates camera parameters, head geometry and 3- D head pose across entire sequences of video images. First a small set of images was used to estimate camera param- eters and 3D face geometry. Markov chain Monte-Carlo methods were then used to recover the most likely sequence of 3D poses given a sequence of video images. Once the 3D pose was known, we warped each image into frontal views with a canonical face geometry. We evaluated the perfor- mance of the approach as a front-end for a spontaneous ex- pression recognition system using support vector machines and hidden Markov models. This system employed general purpose learning mechanisms that can be applied to recog- nition of any facial movement. We showed that 3D tracking and warping followed by machine learning techniques di- rectly applied to the warped images, is a viable and promis- ing technology for automatic facial expression recognition. One exciting