A prototype for automatic recognition of spontaneous facial actions.pdf

A prototype for automatic recognition of spontaneous facial actions M.S. Bartlett, G. Littlewort, B. Braathen, T.J. Sejnowski , and J.R. Movellan Institute for Neural Computation and Department of Biology University of California, San Diego and Howard Hughes Medical Institute at the Salk Institute Email: marni, gwen, bjorn, terry, javier @inc.ucsd.edu Abstract We present ongoing work on a project for automatic recognition of spon- taneous facial actions. Spontaneous facial expressions differ substan- tially from posed expressions, similar to how continuous, spontaneous speech differs from isolated words produced on command. Previous methods for automatic facial expression recognition assumed images were collected in controlled environments in which the subjects delib- erately faced the camera. Since people often nod or turn their heads, automatic recognition of spontaneous facial behavior requires methods for handling out-of-image-plane head rotations. Here we explore an ap- proach based on 3-D warping of images into canonical views. We eval- uated the performance of the approach as a front-end for a spontaneous expression recognition system using support vector machines and hidden Markov models. This system employed general purpose learning mech- anisms that can be applied to recognition of any facial movement. The system was tested for recognition of a set of facial actions defined by the Facial Action Coding System (FACS). We showed that 3D tracking and warping followed by machine learning techniques directly applied to the warped images, is a viable and promising technology for automatic facial expression recognition. One exciting aspect of the approach pre- sented here is that information about movement dynamics emerged out of filters which were derived from the statistics of images. 1 Introduction Much of the early work on computer vision applied to facial expressions focused on rec- ognizing a few prototypical expressions of emotion produced on command