Manifold-Manifold Distance with Application to Face Recognition based on Image Set.pdf

Abstract In this paper, we address the problem of classifying image sets, each of which contains images belonging to the same class but covering large variations in, for instance, viewpoint and illumination. We innovatively formulate the problem as the computation of Manifold-Manifold Distance (MMD), i.e., calculating the distance between nonlinear manifolds each representing one image set. To compute MMD, we also propose a novel manifold learning approach, which expresses a manifold by a collection of local linear models, each depicted by a subspace. MMD is then converted to integrating the distances between pair of subspaces respectively from one of the involved manifolds. The proposed MMD method is evaluated on the task of Face Recognition based on Image Set (FRIS). In FRIS, each known subject is enrolled with a set of facial images and modeled as a gallery manifold, while a testing subject is modeled as a probe manifold, which is then matched against all the gallery manifolds by MMD. Identification is achieved by seeking the minimum MMD. Experimental results on two public face databases, Honda/UCSD and CMU MoBo, demonstrate that the proposed MMD method outperforms the competing methods. 1. Introduction In traditional visual recognition task, objects of interest are trained and recognized from only a few samples. However, with the increase of available video cameras and large capacity storage media, many new applications are emerging in which the image quantity of each object of interest for both training and testing can be very large. For example, as shown in Fig.1, nowadays, in many face recognition applications, a great number of images for each known subject have been able to be collected from video sequences or photo album, and recognition can also be conducted with a set of probe images rather than single probe image. In other words, recognition can be formulated as matching a probe image set against all the galler