Τεχνικές ψηφιακής επεξεργασίας εικόνας και βίντεο με εφαρμογή στην αναγνώριση συναισθηματικής κατάστασης

Abstract

The main field of interest of this thesis includes computational intelligence and image processing techniques in order to recognize facial expressions in images and video. More specifically, a new method for the real-time recognition of the six facial expressions (anger, disgust, fear, happiness, sadness and surprise) is proposed. The information taken under consideration consists of the coordinate differences extracted from the deformed grids calculated between the neutral state and the greatest facial expression intensity, as depicted in the video. The classification is achieved by using either a novel multi-class Support Vector Machines (SVMs) system for the direct recognition of facial expressions or many two-class SVMs systems for the recognition of facial expressions using the activated Facial Action Units (FAUs) that were detected. Moreover, a new Discriminant Non Negative Matrix Factorization (DNMF) method that uses projected gradients to ensure the stability of the limit point of the optimization problem, is proposed. The method is applied in facial expression recognition and frontal face verification problems. Furthermore, in order to achieve neutral state recognition, a new method that combines two sources of information (texture and shape) to achieve classification is presented. The fusion of the two kinds of information is performed using various methods, such as Median Radial Basis Function (MRBF) Neural Networks (NN) and SVMs, having as an input the outputs of the classification systems for each information source (texture and shape) and as an output the recognized facial expression or the set of the activated FAUs. The recognition of the neutral state makes the recognition of facial expressions in every video frame feasible. Finally, an analysis of the way partial occlusion affects the facial expression recognition is presented. More specifically, the effect of eyes or mouth occlusion on the recognition of the six basic facial expressions by human observers, experts or not, as well as by many facial expression recognition systems, is studied.