Analysis and Synthesis of Behavioural Specific Facial Motion

This thesis presents work concerning the analysis of behaviour specific facial motion and its automatic synthesis. Psychology research has shown that facial motion provides important cues to the human visual system for recognition of emotion, identity and gender. Similarly, in Computer Vision facial motion information has been used in face and facial expression recognition. However, the fact that facial motion is behaviour specific has not yet been exploited in facial animation systems. Two parametric modelling techniques have been evaluated, MultiVariate AutoRegressive (VAR) temporal modelling and a tensor framework for modelling facial motion dynamics. Both modelling techniques adopt a ‘black box’ approach to facial motion modelling, where the emphasis of modelling is on motion information and not on textural information. Of these methods, VAR modelling is found to be more suitable for motion synthesis. Nevertheless, it is found that the tensor framework is more suited than VAR modelling as a potential tool for facial motion analysis. It is found that the VAR modelling technique encapsulated the temporal and motion dynamics of facial motion behaviour. VAR models constructed from behaviour specific facial motion generate facial motion sequences which are similar but non-identical to the original facial motion training data (i.e. ‘synthesis by example’). These sequences are novel and can be indefinitely long. Moreover, VAR model analysis demonstrated that the models themselves are behaviour specific. VAR models constructed from gender specific motion were found to have similar statistics to the original motion data. Using these models a human psychology experiment was conducted where it was found that participants could distinguish between synthetic male and female facial motion. Emotion specific VAR models were incorporated into a prototype animation tool. This tool enables a user to explore an interactive ‘emotion space’ and by traversing this space novel emotion specific sequences are automatically generated ‘on the fly’. The study shows that the tensor framework can encapsulate facial motion information. Using speed domain facial motion tensor framework (which encapsulates motion information only) can be used successfully for facial motion recognition (emotion and gender), with recognition greater than chance. Furthermore, it is found that tensor gender recognition results were comparable to those of a human psychology experiment when both utilised the same facial motion data. The results and observations of VAR modelling and tensor framework testing corroborate psychology and computer vision research that motion alone is sufficient to encapsulate emotion specific and gender specific information. Additionally, results indicate that emotion specific information is encoded in a shorter temporal period than gender/identity specific information. This has implications in the automatic synthesis of new facial motion sequences and possible animation systems which wish to exploit motion to generate believable realistic CG characters. In addition the recognition of emotion and gender from just motion/speed information could have relevance to security systems where textural information (low resolution imagery) is poor.