Motion Segmentation Across Image Sequences

Motion segmentation is a fundamental technique for analysing image sequences of real scenes. Motion segmentation 'compresses' the sequence into sets of pixels moving coherently across the sequence with associated motions. We argue the most important component in motion segmentation is motion analysis, particularly \emphmotion estimation and \emphmotion assignment. Traditional motion analysis schemes ignore errors caused by multiple moving regions and the effects caused by \emphkinetic occlusion (i.e., covering/uncovering due to moving objects at differ ent depths). This thesis describes novel techniques which account for these effects. We base our work on analysis within local windows and use iterative refinement to tackle the circularity between motion estimation and assignment. We develop a simple \emphlocal layer model which relates motion/depth configurations and observable effects of \emphkinetic occlusion. We use this to perform robust motion assignment by rating each configuration by how well motion compensated errors fit the model inside regions \emphand at region boundaries. For estimating all the motions applying within a block we develop a technique called \emphpartial correlation. In this we also use the layer model, this time to compensate for depth effects that otherwise can bias the motion estimates. We combine these local analyses using an intermediate \emphmoving object graph where nodes correspond to coherently moving subregions of local analyses. This is a flexible high-level representation which stores global moving objects as components. By adding appropriate links between nodes in graphs for adjacent frames a comprehensive representation can be built up across the sequence. Using this spatio-temporal structure the accuracy of the various analyses can be checked and improved by extending the motion configuration and estimation analyses to work within \emphtemporal windows. Finally we describe how to produce our much more compact final representation, \emphmoving object layers, from the linked moving object graphs and give examples of their use in synthesis applications.