Video Textures Using the Auto-Regressive Process (SIGGRAPH: Sketches and Applications)

Recently, there have been attempts at creating `video textures', that is, synthesising new video clips based on existing ones. Schodl et al. showed new video clips by carefully choosing sub-loops of an original video sequence that could be replayed. We present a different approach to recreating (potentially infinitely long) new sequences. We transform each frame of the video into an eigenspace using Principal Components Analysis (PCA) so that the original sequence can be viewed simply as a signature through this low-dimensional space (see Gibson et al. for another example of using PCA to assist animators). A new sequence can be generated by moving through this space and creating `similar' signatures. This similarity is derived using the auto-regressive process (ARP) as discussed by Blake and Isard. A 2nd-order process is used and provides a statistical framework for assessing the quality of the new sequence. The new signature has the 2nd-order properties of the original and is much more than simply a random-walk through the space. New sequences created with our approach can contain images never present in the original sequence and are very convincing.