Practical Generation of Video Textures using the Auto-Regressive Process

Recently, there have been several attempts at creating `video textures', that is, synthesising new (potentially infinitely long) video clips based on existing ones. One way to do this is to transform each frame of the video into an eigenspace using Principal Components Analysis so that the original sequence can be viewed as a signature through this low-dimensional space. A new sequence can be generated by moving through this space and creating `similar' signatures. These signatures may be derived using an auto-regressive process. Such an auto-regressive process assumes that the signature has Gaussian statistics. For many sequences this assumption is valid, however, some sequences are strongly non-linearly correlated, in which case their statistical properties are non-Gaussian. We show two methods by which such non-linearities may be overcome. The first is by modelling the non-linearity automatically using a spline, and the second using a combined appearance model. New sequences created using these approaches can contain images never present in the original sequence and are very convincing.