Robust deformable shape reconstruction from monocular video with manifold forests

Lili Tao*, Bogdan J. Matuszewski

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)peer-review

3 Citations (Scopus)
230 Downloads (Pure)


Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object’s shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a low-dimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.

Original languageEnglish
Pages (from-to)801-819
Number of pages19
JournalMachine Vision and Applications
Issue number6
Early online date19 May 2016
Publication statusPublished - 1 Aug 2016


  • Deformable shape reconstruction
  • Manifold forests
  • Missing data and outliers
  • Nonlinear manifold learning


Dive into the research topics of 'Robust deformable shape reconstruction from monocular video with manifold forests'. Together they form a unique fingerprint.

Cite this