Incremental Non-Rigid Structure-from-Motion with Unknown Focal Length

Thomas Probst, Danda Pani Paudel, Ajad Chhatkuli, Luc Van Gool; The European Conference on Computer Vision (ECCV), 2018, pp. 756-771

The perspective camera and the isometric surface prior have recently gathered increased attention for Non-Rigid Structure-from-Motion (NRSfM). De- spite the recent progress, several challenges remain, particularly the computa- tional complexity and the unknown camera focal length. In this paper we present a method for incremental Non-Rigid Structure-from-Motion (NRSfM) with the perspective camera model and the isometric surface prior with unknown focal length. In the template-based case, we provide a method to estimate four param- eters of the camera intrinsics. For the template-less scenario of NRSfM, we pro- pose a method to upgrade reconstructions obtained for one focal length to another based on local rigidity and the so-called Maximum Depth Heuristics (MDH). On its basis we propose a method to simultaneously recover the focal length and the non-rigid shapes. We further solve the problem of incorporating a large number of points and adding more views in MDH-based NRSfM and efficiently solve them with Second-Order Cone Programming (SOCP). This does not require any shape initialization and produces results orders of times faster than many methods. We provide evaluations on standard sequences with ground-truth and qualitative re- constructions on challenging YouTube videos. These evaluations show that our method performs better in both speed and accuracy than the state of the art.