Lucas Thies¹    Michael Zollhöfer²    Christian Richardt^2,3,4    Christian Theobalt²    Günther Greiner¹

¹ University of Erlangen-Nuremberg       ² MPI Informatik       ³ Intel Visual Computing Institute       ⁴ University of Bath

International Conference on 3D Vision (3DV) 2016

---

Abstract

We present a novel approach for real-time joint reconstruction of 3D scene motion and geometry from binocular stereo videos. Our approach is based on a novel variational halfway-domain scene flow formulation, which allows us to obtain highly accurate spatiotemporal reconstructions of shape and motion. We solve the underlying optimization problem at real-time frame rates using a novel data-parallel robust non-linear optimization strategy. Fast convergence and large displacement flows are achieved by employing a novel hierarchy that stores delta flows between hierarchy levels. High performance is obtained by the introduction of a coarser warp grid that decouples the number of unknowns from the input resolution of the images. We demonstrate our approach in a live setup that is based on two commodity webcams, as well as on publicly available video data. Our extensive experiments and evaluations show that our approach produces high-quality dense reconstructions of 3D geometry and scene flow at real-time frame rates, and compares favorably to the state of the art.

Bibtex

@inproceedings{HalfwayDomainSceneFlow,
  author    = {Lucas Thies and Michael Zollh{\"o}fer and Christian Richardt and Christian Theobalt and G{\"u}nther Greiner},
  title     = {Real-time Halfway Domain Reconstruction of Motion and Geometry},
  booktitle = {International Conference on 3D Vision (3DV)},
  year      = {2016},
  month     = {October},
  pages     = {450--459},
  doi       = {10.1109/3DV.2016.55},
  url       = {http://richardt.name/publications/halfway-domain-scene-flow/},
}