OmniPhotos: Casual 360° VR Photography

Tobias Bertel    Mingze Yuan    Reuben Lindroos    Christian Richardt

University of Bath

ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020)
Emerging Technologies at SIGGRAPH Asia 2020

Non-technical explainer video produced by Science Animated.

Abstract

Virtual reality headsets are becoming increasingly popular, yet it remains difficult for casual users to capture immersive 360° VR panoramas. State-of-the-art approaches require capture times of usually far more than a minute and are often limited in their supported range of head motion. We introduce OmniPhotos, a novel approach for quickly and casually capturing high-quality 360° panoramas with motion parallax. Our approach requires a single sweep with a consumer 360° video camera as input, which takes less than 3 seconds to capture with a rotating selfie stick or 10 seconds handheld. This is the fastest capture time for any VR photography approach supporting motion parallax by an order of magnitude. We improve the visual rendering quality of our OmniPhotos by alleviating vertical distortion using a novel deformable proxy geometry, which we fit to a sparse 3D reconstruction of captured scenes. In addition, the 360° input views significantly expand the available viewing area, and thus the range of motion, compared to previous approaches. We have captured more than 50 OmniPhotos and show video results for a large variety of scenes.

Overview

OmniPhotos are a new approach for casual 360° VR photography using a consumer 360° video camera.

Downloads

Capture approach

OmniPhotos is the fastest casual 360° VR photography approach on Earth thanks to a revolutionary capturing technique.

We put a 360° video camera on a rotating selfie stick to make the camera motion smoother, more repeatable and fast – 1.7 seconds on average.

Datasets

All data supporting this publication is openly available from the University of Bath Research Data Archive.

Please note:
The original videos for “Ballintoy-5.7K” and “Cathedral” are incomplete; the missing files are here (340 MB).

Comparisons to 3D approaches

Approaches based on 3D reconstruction often fail for fine geometry or uniform regions.

OmniPhotos achieve better visual results thanks to scene-adaptive proxy geometry and flow-based blending for aligning texture details.

References:

Comparison to IBR approaches

Previous image-based rendering methods with flow-based blending rely on basic proxy geometry, which causes vertical distortion artefacts.

Our scene-adaptive proxy geometry deforms to fit the scene more closely, which greatly reduces vertical distortion artefacts.

References:

Virtual Reality

OmniPhotos are immersive 360° VR photographs with high-quality motion parallax and a large 1-metre head box.

Try out our demo (with or without VR):

You can also download just our precompiled binaries:

Our demo and precompiled binaries are for Windows 10 x64. Both include batch files for downloading up to 31 OmniPhotos. VR experience requires an OpenVR compatible headset.

Supplemental video for our paper.

Acknowledgements

We thank the reviewers for their thorough feedback that has helped to improve our paper. We also thank Peter Hedman, Ana Serrano and Brian Cabral for helpful discussions, and Benjamin Attal for his layered mesh rendering code.

This work was supported by EU Horizon 2020 MSCA grant FIRE (665992), the EPSRC Centre for Doctoral Training in Digital Entertainment (EP/L016540/1), RCUK grant CAMERA (EP/M023281/1), an EPSRC-UKRI Innovation Fellowship (EP/S001050/1), a Rabin Ezra Scholarship and an NVIDIA Corporation GPU Grant.

Copyright

© Copyrights by the Authors, 2020. This is the authors’ version of the work. It is posted here for your personal use. Not for redistribution. The definitive version is published in ACM Transactions on Graphics.

Bibtex

@article{OmniPhotos,
  author    = {Tobias Bertel and Mingze Yuan and Reuben Lindroos and Christian Richardt},
  title     = {{OmniPhotos}: Casual 360° {VR} Photography},
  journal   = {ACM Transactions on Graphics},
  year      = {2020},
  volume    = {39},
  number    = {6},
  pages     = {266:1--12},
  month     = dec,
  issn      = {0730-0301},
  doi       = {10.1145/3414685.3417770},
  url       = {https://richardt.name/omniphotos/},
}