Christian Richardt1    James Tompkin2    Jordan Halsey3    Aaron Hertzmann4    Jonathan Starck5    Oliver Wang4

1 University of Bath       2 Brown University       3 VR Playhouse       4 Adobe Research       5 Foundry

SIGGRAPH 2017 Course

Thursday 3 August 2017, 9:00–12:15, Los Angeles Convention Center, Room 403AB

Video for VR – Teaser image

Abstract

Video can capture the dynamic appearance of the real world in a way no other technology does; virtual reality technology, on the other hand, enables the display of dynamic visual content with unparalleled realism and immersion. The fusion of these two technologies—video for virtual reality (VR)—promises to enable many exciting photo-realistic experiences.

Over half a day, this course will provide an overview of three aspects of this exciting medium: the technical foundations, current systems in practice, and the potential for future systems of VR video.

In the first section, we will explore the geometric and optical problems underpinning VR video. Then, we will introduce both 360 degree video and stereoscopic video, including how 360 video is captured, analyzed, and stitched, including the mathematics behind how stereo 360 video can be captured.

This background material provides the prerequisites for understanding current systems in use. In the middle hour of the course, we explain how state-of-the-art stereo 360 video is produced from camera systems and computational processing. Then, we will consider the art of storytelling in VR, and how new tools for editing VR video can aid in the craft of this art production. Finally, this section provides an industry perspective covering current production and post-production choices and practice, including CG integration.

The final part of our course focuses on the next generation of video for VR, where we move to 6 degrees-of-freedom (6DoF) experiences. We introduce the basics and challenges behind light field cameras, processing and displays, and see how they can enable 6DoF experiences. This will be followed by another industry perspective on how light field camera arrays have been used to create cutting-edge experiences integrating volumetric live-action elements. To conclude the course, we will see how far we still must go toward the ideal system, in hopes of inspiring the attendees to push the boundary farther to reach it.

We hope this course is useful to a broad audience—at SIGGRAPH and beyond—as we cover the academic, artistic, and production sides of VR video.


Course Schedule

Time Topic
9:00 Welcome and Introduction
James Tompkin, Brown University
9:15 360 Spherical Video
Oliver Wang, Adobe Research
9:35 Stereoscopic 3D Videos and Panoramas
Christian Richardt, University of Bath
9:55 Q&A followed by a short break
all presenters
10:10 Industry Perspective: 360 Video Cameras in Production
Jonathan Starck, Foundry
10:30 Art, Storytelling, and Tools
Aaron Hertzmann, Adobe Research
10:50 Industry Perspective: Post-production Pipelines for 360 Video
Jonathan Starck, Foundry
11:10 Q&A followed by a short break
all presenters
11:25 Light Field Video Basics and Challenges
James Tompkin, Brown University
11:45 Industry Perspective: Light Field Production and Post Production
Jordan Halsey, VR Playhouse
12:05 Closing and Q&A
all presenters

Downloads

Bibtex

@inproceedings{Video4VR,
  author    = {Christian Richardt and James Tompkin and Jordan Halsey and Aaron Hertzmann and Jonathan Starck and Oliver Wang},
  title     = {Video for Virtual Reality},
  booktitle = {SIGGRAPH Courses},
  year      = {2017},
  doi       = {10.1145/3084873.3084894},
  url       = {http://richardt.name/pub/Video4VR/},
}