Video for Virtual Reality
Christian Richardt1 James Tompkin2 Jordan Halsey3 Aaron Hertzmann4 Jonathan Starck5 Oliver Wang4
1 University of Bath 2 Brown University 3 VR Playhouse 4 Adobe Research 5 Foundry
SIGGRAPH 2017 Course
Thursday 3 August 2017, 9:00–12:15, Los Angeles Convention Center, Room 403AB

Abstract
Video can capture the dynamic appearance of the real world in a way no other technology does; virtual reality technology, on the other hand, enables the display of dynamic visual content with unparalleled realism and immersion. The fusion of these two technologies—video for virtual reality (VR)—promises to enable many exciting photo-realistic experiences.
Over half a day, this course will provide an overview of three aspects of this exciting medium: the technical foundations, current systems in practice, and the potential for future systems of VR video.
In the first section, we will explore the geometric and optical problems underpinning VR video. Then, we will introduce both 360 degree video and stereoscopic video, including how 360 video is captured, analyzed, and stitched, including the mathematics behind how stereo 360 video can be captured.
This background material provides the prerequisites for understanding current systems in use. In the middle hour of the course, we explain how state-of-the-art stereo 360 video is produced from camera systems and computational processing. Then, we will consider the art of storytelling in VR, and how new tools for editing VR video can aid in the craft of this art production. Finally, this section provides an industry perspective covering current production and post-production choices and practice, including CG integration.
The final part of our course focuses on the next generation of video for VR, where we move to 6 degrees-of-freedom (6DoF) experiences. We introduce the basics and challenges behind light field cameras, processing and displays, and see how they can enable 6DoF experiences. This will be followed by another industry perspective on how light field camera arrays have been used to create cutting-edge experiences integrating volumetric live-action elements. To conclude the course, we will see how far we still must go toward the ideal system, in hopes of inspiring the attendees to push the boundary farther to reach it.
We hope this course is useful to a broad audience—at SIGGRAPH and beyond—as we cover the academic, artistic, and production sides of VR video.
Course Schedule
Time | Topic | Final slides |
---|---|---|
9:00 | Welcome and Introduction James Tompkin, Brown University |
PDF (3 MB), PPTX (142 MB) |
9:15 | Introduction to 360° Video Oliver Wang, Adobe Research |
PDF (8 MB), PDF (69 MB) |
9:35 | Stereoscopic 3D Videos and Panoramas Christian Richardt, University of Bath |
PDF (4 MB), PPTX (88 MB) |
9:55 | Q&A followed by a short break all presenters |
|
10:10 | Virtual Reality Video: Storytelling and Tools Aaron Hertzmann, Adobe Research |
PDF (5 MB) |
10:30 | Industry Perspective: 360 Video Cameras in Production Jonathan Starck, Foundry |
PDF (4 MB), PDF (53 MB) |
10:50 | Industry Perspective: Post-production for 360 Video Jonathan Starck, Foundry |
PDF (3 MB), PDF (14 MB) |
11:10 | Q&A followed by a short break all presenters |
|
11:25 | Light Field Video Basics and Challenges James Tompkin, Brown University |
PDF (4 MB), PPTX (98 MB) |
11:45 | Industry Perspective: Light Field Production and Post Production Jordan Halsey, VR Playhouse |
PDF (8 MB), PPTX (7 MB) |
12:05 | Summary and Outlook James Tompkin, Brown University |
PDF (2 MB), PPTX (9 MB) |
Downloads
- Course abstract (PDF, 100 KB)
Bibtex
@inproceedings{Video4VR, author = {Christian Richardt and James Tompkin and Jordan Halsey and Aaron Hertzmann and Jonathan Starck and Oliver Wang}, title = {Video for Virtual Reality}, booktitle = {SIGGRAPH Courses}, year = {2017}, doi = {10.1145/3084873.3084894}, url = {http://richardt.name/pub/Video4VR/}, }