Lech Świrski    Christian Richardt    Neil A. Dodgson

Computer Laboratory, University of Cambridge, United Kingdom

Poster at SIGGRAPH 2011Winner of the ACM SIGGRAPH Student Research Competition


Abstract

A common technique in documentaries, when video footage is not available, is to animate photographs by panning across them slowly. More recently, it has become popular to divide such photographs into layers, and to animate these layers as moving over each other to create a motion parallax effect, commonly known as the “3D Ken Burns effect”. Although this effect is now ubiquitous in documentaries, producing it involves a laborious manual process that requires hours of manual layer segmentation, clone-brushing, positioning in 3D, and adjusting the panning speeds of individual layers.

Given depth information, most of this work can be automated. This project investigated how this could be achieved in practice. I developed a novel workflow which, given an image with depth, mimics the manual creation of this motion parallax effect, by creating a layered image representation. Objects in the image are segmented into separate layers, and the regions behind them are filled by inpainting from the surrounding background. This imitation of the manual process gives a user the means to adjust the layers at various stages; for example, objects can be manually marked for segmentation, or automatically filled regions can be augmented using human knowledge of the scene. However, the amount of user interaction needed is usually minimal; for example, precise segmentation can be achieved with only a rough labelling.

The final result of the system is a layered image which contains colour and depth information for each layer, and can be rendered as the user desires. The project also included a real-time mesh-based renderer that can render the layered image from novel views, with an optional depth-of-field effect.

Video

Downloads

Bibtex

@inproceedings{LayeredPhotoPopup,
    author = {Lech \'Swirski and Christian Richardt and Neil A. Dodgson},
    title = {Layered Photo Pop-Up},
    booktitle = {SIGGRAPH Posters},
    year = {2011},
    month = aug,
    note = {Winner of the ACM SIGGRAPH Student Research Competition},
    doi = {10.1145/2037715.2037757},
    url = {http://richardt.name/photopopup/},
}