Abstract

The unprecedented popularity of Avatar has pushed the entire movie industry to rapidly evolve to be able to produce “3D” movies. In the past year, three main techniques of producing them have been used each with their advantages and flaws. We review them using as examples movies MPC has been involved in: Pirates of the Caribbean: On Stranger Tides, The Chronicles of Narnia: The Voyage of the Dawn Treader and Harry Potter and the Deathly Hallows: Part 2

In the following presentation, we will discuss the principles of each technique, their pros and cons, and the changes they impose on previous workflows and mindsets.

Native Stereo

Native Stereo involves filming with two cameras (one for each eye). Cameras are mounted together using specially built rigs that ensure proper alignment and synchronization between each camera. Having to deal with two sequences of images per shot brings several complications to the VFX process.

It is very hard to acquire completely identical (apart from horizontal shift) sets of images. Even with the latest and most advanced stereo camera rigs slight misalignments or distortion can be present. As in most rigs the slave camera films though a mirror, there is often color shifts between the two views. Using Ocula we were able to re-align and color match the plates. We also adjusted the pipeline around this new process to make it very transparent to the compositors.

Once those differences are rectified the task at hand is still quite large, as many of the processes have to be done twice (one for each view). This is the case for matchmove, paint, and rotoscoping but with the help of modern software it can be made simpler. Having to manage two points of view creates new challenges for compositing and rendering. For this, we leveraged Nuke and Pixar RenderMan to simplify the task, and adjusted our pipeline accordingly.

Post Conversion

We call “post-conversion” the process of creating a stereo version of a mono film. Other terms for this process are “dimensionalization” and “2D to 3D”. This technique was used to create the 3D version of The Chronicles of Narnia: The Voyage of the Dawn Treader.

Creation of the second view almost always requires rotoscoping a large portion of the image to isolate the parts that need to go at different depths. In general, very accurate detail is needed.

Using solely rotoscoping and image shifting can create a cardboard cut-out effect. In those cases a geometry reconstruction of the scene using matchmove and camera modeling is necessary. The scene is then rendered using projections. Another possible approach is to create depth maps for each object and combine them. Each method has situations to which it is better suited than the other.

Whether we choose depth shifting or projection, occluded areas are revealed that need to be painted. This is one of the most time-consuming tasks of the post-conversion process. Using custom edge reconstruction or temporal algorithms we can reduce the manual painting needed.

Mixed Stereo

For the last of the Harry Potter series Harry Potter and the Deathly Hallows: Part 2 the Mixed Stereo technique was used. Mixed Stereo represents the hybrid approach. The goal is to use the two previously described techniques where they are most appropriate.

This method has gained a lot of popularity in the last year as it can give results that are similar to Native Stereo without some of the cost and filming complications.

Scheduling Mixed Stereo is hard due to the much larger number of tasks and their interdependencies. But using a parallel approach where the film plates get processed at the same time as the visual effects are developed and incorporated we can improve conversion quality while keeping tight deadlines.

The geometric 2D to 3D technique is particularly useful in mixed stereo projects and we made heavy use of VFX assets to speed up the conversion process while improving the overall quality of the output. Most of the CG elements were rendered using stereo cameras, therefore achieving optimal quality.