Color grading immersive media

Color grading and correction is an essential part of immersive media post-production workflow. This article includes tips and best practice advice for professional results.

Color grading is an essential part of any video production, from traditional rectilinear 2D work through to fully immersive 3D (stereoscopic) 360. For the most part color grading for immersive media works in the same way as it does for traditional video, but there are a few key things to understand and watch for:

1. Since many cameras record in a highly compressed format there is often less dynamic range to play with, which places limitations on how far colors and tones are able to be pushed and isolated.
2. When using masks or ‘Power Windows’ to create local adjustments they shouldn't go beyond the equirectangular edges of the footage or a seam line will develop. Any non-global, non-VR aware adjustments can result in a visible seam when working with 360 content. Note how in the following image the mask over the left edge of the frame doesn’t carry over to the opposite side, causing a visible seam when viewing inside a headset.
3. Coloring is done on an equirectangular projection of the image, and this will look different on screen than it does projected onto a sphere inside a headset. Monitoring this VR view is essential to check the work, as elements may feel a lot bigger and more important when viewed in a headset.
4. Grading stereoscopic content requires even more care to ensure all edits are identical in the left and right parts. If edits don’t match up perfectly in both eyes the result will be a little distracting at least and can feel very off-putting. Masks and Power Windows require particular care.

Two of the most powerful color correction software tools with proper support for 360 immersive video are ScratchVR by Assimilate and Mistika Boutique by SGO.

Two other popular color grading tools for immersive media are Blackmagic Design DaVinci Resolve and Adobe Premiere Pro, but these do have some limitations for immersive media work. DaVinci Resolve does not have native 360 support in the color grade panel, meaning that Power Windows do not carry across the stitch line. Adobe Premiere Pro supports in-headset previewing during grading, and for basic masking it can work very well, but it is not able to carry a mask across the stitch line. Premiere also does not currently have native support for stereoscopic media when working with masks. When making detailed masks be careful to duplicate the mask to the other eye and place it accurately. Some third party plugins such as Mocha Pro can be very helpful since they are able to work with stereoscopic media and apply masks with the stereo disparity taken into account.

Keep the following things in mind when grading for immersive media:

• Bright whites can feel overwhelmingly bright in a headset, and if the headset refresh rate isn’t above 72Hz there may be some perceived flickering of the image. Consider bringing your whites down slightly below +100 IRE. In addition, the perception of areas of black can feel different in headsets compared to the equirectangular view because of the amount of black that surrounds the viewer and the way equirectangular projections exaggerate the size of what’s at the top and bottom of the frame.
• Using color and exposure masks to draw attention is a great creative tool. For example, some creators put a subtle gradient at the nadir to darken the area below the camera where nothing important typically happens. Really soft adjustments to the exposure can help draw attention in a scene.
• Not all headsets look the same, and some have very different outputs. It is a good idea to check the graded media in several different models of headset, or at least the headset being targeted for release.
• Headset displays work to specific color spaces, just as regular monitors do. Rec.709 is a useful target for final rendering and will deliver reliable and accurate color in Meta Quest 2 headsets, so for this device grade to Rec.709 unless you have a very specific reason for using a different color space. Do remember, however, that system updates can introduce changes and new standards, and if the aim is to target more than one model or brand of headset then it may be necessary to render slightly different versions from the master files.

Shooting in Log mode where the camera supports it can be very helpful in achieving results that look good and make the most of the sensor’s capabilities and dynamic range. In simple terms Log recording maximizes the information that is captured, using logarithmic gamma curves mapped to the specific camera sensor. It isn’t RAW sensor data, but it still needs processing to turn it from a very flat, grey appearance to a more acceptable range of hues and contrasts. This is done using look-up tables, or LUTs, that are created for a specific camera and convert the Log video image to something more recognizable and useful, such as Rec.709. Different LUTs will give the video different looks, and there are some third-party LUTs as well as those provided by camera manufacturers.