Encoding High-Resolution 360 and 180 Video for Oculus Quest and Oculus Go [Updated to Include h.265 Support]
Oculus Creators Blog
|
Posted by Eric Cheng
|
October 15, 2019
|
Share

Note From Oculus Team: This article was originally posted on January 15th, 2019 and was updated for accuracy and comprehensiveness in October, 2019.


Oculus Quest and Oculus Go are fantastic VR headsets for experiencing immersive media. They can decode high-resolution video files that are close to saturating display resolutions (up to 5.7K), providing peak-quality viewing experiences. This article details the processes by which high-resolution video files should be encoded to play at high quality in Oculus headsets.

Although Oculus Quest and Go can decode high-resolution video up to 5760x2880 / 30fps, some encoding workflows can result in the export of files that show visible corruption artifacts when played back in Oculus Gallery, especially when encoding in h.264 with long gaps between iFrames.

The reason for these artifacts is that motion vector range (mvrange) can exceed levels permissible by the h.264 level 5.2 specification, and video encoders can open mvrange to level 6 [-32768, 32767] when encoding video at the high resolutions that fit into level 6. While the video decoders in Oculus Quest and Go allow for resolutions exceeding the level 5.2 spec, they require that mvrange stay within level 5.2 compliance [-2048, 2047]. When encoding h.264 using libx264, ffmpeg allows for level 5.2 mvrange compliance by using the arguments, "-x264-params mvrange=511". mvrange needs to be under 512 to be in compliance.

The maximum recommended video resolutions for Oculus Quest and Go are as follows:

3840x3840 30fps H264 H265

4096x4096 30fps ---- H265

5760x2880 30fps H264 ----

2880x5760 30fps ---- H265 (* see below)

2880x2880 60fps H264 H265

4096x2048 60fps H264 H265

Recommended peak-quality resolutions are:

3D-360:

3840x3840 30fps H264 H265

4096x4096 30fps ---- H265

2880x2880 60fps H264 H265

360 / 3D-180:

5120x2560 30fps H264 ----

2560x5120 30fps ---- H265 (*1)

4096x2048 60fps H264 H265

(*1) h.265 playback is limited to 4096 horizontal pixels, but can extend beyond 4096 vertically. Because of this, we encode high-resolution h.265 in 1:2 aspect ratio instead of 2:1.

We recommend 5120x2560 for 3D-180 (2560x2560 per eye) because it will effectively saturate the display resolution in Oculus Go. We recommend encoding videos below 150Mbps, but lower bitrates are usually preferred to reduce distribution size and for a performance safety margin. Regardless of your chosen bitrate for encoding, it's important to test playback in the headset before distribution. Using a crf value in ffmpeg along with setting maxrate is good practice.

The following sample ffmpeg commands will encode 3D-180 video optimally for Oculus Quest and Go (5120x2560 at 30fps), with an arbitrary maximum bitrate of 50Mbps to keep distribution size reasonable. This is usually sufficient for visually-lossless encoding of locked-off (tripod-mounted) shots at 5K, but moving-camera scenes will require much higher bitrates.

h.264, 30fps (side-by-side 3D-180 video):

ffmpeg -i "input.mp4" -c:v libx264 -preset fast -crf 18 -x264-params mvrange=511 -maxrate 50M -bufsize 25M -vf "scale=5120x2560" -pix_fmt yuv420p -c:a aac -b:a 160k -movflags faststart "output_h264.mp4"

h.265, 30fps (side-by-side 3D-180 video re-arranged to over/under):

ffmpeg -i "input.mp4" -c:v libx265 -preset fast -crf 18 -maxrate 50M -bufsize 25M -vf "stereo3d=sbs2l:abl, scale=2560x5120" -pix_fmt yuv420p -aspect 1:2 -c:a aac -b:a 160k -movflags faststart "output_h265.mp4"

The same encoding recommendations apply for encoding 360 video, except that the vertical aspect ratios for h.265 playback require a player that transposes (rotates). See the discussion at the bottom for more about high-resolution 360 and h.265. (*2)

Note that if you're encoding h.264 on a machine with many CPU cores, you might need to include "-threads 16" for libx264 stability. libx265 does not have this stability issue, and can be run without limiting threads (tested on an 18-core machine).

During playback, Oculus Gallery will automatically choose the correct stereoscopic arrangement based on video aspect ratio.

I've provided sample video files that illustrate the mvrange problem and solution, which can be downloaded here.To play these files in Oculus Gallery, sideload the two 5k video files by copying them to your Oculus headset’s file system, or you can play them from a media server or Dropbox (using Gallery).

Note that encoding recommendations are different for sharing 360 and 180 videos to Facebook. Please see this article at the Facebook Help Center for more information.

- Eric Cheng




(*2) Additional discussion about high-resolution 360 and encoding in h.265:

Encode with 90 degree rotation and inject inverse rotation metadata into the container (add your own quality settings, as they are excluded in the command below for simplicity).

MP4:

ffmpeg -i "5k_input.mp4" -y -vf "transpose=1" -c:v libx265 -c:a copy temp.mp4 && ffmpeg -i "temp.mp4" -y -c copy -metadata:s:v:0 rotate=90 output_h265_rotated90.mp4 && rm -f temp.mp4

MKV:

ffmpeg -i "5k_input.mkv" -y -vf "transpose=1" -c:v libx265 -c:a copy output_h265_transpose1.mkv

The above ffmpeg commands strips the spherical metadata, so you may want to re-inject MP4s using Spherical Metadata Injector, Adobe Premiere Pro / Media Encoder, or VR180 Creator.

If you are working with higher-order spatial audio, you will likely be using Matroska Video (MKV). Unfortunately, the MKV container doesn’t have a way to hold rotation metadata. As a workaround, Oculus Gallery will automatically rotate during playback 360 videos with a vertical aspect ratio in MKV. If you are implementing your own player, you will have to manually account for transposition in video in MKV containers.

In general, working with transposed video is off the standard path, so additional experimentation may be required. If you don’t need the file size savings of h.265, working in h.264 without transposition will be easier.