This article describes using Mistika VR to stitch together 6 lens monoscopic 360 video captured with an Insta360 Pro 2. The basics are applicable to most commercially available VR cameras.
Media: Spencer Lindsay / Lindsay Digital
Read Time: 20 Minutes
Mistika VR is a powerful stitching tool that many production teams utilize when creating immersive video for VR. Mistika VR includes presets for most immersive media cameras, and also supports import of camera calibration data for Insta360 Pro, Pro 2, Titan, and cameras from KanDao. It also supports import of gyro metadata for stabilization from the Pro 2, Titan and KanDao cameras.
This article provides a step by step overview of using Mistika VR to stitch together 6 lens monoscopic 360 video captured with an Insta360 Pro 2 camera. The steps that follow are a typical workflow for stitching with Mistika VR. The basics are applicable to most commercially available 360 cameras in use today.
The Pro 2 camera records each lens’ video onto six separate microSD cards. A seventh SD card is used to record the in-camera preview, gyro and calibration data. From the camera's removable media, copy the origin_0.mp4 through origin_5.mp4 media files, the preview.mp4 and the pro.prj files into a folder on the computer which will be running Mistika VR. The origin_#_lrv.mp4 files are low-res versions of the video and will not be used during this stitching process
Organizing the media folder on the computer for Mistika VR
After launching Mistika VR, create a New Project for the video sequence when prompted. Enter in a name for the project. Select the Video Format dimensions and Frame Rate which match those recorded in camera, in this project’s case 7680x3840 and 60 FPS.
Mistika VR new project settings
Select all six origin_0.mp4 through origin_5.mp4 videos located in the folder created in the first step. Drag and load those videos into the Clip Stack on the left hand side of Mistika VR. Do not drag the preview.mp4 or origin_#_lrv.mp4 into the Clip Stack. Mistika VR will automatically import the six origin videos into the sequence.
Select, drag and copy the six camera videos into Mistika VR
Once the six video files are loaded into the Clip Stack, select Files Per Segment and enter the number of videos to be used - in this case 6. Check Use Default Preset in the Camera Stack Options panel, and select the applicable camera preset for the project. This project was shot with an Insta360 Pro 2 camera, in mono at 3840x1920 per lens. The preset is displayed with the camera name and the captured dimensions of each individual lens: Insta360Pro2_3840x1920. If the project had been recorded in stereo, the Pro 2 3840x2880 per lens preset dimensions would have been used.
Select and load the correct preset for the camera and dimensions for the mode used (mono / stereo)
Mistika VR includes presets for a large number of current and legacy immersive media monoscopic and stereoscopic cameras. These presets include camera specific lens and camera characteristics critical for the preliminary stitch. Unsupported or custom-built camera rigs will require manually defining those characteristics in Mistika VR prior to stitching.
Mistika VR supports camera calibration data for the Insta360 Titan and Pro line of cameras, and cameras from KanDao. Calibration data is created for each individual camera when following the manufacturer’s instructions. Slight variations in the camera's lenses may exist, and these are evaluated and recorded during the calibration process. Recalibrating VR cameras before the start of every recording project is a good practice, as the data provides an accurate baseline of the camera's current state. Here are Insta360’s calibration instructions in their Pro 2 User Manual
To use this calibration data, from the Stitch pop-up menu below the Clip Stack, select Use Insta360 Pro Calibrate. Mistika VR will look in the media folder created in the first step for the pro.prj file, which includes the calibration information. Alternatively, the pro.prj file can be dragged from the media folder and loaded in.
Importing camera calibration data
Mistika VR can intelligently analyze and optimize vertical and angular alignment of the stitch for the shot. The stitch can be quickly improved by opening up the Positions pop-up and selecting Improve Offsets first then selecting Improve Angles. Improve Offsets can be applied several times consecutively, with each pass further enhancing the stitch. Reapply Improve Offsets multiple times until there is no visual improvement in the shot. Generally Improve Angles only needs to be run once.
Improve Offsets and Improve Angles will optimize the quick stitch
Optical Flow is a powerful tool which can quickly assist with pixel alignment of scene elements between adjacent lenses. When scene elements display ghosted artifacts between lenses as shown in the illustration below, activating Optical Flow can improve the stitch.
Note the ghosting on the ground crewman, the helicopter windshield frame, and the instrument panel with optical flow off. With optical flow on, the majority of the ghosting is eliminated. The ground crewman's helmet has a slight artifact which is best addressed by using Edge Points. Image: Steve Cooper
To use Optical Flow click on the checkbox found in the Options panel, below the Clip Stack. Optical flow will be immediately activated and can be previewed in the application’s main window, the Visual Editor. Optical flow can be toggled on and off to compare the results.
Optical flow may not work in all situations. Hard vertical lines in the video that move from one lens to another may cause optical flow to produce a ripple artifact that can flicker over time. Experimenting with the optical flow parameters can help alleviate this; lowering the Stitch Feather from 50 to 20 for example. However, the solution will often require Edge Points between lenses to prioritize critical elements in the stitched video. This article covers Using Edge Points in Mistika VR
Optical Flow can be activated on/off in the Options panel. Parameters below can be used to fine tune optical flow.
Many VR cameras, including the Pro 2, support manual controls for each lens. In a locked down lighting shot, manually setting exposure for lenses that face bright light sources like windows can help to balance out the exposure with darker interior areas. However, in the case of moving lights, or a moving camera, it is not advised to lock down exposure for individual lenses.
The Match Color in Time option is located in the Color pop-up menu
Stabilizing moving camera footage is crucial for a comfortable viewing experience in VR. Footage shot from a static camera generally will not require stabilization, but if the camera moves, it will most often require stabilization. The use of mechanical camera stabilizers during shooting can help reduce unwanted tilt and sway, but stabilizing in post will still be beneficial.
In Mistika VR, stabilization runs on the Output Camera, generating a set of S-Yaw, S-Pitch and S-Roll values. These stabilization values will be displayed as keyframed values in the output camera. The Yaw, Pitch and Roll parameters act independently of S parameters and can be edited without affecting the stabilization.
Mistika VR supports import of the Insta360 Pro 2 or Titan gyro metadata which was captured during each individual shot. The camera’s recorded metadata is stored in the pro.prj file for each shot and is utilized when selecting Import Stabilization Metadata under the Stabilize pop-up menu. In the case of this helicopter shot, the recorded metadata is very useful for stabilizing the shot.
For more information read the Stabilizing 360 video in Mistika VR article.
Mistika VR also has a manual method for stabilization following the heading or direction of footage. This technique is processor intensive and can be time consuming, but is useful for cameras that do not capture gyro metadata supported by Mistika VR.
Stabilization options are found in the Stabilize pop-up menu
Once stabilization has been imported via metadata, or calculated by Mistika VR, the S-Yaw, S-Pitch and S-Roll values will be displayed in the Output Camera. These stabilization S values for the output camera are independent of the standard Yaw, Pitch and Roll values. This enables reorienting the video or leveling the horizon without impacting the stabilization.
Reorienting is discussed in this Reorienting a 360 video in Mistika VR article.
To learn more about setting the horizon line read this Leveling a 360 video in Mistika VR article
S-Yaw, S-Pitch and S-Roll values are produced when stabilization is calculated or stabilization metadata has been imported. The Yaw Pitch and Roll parameters can be modified independently for leveling and reorientation.
Once the shot is fully stitched it can be rendered as a equirectangular 360 video with controls set-up in the Export Options panel. Depending on how the footage is to be used next in the production pipeline, it can be rendered out with a large range of formats and quality options.
The Export Options render panel