MV-HEVC: How to optimize compression of immersive 3D content

Multiview High Efficiency Video Coding (MV-HEVC) is an HEVC extension focused on efficiently coding spatially related images, such as left eye and right eye views of 3D stereoscopic content. MV-HEVC was released in the second version of HEVC back in October 2014 ​[1]​. MV-HEVC was created alongside another extension named SHVC, which is already employed in ATSC 3.0 use cases since 2018 ​[2]​. The two extensions share many fundamentals, codified in an annex for common specifications for multi-layer extensions. Their base principle is scalability: A traditional HEVC decoder can rely on the base layer only, hence providing backward compatibility, but a more elaborate decoder can use the second layer as well to improve the rendition.

The MV-HEVC format gained widespread attention with Apple announcing support for 3D movies on the Apple Vision Pro at WWDC23 ​[3]​​ [4]​. Apple provided public guidelines on stereo video encoding ​[5]​, packaging ​[6]​, and streaming ​[7]​ as well as on providing parallax information to improve caption placement​ [8]​.

This presentation will describe the MV-HEVC standard and how it works in theory. An implementation of MV-HEVC [9] will also be studied to assess performance in practice in industrial settings. Deployment aspects such as packaging and captioning will be discussed. Finally, 3D-specific optimization strategies will be analyzed, including software architecture and rate control.

Thomas Guionnet| ATEME | Rennes, France
Khaled Jerbi | ATEME | Rennes, France
Thomas Burnichon| ATEME | Rennes, France
Mickaël Raulet | ATEME | Rennes, France