Perceptually Aware Live VBR Encoding Scheme for Adaptive AVC Streaming

Currently, a fixed set of bitrate-resolution pairs termed a “bitrate ladder” is used in live streaming applications. Similarly, two-pass variable bitrate (VBR) encoding schemes are not used in live streaming applications to avoid the additional latency added by the first-pass. Bitrate ladder optimization is necessary to (i) decrease storage or delivery costs or/and (ii) increase Quality of Experience. Using two-pass VBR encoding improves compression efficiency, owing to better encoding decisions in the second-pass encoding using the first-pass analysis. In this light, this paper introduces a perceptually-aware constrained Variable Bitrate (cVBR) encoding Scheme (Live VBR) for HTTP adaptive streaming applications, which includes a joint optimization of the perceptual redundancy between the representations of the bitrate ladder, maximizing the perceptual quality (in terms of VMAF) and optimized constant rate factor (CRF). Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for every video segment, namely, brightness, spatial texture information, and temporal activity, are extracted to predict a perceptually-aware bitrate ladder for encoding. Experimental results show that, on average, Live VBR yields bitrate savings of 7.21% and 13.03% to maintain the same peak PSNR and VMAF, respectively, compared to the reference HTTP Live Streaming (HLS) bitrate ladder Constant Bitrate (CBR) encoding using x264 AVC encoder without any noticeable additional latency in streaming. Additionally, Live VBR results in a 52.59% cumulative decrease in storage space for various representations, and a 28.78% cumulative decrease in energy consumption, considering a perceptual difference of 6 VMAF points.

Vignesh V. Menon | Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität | Klagenfurt, Austria
Prajit T. Rajendran | CEA, List, F-91120 Palaiseau, Université Paris-Saclay | France
Christian Feldmann | Bitmovin | Klagenfurt, Austria
Martin Smole | Bitmovin | Klagenfurt, Austria
Mohammad Ghanbari | Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität and School of Computer Science and Electronic Engineering | Klagenfurt, Austria and University of Essex, United Kingdom
Christian Timmerer | Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität | Klagenfurt, Austria