In this talk, we will focus on the Pseudo HDR technologies and their effective and efficient implementations, targeted to provide HDR-like experiences for SDR videos streamed to ordinary rendering devices that may not be HDR capable. In particular, we are focused on the enhancement of the Quality of Experiences (QoE) for those UGC (User Generated Content) videos, which have observed an explosive growth thanks to the evolution of affordable consumer devices and tremendous popularity of social media platforms. The qualities of UGC videos actually vary over a fairly wide range, quite unpredictable. One category of methodologies to enhance the QoE is to adopt a pre-processing stage (or a pre-encoder) before the videos are fed into the encoder/transcoder to further distribute over the Internet. Our Pseudo HDR approach falls into such a pre-processing category. It maintains the same bit depth, e.g. 8-bit, for the input videos, aiming to enhance such content that contains significant super darker or lighter regions, to generate more visually impactful content and present PGC-like experiences on the end user side without the need of the availability of special rendering platforms, such as HDR-capable devices.

Our essential algorithm behind our Pseudo HDR approach is based on the so-called Contrast Limited Adaptive Histogram Equalization (CL-AHE: https://en.wikipedia.org/wiki/Adaptiv…, where different histogram equalization transforming functions are derived for different regions in one video frame, targeted to especially enhance the contrast in those regions that are significantly lighter or darker. With contrast limitations, it will prevent too much noise from being amplified that may deteriorate the overall visual qualities. We are adopting CL-AHE as part of the pre-processing procedure on videos before their transcoding. Advantages of applying such a procedure to the video source are two fold: Unanimous quality enhancement without relying on the ultimate rendering devices, and best leveraging the relatively abundant computational resources available at the transcoder side. UGC many times are first being uploaded to the cloud after their creation and before being widely distributed, and ultimately consumed over mobile devices, where the consumed power is limited. The power consumption performance for any mobile apps have significant impact on device heating status and battery life, eventually influencing the overall QoE to the ender users. Existing CL-AHE approaches have been mainly studied and applied to still images. Even when considered for video applications, they are usually exploited for the processing of premium content instead of UGC. Overall, we are targeted to develop a Pseudo HDR approach that applies to UGC videos with effective and efficient implementations so that they are deployable to enhance the QoE for both VOD and live end-to-end streaming solutions. The underlying challenges for Pseudo HDR pre-processing on videos are indeed significant: (a) Noises presented in those super bright or dark areas, in particular those specific to UGC such as compression artifact noises, will be easily amplified by the AHE transformation and are visually fairly annoying. (b) It’s always critical to preserve the temporal consistency for videos as one video is not simply a series of still images. If the pixel transformation for contrast enhancement does not take into account the temporal domain characteristics, artifacts such as flickering across frames especially in those dark areas would be very visible, deteriorating the overall visual experience. The smoothness and fluidity in the temporal domain for videos are even more critical than the per-frame visual quality represented in the spatial domain. (c) The resulting transformed videos could be much larger in size, as edges and fine textures in the original dark areas, for instance, can be compressed out in the original SDR content without incurring much visual degradation. In contrast, after Pseudo HDR processing, more finer textures are exposed and hence demands more bits allocated to achieve sufficiently good visual quality. Pseudo HDR is an innovative direction that we’d like to drive to enable video end users to enjoy the best possible visual quality. To address the above challenges and other major issues during our product launching with our customers, we have implemented and deployed following methodologies: Presented at Demuxed 2021.