Content providers, who frequently rely on third party software and services (Players, CDNs, Origin Services), struggle to develop the observability necessary to achieve their QoE goals. They are essentially responsible for the entire customer experience but only able to fully observe what they themselves instrument and what their service providers are able/willing to share. In recent years, there has been some progress along these lines in the form of CMCD-based data propagation of player/app metrics and CDN log streaming but logs and metrics are still insufficient for enabling the deep observability necessary for consistently amazing user experiences.

In addition, 3rd party service providers (e.g. CDNs, origin services, packages, etc) struggle to provide optimal service to their customers (e.g. content publishers) due to the same observability challenge. In short, telemetry is fragmented and siloed making it virtually impossible to get the complete architectural or operational picture. As a result of these conditions the Steaming Video Alliance’s QoE working group is developing methods for augmenting logs and metrics with the third pillar of observability, distributed request tracing. Typically distributed request tracing is performed using standardized methods based on the open telemetry project in the context of a single service architecture. Our project is designed to span 3rd party services across the video ecosystem from player to CDN to Origin and beyond using logging mechanisms already in place. In this talk we’ll cover the methods for implementing our first phase of this initiative from player to Origin and back. We’ll talk about early results and how tracing can be leveraged to turn high level QoE warning signals into deep root cause analysis. If successful this effort promises to enable game-changing quality for not only content publishers but the 3rd party services that support them. Presented at Demuxed 2021.