Many setups in large scale on-demand video streaming nowadays rely on a just-in-time packager to deliver remotely stored MP4 content. Unfortunately, the latency to such a storage backend can be suboptimal and a just-in-time packager needs to make a relatively high number of requests to the source content to package it dynamically. This will impact the overall performance of the packager, as well as the start-up delay that customers will experience when a stream that hasn’t been cached on the CDN is requested (i.e., longtail content).

In my talk, I will propose how the number of requests to the remote storage backend can be minimized by using a novel but spec compliant approach to packaging MP4. In essence, it’s about generating an additional MP4 that acts as an intermediary between the packager and the source content. This MP4 does not contain media data, but merely references the source content using MP4’s ‘dref’ box. What this ‘dref MP4’ does contain, is the index information (sample tables) of the original track, stored in a ‘moov’ box. This is the small but vital bit of information that a just-in-time packager needs to handle each and every incoming request. By omitting the original media data, a dref MP4 becomes very small and can be cached easily in a reverse proxy cache that sits between the just-in-time packager and the remote storage backend. Or it can be stored locally on the packager itself. Either way, this approach will ensure that the just-in-time packager will have much faster access to information that it needs to access very often. The impact of this approach is considerable. In my tests with Unified Origin, Unified Streaming’s just-in-time packager, an MPD (DASH client manifest) was generated 70x times faster (as it contains timelines for all track in a stream, a packager a will need to read the index information of each of those tracks). The gains when packaging a media segment were much smaller, but still quite significant with an improvement of 1.5x. Also, the general throughput of the packager improved by ~10-20%. In my talk I will explain the above by going into the nitty gritty details of the relatively high number of requests that Unified Origin make to a storage backend, explaining what each of these requests are for to give the audience a more thorough understanding of the inner workings of a just-in-time packager in general, and Unified Origin in specific, while also introducing them to the concept of a dref MP4, which – based on my experience thus far – will be a format that most have never heard of. With this unique blend of technical insight, practical gains and an introduction to a novel type of MP4, I think this talk will serve the audience at Demuxed particularly well. I hope you do, too. Presented at Demuxed 2020.