What is a video decoder? In essence it’s a highly specialized painting engine that operates using special instructions which are packed into something we call ‘bitstream’. These instructions include things like “move block from A to B” or “paint directionally into that direction”. A normal encoder tries to use these instructions as efficiently as possible to “paint” a video that is as close to a reference video as possible. Everybody can do that. What if we use the tools that the decoder offers to modify, morph or filter images or even to paint directly with the tools? So with a modified custom encoder we can directly write a standard conform bitstream that produces really fascinating output. In this talk I want to reveal the art in the video codec but also teach a bit about the inner workings of video decoders that produce this art.

This talk was presented at Demuxed ’22, a conference for video nerds in San Francisco featuring amazing talks like this one. Demuxed ’22 was made possible by sponsors like our Platinum sponsor Daily (https://daily.co) and organized by people from Mux (https://mux.com). For more information about the conference and community, see https://2022.demuxed.com.