Conference Proceedings
- Home
- Reconstructing 3D from Compressed Video: An AV1-Based SfM Pipeline
Reconstructing 3D from Compressed Video: An AV1-Based SfM Pipeline
Description
We present an AV1-based Structure from Motion Pipeline repurposing AV1 bitstreams metadata to generate correspondences for 3D reconstruction.
The process of creating 3D models from 2D images, known as Structure from Motion (SfM), traditionally relies on finding and matching distinctive features (keypoints) across multiple images. This feature extraction and matching process is computationally intensive, requiring significant processing power and time.
We can actually bypass this expensive step using the powerful motion vectors of AV1 encoded videos. We present an analysis of the quality of motion vectors extracted from AV1 encoded video streams.
We finally present our full AV1-based Structure from Motion pipeline, re-using motion vectors from AV1 bitstreams to generate correspondences for 3D reconstruction, yielding dense inter-frame matches at a near-zero additional cost. Our method reduces front-end time by a median of 42% and yields up to 8× more 3D points with a median reprojection error of 0.5 px making compressed-domain correspondences a practical route to faster SfM in resource-constrained or real-time settings.
This talk was presented at Demuxed 2025 in London, a conference by and for engineers working in video. Every year we host a conference with lots of great new talks like this – learn more at https://demuxed.com
Conference
Speakers
Learning Categories
Other Proceedings
Here are some other proceedings that you might find interesting.
What Codec Should I Use?
Alan Resnick
Doing Server-Side Ad Insertion on Live Sports for 25.3M Concurrent Users
Ashutosh Agrawal
Is now the time to solve the deepfake threat?
Roderick Hodgson
Super Resolution: The scaler of tomorrow, here today!
Nick Chadwick
The do's and don'ts about Streaming security
Javier Brines Garcia
Modeling the conceptual structure of FFmpeg in JavaScript
Ryan Harvey
Objectionable Uses of Objective Quality Metrics
Richard Fliam
RTMP: web video innovation or Web 1.0 hack… how did we get to now?
Sarah Allen
Large-Scale Media Archive Migration to the Cloud
Konstantin Wilms
HEVC Upload Experiments
Chris Ellsworth
Related Courses
Below are some courses that might interest you based on the learning categories and topic tags of this conference proceeding.
What Codec Should I Use?
Alan Resnick
Doing Server-Side Ad Insertion on Live Sports for 25.3M Concurrent Users
Ashutosh Agrawal
Is now the time to solve the deepfake threat?
Roderick Hodgson
Super Resolution: The scaler of tomorrow, here today!
Nick Chadwick
The do's and don'ts about Streaming security
Javier Brines Garcia
Modeling the conceptual structure of FFmpeg in JavaScript
Ryan Harvey
Objectionable Uses of Objective Quality Metrics
Richard Fliam
RTMP: web video innovation or Web 1.0 hack… how did we get to now?