Back
x min
14.08.2026

Turning dashcam footage into virtual 3D scenarios

ADAS-equipped vehicle fleets process vast amounts of driving data daily. This includes safety-critical situations as well as rare, long-tail scenarios that challenge system assumptions and expose performance gaps. While vehicles use triggers like hard braking, emergency steering, or driver honking to catch these events, most of the footage remains unused for systematic ADAS improvement to date.

To improve on this, a continuous data loop is required, where fleet data is collected, fed into a scenario reconstruction pipeline, and finally used to train and test current- and next-generation ADAS algorithms. However, two typical challenges stand in the way. First, raw fleet recordings often come with technical constraints, such as low video resolution, no LiDAR available, and limited onboard perception. Second, most reconstruction pipelines need multi-modal, high-quality sensor data and manual annotation, which simply does not scale to millions of events.

As a result, valuable real-world data sits idle while teams fall back on synthetic or curated scenarios that fail to capture the full complexity of human driving. To close this gap, DeepScenario has built an autolabeling pipeline that enables automated 3D scenario reconstruction directly from low-resolution dashcam footage.

From raw footage to accurate 3D reconstruction

DeepScenario's autolabeling pipeline reconstructs scenarios in 3D directly from low-resolution dashcam footage, entirely without LiDAR. The solution combines visual localization with 3D object detection and tracking. First, the camera pose is recovered using Visual Inertial Odometry (VIO), fusing tracked visual features with inertial measurements to deliver precise localization. In a second step, the pipeline extracts 3D object tracks with centimeter accuracy for all dynamic objects in the scene. DeepScenario’s 3D detection and tracking algorithms achieve leading accuracy on state-of-the-art perception datasets, including KITTI and Waymo – see this recent ECCV 2026 publication, LeAD-M3D, for details.

One particularly noteworthy fact is that the pipeline works from standard ADAS fleet data, making it viable for large-scale, automated processing of fleet events. At the same time, it delivers robust performance even in challenging lighting conditions like nighttime driving.

Powering ADAS training and testing at scale

The reconstructed scenarios from DeepScenario’s pipeline power two main use cases: training and testing.

For training, the scenarios can be used to develop both modular and end-to-end ADAS functions, helping future systems behave in a more human and safe way, even in critical situations. In modular stacks, the extracted object tracks can, for example, enhance the prediction capabilities of the ADAS function. In end-to-end stacks, the reconstructed ego trajectory can be used to train a driving policy, while the reconstructed object tracks enable interpretable model outputs.

For testing, the scenarios can be exported into the OpenSCENARIO format and simulated in closed loop. This makes it possible to benchmark ADAS performance against real-world edge cases that go beyond the complexity of standard test benchmarks like EuroNCAP. It reveals how the current function behaves, where false positives happen, and where systematic mistakes are made. Over time, teams can also build a scenario catalogue that can be used as tests in a CI/CD pipeline.

Together, this data-driven approach creates a scalable feedback loop and enables a deep understanding of the real-world situations that vehicles actually encounter. Thus, the reconstructed fleet data serves as a systematic, repeatable driver of next-generation ADAS development.

We use cookies to enhance your browsing experience, evaluate website usage, and serve marketing content. Visit our Privacy Policy for more information.