BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model
BirdDreams predicts surrounding traffic from a single camera image without requiring other vehicles’ future trajectories, an HD map or vehicle-state inputs.
The model generates camera-view video alongside a schematic bird’s-eye view, letting each stream inform the other. The top-down view tracks roads and vehicles at a fixed metric scale, while the camera view retains details such as traffic signals. Shifting and rotating predicted bird’s-eye frames along an input driving trajectory provides guidance for ego motion. On PhysicalAI-AV and nuScenes, the authors report better trajectory following than prior driving world models; human evaluators also judged surrounding-vehicle behavior more realistic than a perspective-only baseline.
HF Daily Papers' note
The model generates camera-view video alongside a schematic bird’s-eye view, letting each stream inform the other. The top-down view tracks roads and vehicles at a fixed metric scale, while the camera view retains details such as traffic signals. Shifting and rotating predicted bird’s-eye frames along an input driving trajectory provides guidance for ego motion. On PhysicalAI-AV and nuScenes, the authors report better trajectory following than prior driving world models; human evaluators also judged surrounding-vehicle behavior more realistic than a perspective-only baseline.
HF Daily Papers' note
score 5