MILER: Semantic Mid-Level Representation for Sim-to-Real Reinforcement Learning in Unstructured Autonomous Driving
MILER reports zero-shot sim-to-real autonomous driving on an unstructured test track without human intervention.
The system trains a reinforcement-learning policy offline in a semantic mid-level simulator, then matches real camera and LiDAR perception to that representation through BEVFusion.
On the vehicle, the learned actions are routed through a trajectory-alignment strategy rather than applied directly.
The authors say the framework drove 17.3 km across two vehicles on a 3.0 km track with obstacles, hairpin curves, off-road sections, and speeds up to 33.6 km/h.
They also report that the full software stack runs on a Jetson AGX Orin.
ArXiv · AI/CL/LG's note
The system trains a reinforcement-learning policy offline in a semantic mid-level simulator, then matches real camera and LiDAR perception to that representation through BEVFusion.
On the vehicle, the learned actions are routed through a trajectory-alignment strategy rather than applied directly.
The authors say the framework drove 17.3 km across two vehicles on a 3.0 km track with obstacles, hairpin curves, off-road sections, and speeds up to 33.6 km/h.
They also report that the full software stack runs on a Jetson AGX Orin.
ArXiv · AI/CL/LG's note
score 4