taco-group/OpenEMMA

Integrates pretrained Vision Language Models (GPT-4o, LLaVA, Llama, Qwen) with front-view camera inputs and operates on the nuScenes dataset to predict ego vehicle waypoints and generate natural language decision rationales. The architecture leverages VLM world knowledge for end-to-end motion planning, incorporating 3D object detection via YOLO-3D to identify critical scene elements. Outputs include predicted trajectories, textual reasoning explanations, and annotated video visualizations of planned behavior.

906 stars and 112 monthly downloads. No commits in the last 6 months. Available on PyPI.