mees/calvin

The benchmark includes diverse sensor modalities (RGB-D from static and gripper cameras, tactile sensing, proprioception) and flexible action spaces (absolute/relative Cartesian, joint-based), enabling researchers to explore different visuomotor control configurations. Training leverages PyTorch Lightning with Hydra configuration management, supporting multi-GPU distributed training and modular sensor/encoder combinations. The evaluation protocol emphasizes long-horizon sequential task execution—chaining multiple language instructions in continuous control episodes—with standardized difficulty levels across different sensor and environment subsets.

854 stars. No commits in the last 6 months.