OSU-NLP-Group/TravelPlanner

Comprises a multi-constraint planning benchmark with three evaluation modes: a two-stage tool-use setup where agents search for information before planning, a sole-planning mode providing pre-gathered data, and strategy variants (direct, chain-of-thought, ReAct, reflexion). The environment includes a structured database (JSON format), GPT-4-based postprocessing for natural language-to-JSON conversion, and comprehensive evaluation metrics that assess adherence to environment, commonsense, and hard constraints. Supports multiple LLM backends (GPT-3.5/4, Gemini, Mistral, Mixtral) with fine-tuned model variants available via HuggingFace and LLaMA-Factory integration.

491 stars.