ag2ai/Agents_Failure_Attribution

Introduces the "Who&When" benchmark with 184 annotated failure trajectories from both algorithm-generated (CaptainAgent) and hand-crafted (Magnetic-One) multi-agent systems, providing fine-grained labels for responsible agents, critical error steps, and failure explanations. Implements three attribution methods—All-at-Once, Step-by-Step, and Binary Search—that work with multiple LLM backends (GPT-4o, Llama, Qwen) to automatically pinpoint failure causes in complex agentic workflows. Evaluates performance on realistic scenarios derived from GAIA and AssistantBench datasets, enabling rapid debugging iteration and reward signals for agent self-correction.

349 stars.