LLM-driven multi-agent systems are increasingly used to solve complex problems through collaboration. But when these systems fail, developers face a daunting task: sifting through lengthy interaction logs to find which agent made the error and at what point. This debugging process is manual, time-consuming, and requires deep system knowledge.
Researchers from Penn State University and Duke University, in collaboration with Google DeepMind, the University of Washington, Meta, Nanyang Technological University, and Oregon State University, have introduced a new research problem called automated failure attribution. Their work, accepted as a conference paper, aims to systematically identify the responsible agent and the specific step where the decisive error occurred.
The team constructed the Who&When dataset, the first benchmark for this task. It includes failure logs from 127 LLM multi-agent systems, generated algorithmically or hand-crafted by experts, with fine-grained human annotations for the responsible agent, error step, and a natural language explanation.[reference:0]
To tackle attribution, the researchers designed and evaluated three methods:
- All-at-Once: The LLM receives the full query and log in one pass. Cost-effective but struggles with long contexts.
- Step-by-Step: The LLM reviews the log sequentially, mimicking manual debugging. More precise but expensive and prone to error accumulation.
- Binary Search: A middle-ground approach that recursively halves the log to isolate the error segment.
Experiments using GPT-4o and other models revealed that even the best single method achieved only about 30% accuracy in identifying the responsible agent and a mere 20% in pinpointing the exact error step. Some methods performed worse than random guessing. Interestingly, All-at-Once was better at identifying "who," while Step-by-Step excelled at determining "when." Combining methods improved performance but increased computational costs.
Surprisingly, even advanced reasoning models like OpenAI o1 and DeepSeek R1 found the task challenging, highlighting the difficulty of automated failure attribution. Providing explicit prompts that require the LLM to explain its reasoning improved performance. The study also found that as context length increases, accuracy decreases, especially for identifying the error step.
The researchers emphasize that automated failure attribution is crucial for the development lifecycle of multi-agent systems. It can transform the challenge of identifying what went wrong from a mystery into a quantifiable problem, bridging the gap between evaluation and improvement. The code and dataset are fully open-source, paving the way for more reliable, intelligent, and trustworthy multi-agent systems.