A growing view among observability engineers holds that the reasoning ability of large language models is no longer the bottleneck in AI-assisted root cause analysis. The harder problem now sits in the pipeline that decides what data reaches the model[reference:45]. For teams adding LLMs to incident response, the practical takeaway is that effort spent preparing context may pay off more than reaching for a larger model.
Two Approaches to AI RCA
Most AI RCA efforts fall into two camps. Agent-based designs hand the model tools and let it investigate, choosing what telemetry to fetch as it reasons. Deterministic designs correlate signals up front and hand the model a single prepared context.
The two approaches break in different ways, which makes it hard to tell whether a failed diagnosis came from weak reasoning or from a harness that fed the model the wrong evidence.
Coroot's Research
Recent research from observability vendor Coroot tries to separate those two variables[reference:46]. Engineer Nikolay Sivko split RCA with an LLM into two jobs: reasoning over the data in front of it, and the harness that decides what data reaches the model and in what shape[reference:47]. He argued that "can AI do RCA?" is the wrong question, because the two jobs need to be measured apart[reference:48].
Coroot's pipeline correlates signals into findings and passes them to the model in one focused context with no agent loop[reference:49]. This lets a wrong answer be blamed on the model rather than on missing evidence[reference:50].
Testing Eleven Models
To test it, Sivko built one scenario: a NetworkChaos experiment injecting delay between a catalogue service and its Postgres database, slowing queries and pushing the front-end into 502 errors. The context deliberately included misleading signals, including query timings inflated by network round-trip time.
He then ran the same prompt, around 9,800 tokens, against eleven models, asking each for the root cause, the cause-and-effect chain, and the immediate fix.
Closed frontier models Claude Opus 4.8, GPT-5.5 and Gemini 3.1 Pro all passed, naming the experiment and flagging the need to delete both it and its schedule. Larger open-weight models mostly kept pace. Lower down, Gemma 4 31B was the only self-hostable model to identify the root cause, while the larger Qwen3.6 35B and Qwen3 Coder Next both missed it.
The Debate Continues
The findings do not settle the debate. Agent-based approaches keep a genuine advantage, since a model that fetches its own data can pull signals a fixed pipeline never anticipated. This matters for novel incidents outside a predefined correlation set.
They pay for that flexibility in operability, though. Accounts from production describe multi-agent LLM investigations as notoriously hard to debug. A failed run leaves no clean stack trace, only unpredictable prompt interactions and emergent coordination between agents.
That fragility pushes many practitioners the other way. Engineers report scrapping fully agentic designs in favour of mostly deterministic workflows with a narrow LLM step, citing better reliability and lower token cost. Deterministic pipelines trade some flexibility for repeatability and cleaner evaluation.
The Cost Reality
On cost, Sivko noted that a single short call runs a few cents even on frontier models, because the correlation work happens before the model is called. He judged that the reasoning part of AI RCA "is basically solved" and that the real work is now "preparing the right, compact context for the model before you call it."
That conclusion points the next round of engineering effort at the harness rather than the model. The framing echoes a wider push around context engineering. Guidance from multiple sources converges on the same idea: curating the smallest set of high-signal, compact context is now a core discipline for making LLM-based reasoning and observability reliable.