Augment Code has published benchmark results showing its AI coding assistant achieves 33% better token efficiency than Claude Code when running the same model on identical tasks, reopening a debate about whether AI coding tools should pre-build context around a codebase or discover it on the fly.
The claim comes from Vinay Perneti, Augment's VP of Engineering, who spoke about the company's fundamentally different approach to what the industry calls a "harness" - the software layer that decides what an AI model sees and what actions it can take inside a developer's project. While Anthropic's Claude Code team favors a lean, minimal harness that lets the model do the heavy lifting, Augment has spent over 18 months building a context engine that pre-indexes repositories using embeddings, retrieval models, and vector databases.
Why the Two Companies See Different Results
The disagreement is not just philosophical. It may stem from the fact that the two teams are measuring different things.
Cat Wu, head of product for Claude Code, told reporters earlier this summer that Anthropic does not see "a measurable improvement in performance" from adding semantic code-navigation tools. Wu was specifically referring to language server protocols (LSPs), a narrower technology than Augment's full context engine. Perneti pointed out that "not all retrieval systems are equal, just like not all databases are equal." Augment's system, he said, was researched and built from the ground up starting in 2022, before ChatGPT existed, specifically for large private codebases.
The core difference: Claude Code relies on grep-style search, letting the model query the codebase as needed during a task. Augment Code pre-computes a semantic map of the entire repository, enabling sub-millisecond retrieval of conceptually related code. Perneti argues that on public open-source repositories, frontier models have largely memorized the code, so grep works fine. In private repositories the model has never seen, semantic retrieval dramatically shortens the exploration loop.
The Private Repo Problem
Perneti's argument hinges on a distinction that matters for enterprise customers. Public benchmarks are run on open-source repositories that large language models have likely encountered during training. A model asked to modify a function in Linux kernel code or a popular Python library often "already knows where to look."
Private codebases are a different story. When a model has never seen a company's internal monorepo, it must spend tokens searching, reading files, and building mental context before it can make a single edit. Augment's bet is that pre-indexing eliminates that waste. In their Terminal-Bench test using the same underlying model as Claude Code, Augment matched accuracy while using one-third fewer tokens.
For engineering leaders watching their monthly AI API bills, that efficiency claim is not abstract. It is a direct cost reduction.
Cost, Trust, and the Human in the Loop
Perneti addressed two common objections to AI coding tools: runaway technical debt and unsustainable costs.
On trust, he was blunt. Agents are not good at writing specifications. They excel at execution. The model, in his view, works best as part of a team of humans and agents, with engineers retaining judgment calls at spec review and architecture decisions. Augment itself runs "focused sprints on reducing tech debt with agents" because, as Perneti noted, "agents are very good at duplicating code."
On cost, he predicted a tipping point. As open-source models improve, teams will route well-specified, routine coding tasks to cheaper or locally hosted models while reserving frontier models like Anthropic's Opus for the hardest problems. A harness that can intelligently route tasks and minimize token burn on context gathering becomes a strategic cost control layer.
What Both Sides Agree On
Despite their different harness philosophies, Wu and Perneti converge on one point: frontier models are improving exponentially, and that trajectory is reshaping software development for good. Both expect this pace to continue for at least the next year.
Where they diverge is whether to build infrastructure around today's limitations or stay lean and let the models catch up. Anthropic believes models will soon be smart enough that elaborate context engines become unnecessary overhead. Augment believes intelligence and context are separate ingredients, and that even a genius model wastes money if it must rediscover what a vector database already knows.
The next 12 months will test which assumption ages better. Watch for enterprise benchmarks on large private repositories, not just public open-source leaderboards, to settle the argument.