In early 2023, GPT-4-class capabilities cost roughly 1 — and some providers are pushing below $0.10[reference:36]. Across benchmarks, inference prices have fallen between 9x and 900x per year, with a median decline near 50x[reference:37]. Even frontier models are getting dramatically cheaper each generation[reference:38].
We're entering the era of virtually free intelligence[reference:39]. Not "Nobel-Prize-winning genius-level" intelligence, necessarily, but the kind that suffices for the vast majority of knowledge work[reference:40]. And that changes everything — especially for data systems.
A group of researchers from UC Berkeley's EPIC Data Lab, led by Aditya Parameswaran, recently laid out what this means[reference:41]. Their argument: three new challenges — and opportunities — stem from near-zero inference costs[reference:42].
Data Systems For Agents
Agents querying databases don't behave like humans or BI tools[reference:43]. They perform what the researchers call "agentic speculation" — a high-volume, heterogeneous stream of work spanning schema introspection, columnar exploration, and query formulation[reference:44]. A single user request — "why did coffee sales in Berkeley drop this year?" — could amount to thousands of individual SQL queries as agents explore the hypothesis space[reference:45].
The redundancy is actually helpful — more attempts mean higher success rates. But from the data system's perspective, it's mostly wasted work[reference:46]. Only 10-20% of sub-plans are distinct; 80-90% perform duplicate work.
An agent-first data system could change that. It could reuse results across overlapping sub-plans, draw on decades-old work on materialized views and common subexpression elimination. Or it could return approximate answers — good enough for agents to make progress — and stream intermediate results to help agents decide if seeing the rest is necessary[reference:47].
Data Systems Of Agents
Beyond query execution, agents need a place to live — to manage state over long-running tasks, coordinate with each other, reach consensus, and deal with failures[reference:48].
Memory is a particular challenge. The current wisdom is that agents should write to unstructured markdown files and search them with grep or embedding-based retrieval[reference:49]. At scale, this breaks down. Limited context windows can't hold everything, and stuffing all relevant fragments into context is inefficient[reference:50].
What's needed is structured memory — retrieval across multiple attributes or facets[reference:51]. An agent debugging a flaky test should pull only memories tagged with the relevant module, language, framework, and failure mode. Raw agent traces with mistakes aren't useful; we want corrective memory[reference:52].
The researchers propose organizing memory across dimensions — columns and tables, type of operation, and natural-language corrective instructions. One open question is defining a "memory schema" — akin to defining a schema for each application[reference:53].
Data Systems By Agents
If intelligence is free, we can employ it to synthesize new data systems from scratch[reference:54]. Recent work shows that agentic pipelines can synthesize complete, workload-specific analytical engines in minutes to a few hours at a cost of a few dollars. When the workload shifts, you simply regenerate them[reference:55].
The challenge is verification[reference:56]. Specifications are typically imperfect and don't cover corner cases. Present-day agents exploit missing specifications to reward-hack their way to high performance metrics[reference:57]. One solution: auxiliary verification agents that generate test cases to catch corner-case exploitation, effectively expanding the specification[reference:58].
Another approach: generate both a system and a proof of its correctness together[reference:59]. More work is needed to solidify this approach.
The Blurring Boundary
Looking further out, the boundaries between agents and data systems will likely start to blur[reference:60]. Agents may design the data systems they themselves run on. Both interfaces and internals could evolve over time in a form of recursive self-improvement[reference:61]. Data systems may become a holistic source of truth for all relevant state — raw data, memory, and coordination state — erasing distinctions between what's being queried and what's generated by agentic activity[reference:62].
Data systems may incorporate agentic components, evolving from passive computation engines into intelligent, proactive, self-optimizing architectures[reference:63].
It's hard to predict what the future holds. But one thing is clear: as intelligence becomes free, data systems matter more than ever. We're in for a wild ride.