Most enterprises are pouring money into AI infrastructure without knowing what they are really paying for. A fresh survey of 107 mid-market companies by VentureBeat Pulse Research found that 83% of organizations running their own GPUs operate them at 50% utilization or less, and fewer than half can rigorously track the cost and return of their AI compute. The gap between spending and visibility is widening fast.
The study, fielded in June 2026, paints a picture of an industry in motion. Three-quarters of respondents are still experimenting or running only partial production workloads, yet their investment intentions point well beyond their current stacks. The tension is clear: organizations want more compute, but they cannot yet account for what they already own.
The Current Stack: Hyperscalers Rule, Neoclouds Barely Register
Enterprises today run AI almost entirely on platforms they already know. Google Cloud leads at 48%, followed by Microsoft Azure at 29%, AWS at 22%, and Oracle Cloud at 22%. Major model APIs from OpenAI, Anthropic, and Gemini round out the picture. The specialized "neocloud" providers that dominate AI infrastructure headlines, CoreWeave, Lambda, Crusoe, and Nebius among them, register at or near zero in current usage. Only 6% of respondents operate on-prem GPU clusters, and a mere 4% run custom open-source stacks.
This concentration is partly a function of the sample, which skews toward the mid-market. Organizations with 101 to 250 employees make up 36% of respondents, and those with 251 to 1,000 employees account for another 27%. For these buyers, sticking with familiar hyperscalers and model APIs is the path of least resistance. But that loyalty may not last.
Where the Next Dollar Is Going
The sharpest finding in the report is the disconnect between today’s stack and tomorrow’s plans. 45% of enterprises plan to evaluate AI-specialized clouds over the next 12 months, the single most-cited evaluation area and a category almost none of them use today. Nearly a third (32%) intend to assess non-Nvidia accelerators such as AWS Trainium, Google TPU, and AMD Instinct. Even decentralized compute networks (16%) and sovereign compute (11%) are drawing meaningful interest.
This is not incremental expansion. It reads like the early stages of a re-platforming. When asked about net momentum, specialized AI clouds scored +24, edging out hyperscalers at +22. The direction of travel is unmistakable: a meaningful share of enterprise AI compute is preparing to move off general-purpose clouds.
The trend is not new. VentureBeat’s April-May wave found the same pattern. CoreWeave, Lambda, and Crusoe each sat at 2-4% usage back then, yet 33% of enterprises cited moving workloads to specialized AI clouds as their top planned strategy change. Two survey waves, two differently worded questions, one consistent signal.
Provider Churn Is Accelerating
For a category as foundational as compute, the intended movement is striking. 64% of enterprises plan to switch or add an infrastructure provider within the next 12 months, and 38% intend to do so within the next quarter alone. Only 36% have no plans to change.
The near-term switching interest, however, is concentrated among incumbents. Microsoft Azure and Google Cloud each draw 33% switching consideration, OpenAI 30%, and Gemini 22%. This suggests the immediate churn is less about defecting to upstarts and more about reshuffling spend among the majors. The neocloud evaluation is a 12-month thesis; the quarter-three switching is mostly the big players trading share.
What Actually Drives Buying Decisions
Here is where the story gets uncomfortable for vendors. Integration with the existing stack tops the list at 41%, followed by total cost of ownership at 35%. The metric vendors compete on loudest, cost per million tokens, is dead last at 8%. Buyers are optimizing for fit and true operating cost, not headline price.
The problem is that most cannot measure that true cost. Enterprises say TCO matters, yet the majority lack the instrumentation to calculate it. This mismatch between stated priority and actual capability sits at the heart of the compute gap.
GPUs Sitting Idle, Costs Going Untracked
The utilization numbers are sobering. Nearly half (49%) of GPU-operating enterprises run at 25% utilization or below. Only 12% clear the 50% mark. A further 8% do not measure utilization at all. Idle accelerators are expensive accelerators, and the current fleet has vast efficiency headroom that is largely unmeasured.
The tracking picture is no better. Fewer than half (44%) rigorously track AI compute costs and returns. The rest track partially (39%), cannot quantify yet (20%), or have not prioritized it (6%). Satisfaction with current infrastructure averages 4.0 on a five-point scale, but value for money trails at 3.9 and ease of implementation at 3.8. The softness lands exactly where you would expect: on cost, the dimension hardest to judge without measurement.
The Memory Frontier Nobody Is Watching
There is another gap forming, and most enterprises do not see it coming. As large-scale inference scales, the binding constraint is shifting from raw GPU compute to memory bandwidth, specifically KV-cache capacity. Roughly one in five respondents (18%) either do not recognize this shift or have not begun to address it.
Among those who have, the market is fragmented. Dell leads at 31%, Nvidia follows at 16%, and the remainder splits across Hammerspace, DDN, VAST Data, WEKA, and open-source tooling. For a transition that will reshape inference architecture and cost, enterprise preparedness is scattered at best.
What This Means for the Industry
The compute gap is not simply a capacity problem. Throwing more hardware at it will not close the loop. The deeper issue is instrumentation: enterprises are buying infrastructure faster than they are building the visibility to spend wisely. This mirrors the early days of cloud adoption, when companies migrated workloads before they had FinOps practices in place, then spent years untangling shadow IT and surprise bills. AI infrastructure appears to be following a similar arc, only at a steeper velocity.
For mid-market enterprises, the risk is real. They are the ones driving the evaluation of neoclouds and alternative accelerators, yet they are also the least likely to have dedicated infrastructure finance teams. The next wave of purchasing may deliver better price-performance on paper, but without cost tracking, the savings will be theoretical.
The open question is whether enterprises build that visibility before the re-platforming arrives, or whether they buy the next layer of infrastructure as blind to its economics as the last.