The tradeoff the paper attacks
A team of twelve researchers has published a new architecture for processing multi-dimensional data without paying the quadratic cost of standard attention. The approach, called Native Multi-Dimensional Subquadratic Operators, replaces the attention matrix with input-dependent long convolutions and gets around the usual tradeoffs that have kept convolutions and attention locked in their respective lanes.
For one-dimensional sequences, the choice between subquadratic attention and convolution is mostly settled. Linear attention, state-space models, and long convolutions all work, and the question is which one you tune. For two-dimensional and three-dimensional data, the menu is much thinner. Standard convolutions have the right shape but lack global receptive fields, and they are not input-dependent. Recurrent models can match the receptive field, but they require rasterizing images, volumes, and PDE grids into a one-dimensional order, which is a workaround that rarely works well.
The paper's argument is that this dichotomy is artificial. With the right operator design, you can have input dependence, global receptive fields, and multi-dimensional structure without forcing the data into a sequence.
How the architecture works
The core operator is an input-dependent long convolution that runs natively over multi-dimensional grids. Because the convolution kernel is a function of the input, the model adapts its behavior per sample rather than using a fixed filter. That gives it the input dependence of attention without paying attention's quadratic cost.
The paper's experiments show that the architecture holds up across image, volumetric, and PDE settings, including the kinds of tasks where rasterization-based recurrent models tend to break. The team also reports that the approach composes with the surrounding training stack, which is the part that often gets glossed over in architecture papers.
Why it is interesting now
Subquadratic attention has been one of the most active research areas in deep learning for three years, and most of the wins have been on language. This paper is one of the cleanest demonstrations that the same family of techniques can extend cleanly to vision and to scientific computing, where the data structures do not look like sequences.
That matters because a lot of real workloads (medical imaging, climate modeling, computational physics, geospatial) are inherently multi-dimensional, and the field has been stuck choosing between slow-but-expressive attention and fast-but-limited convolutions. A workable middle path would change the cost structure for a meaningful slice of applied AI.
The paper is on arXiv, and the broader question is whether the architecture holds up at scale on standard benchmarks. The early results are promising, and the framing is careful enough that it is worth a serious read by anyone working on vision or scientific ML.