The Universal Approximation Theorem: Why One Hidden Layer Can Learn Any Function

In 1969, two MIT professors published a book that nearly killed an entire field of research. Marvin Minsky and Seymour Papert's Perceptrons proved, with unforgiving mathematical rigor, that the simplest neural networks of the day—single-layer perceptrons—could never learn something as basic as the XOR function: output 1 if exactly one of two inputs is "on," otherwise output 0. No amount of extra neurons in that single layer could fix it. Funding dried up. The field entered what historians now call the first AI winter, a cold spell that lasted into the early 1980s.

Twenty years later, in 1989, a quiet piece of mathematics helped thaw it. Working independently, three groups of researchers proved something that sounds almost too good to be true: a neural network with just one hidden layer, given enough neurons in that layer, can approximate any continuous function to any level of precision you want. Not just XOR. Not just a narrow family of well-behaved curves. Any continuous function on any bounded region of space. This result became known as the Universal Approximation Theorem, and it is one of the reasons neural networks were ever taken seriously as a general-purpose tool for learning from data, rather than a narrow trick for a handful of toy problems.

The Concept

Strip away the jargon and a neural network's hidden layer is doing something fairly mechanical: it takes your input, multiplies it by a set of weights, adds a bias, and passes the result through a nonlinear "activation function"—classically a sigmoid curve, shaped like a stretched-out S. Each of these units draws a smooth little bump or step somewhere in the input space. The output layer then adds up a weighted combination of all those bumps.

The Universal Approximation Theorem says: if you're allowed to use as many of these bump-drawing units as you like, you can stack enough of them, in the right places with the right heights, to sculpt any continuous function out of those bumps—the same way you can approximate any smooth curve by laying down enough narrow rectangles underneath it, a trick familiar from introductory calculus. Make the rectangles (or bumps) numerous and thin enough, and the approximation error shrinks toward zero.

George Cybenko proved this in 1988–89 for networks using continuous sigmoidal activation functions, showing that such single-hidden-layer networks are dense in the space of continuous functions on any compact (closed and bounded) region—meaning you can get arbitrarily close to any target function. That same year, Kurt Hornik, Maxwell Stinchcombe, and Halbert White published an independent proof in the journal Neural Networks, extending the result to a much broader class of bounded, non-constant activation functions, and to any Borel measurable function—a far larger universe of functions than just the continuous ones. A Japanese researcher, Ken-Ichi Funahashi, arrived at a closely related result the same year as well. Three independent teams, converging on the same answer, is usually a good sign a result is both true and important.

Why It Matters

Before 1989, it was reasonable to wonder whether neural networks were fundamentally limited—whether the field's critics, echoing Minsky and Papert, were right that these networks were a dead end for anything beyond toy problems. The Universal Approximation Theorem answered the "can they, in principle, learn this?" question with a clean yes, for essentially any function you could care about: stock price curves, image classification boundaries, the physics of a robot arm, the mapping from a sentence to its translation. If the relationship between inputs and outputs can be described by some continuous function, a big enough one-hidden-layer network can represent it.

This theoretical green light mattered enormously for how the field thought about itself. It reframed the central challenge of machine learning: the question was no longer "is this architecture capable, even in principle, of learning our target?" but "can we find the right weights, with the data and compute we actually have?" That second question—a question about learning, not capacity—is the one that has occupied the field for the three and a half decades since, and it is the one deep learning's practical breakthroughs (better optimizers, more data, GPUs, architectural tricks like convolutions and attention) have been chipping away at ever since.

There's a subtlety worth sitting with, though, because it's the part most popular explanations skip: the theorem is an existence proof, not a construction. It tells you a network of sufficient width exists; it says nothing about how many neurons "sufficient" actually means, and nothing about how to find the right weights by training. For some functions, that required width turns out to be astronomically, impractically large. The classic example is the parity function—does a string of n bits contain an even or odd number of 1s? A single hidden layer needs a number of neurons that grows exponentially with n to compute parity reliably, while a deep network with several layers can do it with a number of neurons that only grows polynomially, using roughly log(n) layers. This is the mathematical seed of why the field eventually abandoned "wide and shallow" in favor of "deep": depth buys you an efficiency that width alone cannot.

The Details

Later work sharpened and generalized the original results considerably. Hornik extended his own 1989 result in 1991 to cover approximation of derivatives, not just function values—important for anything involving gradients or dynamics. Then in 1993, Moshe Leshno, Vladimir Lin, Allan Pinkus, and Shimon Schocken proved perhaps the most practically consequential generalization: the universal approximation property holds for any activation function that is not a polynomial. This is a big deal in hindsight, because it means the theorem's guarantee extends automatically to ReLU (the "rectified linear unit," essentially max(0, x))—the simple, non-smooth, unbounded activation function that would come to dominate deep learning two decades later, long after sigmoids fell out of favor for being slow to train. The theoretical floor was already laid for ReLU before anyone was using it at scale.

It's worth placing the Universal Approximation Theorem next to an older, related idea: the Kolmogorov–Arnold representation theorem, proved by Andrey Kolmogorov in 1957 and refined by his student Vladimir Arnold shortly after. Kolmogorov's theorem says any continuous multivariable function can be written as a finite sum of continuous single-variable functions, composed and added together—an exact representation, not an approximation. It's tempting to treat these as the same idea, but they're subtly different: Kolmogorov-Arnold gives an exact decomposition using functions that can be wild and problem-specific, while the Universal Approximation Theorem gives an approximate one using a fixed, simple family of functions (the activation function) that you choose in advance. Decades later, Kolmogorov-Arnold's representation would inspire a modern architecture—Kolmogorov-Arnold Networks, or KANs—that puts learnable functions on the network's edges instead of fixed activations on its nodes, an active area of research as recently as the last couple of years.

Researchers have continued probing the edges of universal approximation into the present day. A 2020 result by Sejun Park, Chulhee Yun, and Jaeho Lee pinned down the minimum width a deep network needs to universally approximate functions between finite-dimensional spaces, showing the question of "how many neurons, minimally" has sharp, provable answers once you fix the depth. The existence-versus-constructiveness gap that the original theorem left open has itself become a decades-long research program.

It's also worth pausing on just how strange the "single hidden layer" framing actually is, once you've absorbed the exponential-width caveat. The theorem is simultaneously completely true and almost useless as engineering advice. True, because no continuous function escapes its net, given enough neurons. Almost useless, because "enough neurons" can mean more neurons than there are atoms in the observable universe for some functions, and because the theorem says nothing about whether gradient descent can ever find the right weights even if a tiny, sufficient network exists. Theoretical capacity and practical learnability turned out to be two very different mountains to climb, and 1989 only settled the first one.

Takeaways

The Universal Approximation Theorem (Cybenko, and independently Hornik–Stinchcombe–White and Funahashi, all 1989) proved that a neural network with a single hidden layer can approximate any continuous function to arbitrary precision, given enough neurons.
It arrived twenty years after Minsky and Papert's 1969 Perceptrons had shown single-layer networks couldn't even solve XOR—helping restore theoretical confidence in neural networks after the first AI winter.
The theorem is an existence proof, not a recipe: it says a sufficiently wide network exists, but gives no bound on how wide "sufficient" is, and some functions (like parity) need exponentially many neurons in one layer versus only polynomially many across several layers.
That width-versus-depth gap is a large part of why the field moved toward deep networks rather than ever-wider shallow ones.
Leshno et al.'s 1993 generalization to any non-polynomial activation function quietly pre-justified ReLU, the activation function that would power the deep learning boom a decade later.

Resources: - Cybenko, G. (1989). "Approximation by Superpositions of a Sigmoidal Function." Mathematics of Control, Signals, and Systems. - Hornik, K., Stinchcombe, M., & White, H. (1989). "Multilayer Feedforward Networks are Universal Approximators." Neural Networks, 2(5), 359-366. - Leshno, M., Lin, V. Y., Pinkus, A., & Schocken, S. (1993). "Multilayer Feedforward Networks With a Nonpolynomial Activation Function Can Approximate Any Function."