In Deep Learning, Which Paper Has the Most Astonishing Idea?

I once dreamed of a paper like this.

It was no longer the formulaic ICLR paper, with a dispensable proof and experiment pasted onto some wildly unrealistic setting; nor was it another monster born from piling on compute, a new architecture fed by infinite data and electricity.

It made one gentle move and wrote down a one-line formula: the definition of intelligence.

At first glance, the formula looked utterly natural, as if it had come from certain concepts in thermodynamics, while quietly containing the statistical relationship between sample size and accuracy.

It was simple: no more than two or three variables, one or two layers of summation or integration.

It was precise: it quantified the gap between a human, a dog, and ResNet.

It was profound: sweeping away every distracting condition, it pointed straight at the root of intelligence.

And so heaven and earth changed.

It could clearly give the lower bounds on the information entropy of the data needed for training, and on the amount of computation required. It also gave you a hint, enough to send you chasing the “perfect” model that would keep approaching the theoretical limit.

The boundary between supervised learning and unsupervised learning was wiped clean. In its eyes, anything that contained information was the same, and all of it could be used to improve intelligence.

Every old model and training method was overthrown as well. People discovered that these bloated architectures, which countless researchers had experimented on, maintained, and analyzed, had been so crude after all.

With it, people finally dared to look directly at their own brains. It turned out that the Creator had known this formula only a few hundred thousand years earlier than we did, and only in the final moment of a long evolution had He created us.

All stale theoretical research was cast aside, and a unified theory simplified machine learning until anyone could master its mysteries. Many people lost the tools they had depended on for paper-padding, but even more rushed in, using elegant proofs and intricate, unreadable frameworks to build one unimaginable intelligent application after another. Yet all of it was only a footnote to that one-line formula, already determined the moment it appeared.

In later generations, undergraduates would pick up a textbook called The Theory of Intelligence. Reading its concise theory and clear derivations, they would ask us:

“This thing is so obvious. How did you fail to think of it?”

We would grow emotional, telling them how generations of scientists moved from rule-based intelligence to statistics as the foundation, and then, step by step, from linear models to a colossus like ChatGPT.

But the students would dismiss all of it:

“Wouldn’t it have been better to think of this definition earlier? Look at all that electricity burned, all that labor wasted, all that data labeled. All useless effort. It is not even as good as the code undergraduates write now, solving the problem in one line. You were exactly like Ptolemaic astronomers, your imagination bound by religion, studying geocentrism day after day. Such beautiful heliocentrism was right there, and yet you could not see it…”

We would argue that only in darkness can people know the value of candlelight. But then, suddenly, we would remember that they had grown up beneath the sun.

A tear fell from the corner of my eye. So that was it: we had grown old.

When I stared again at that definition, it suddenly blurred. One Greek letter after another diffused into the void, becoming wrinkled and hard to recognize. I tried everything to bring that one-line formula out from the dream. But all I brought back was that tear, reflecting the morning sun outside the window.

I got up, sat down at my computer, and returned to the tedious business of tuning parameters, reading papers, and squeezing out papers.

But I always remember that definition, from that beautiful dream…