How Can AI Researchers Save Energy? By Going Backward. | Quanta Magazine

This loss is a fundamental aspect of how computers operate. For example, when a computer adds two numbers together, it returns a single number for the total: 2 + 2 = 4. There’s a loss of information as you go from two numbers to one. You could have added 2 and 2, or you could have combined 1 and 3. The missing information makes the calculation irreversible. Computers that process information this way — and almost all of them do — are always going to lose some information as heat, no matter what.

Landauer wondered if a machine could get around this limitation by simply never deleting data. Such a device would need to keep a record of every operation, every pair of numbers added at every step. These records would rapidly fill its memory, making such a computer unusable in practice, despite the energy savings. Landauer soon moved on, convinced that reversible computing was a dead end.

A decade later, he learned that he had been mistaken.

Hitting Reverse

Charles Bennett, a younger colleague of Landauer’s at IBM, argued in 1973 that there was another option. Instead of saving every single scrap of information, you could run each calculation forward, store the result you care about, then run the calculation backward. Bennett’s idea, which he called uncomputation, is a little like if Hansel and Gretel picked up their trail of breadcrumbs on the way back home: The pair are guaranteed not to get lost, and they don’t waste any breadcrumbs. Uncomputation means you are left with only the data you want, and you never lose track of it. Because none of the initial information is deleted, you never lose energy to heat.

Unfortunately, uncomputation also takes twice as long as an ordinary computation, which makes it impractical.

Still, Bennett continued to improve on his idea. In 1989, he showed that you can uncompute in much less time by using slightly more memory. Researchers began to tinker with the details, finding ways to shave off further memory and time.

But computers don’t lose energy just from deleting data. The way their transistors are linked is inherently inefficient, so for a computer to save a meaningful amount of energy from reversible computing, it must be designed with low heat loss in mind from the beginning.

In the 1990s, a group of engineers at the Massachusetts Institute of Technology set out to do exactly that. The team built prototype chips that improved the inefficient circuits. Frank joined the group as a doctoral student in 1995 and soon became one of reversible computing’s chief proponents.

Then, at the start of the new millennium, interest slowed. The chips didn’t yet have what it took to save energy in the real world. Support was scarce.

“Program reviewers would say, ‘This stuff sounds really useful, industry should be funding it,’ and yet you go to industry and they have no idea what you’re talking about,” Frank said. “It sounds crazy to them,” because it dealt with a problem that seemed so remote. Regular computer chips were improving exponentially. Why worry about theoretical alternatives?

Frank abandoned the work, and for a while he even left the field to open an internet café. But before long, the concerns that industry thought were distant got closer. Computer circuits were getting so small that they were hitting fundamental physical limits that would stop them from shrinking any more.

“You aren’t going to be able to scale conventional technology any further,” he said.

Frank built a research group at Sandia National Labs and began trying to rally attention toward energy efficiency.

Then, in 2022, Hannah Earley, at the time a researcher at the University of Cambridge, delivered a rigorous account of how efficient these computers could be. A reversible computer emits much less heat than a conventional one, but she found it must still emit some heat. When voltage turns on in a wire, for example, the metal heats up, with more heat the faster the voltage changes. The more slowly a reversible computer runs, the less heat it emits, a relationship Earley calculated precisely.

That relationship between heat and speed is crucial for reversible computing’s most promising application: AI. Computations in AI are often run in parallel, meaning different processors each run one part of a computation. This creates an opportunity for reversible computing to shine. If you run reversible chips more slowly, but use more of them to compensate, you end up saving energy: The advantage of running each chip more slowly wins out against the disadvantage of running more chips. And if you run them slowly enough, you might get away with not needing as much cooling, which will let you pile chips closer together to save on space, materials, and time spent shuttling data between them.

Investors have taken notice. Earley has co-founded Vaire Computing, where she and Frank are working to create a commercial version of a reversible chip.

After decades of theory, said Torben Ægidius Mogensen, who works on reversible computing at the University of Copenhagen, we may finally see this approach in action. “The most exciting thing would be actually seeing reversible processors made in practice so we can use them.”