OUT THERE

Mystery of Big Data’s Parallel Universe Brings Fear, and a Thrill

Elwood H. Smith

By DENNIS OVERBYE

Published: June 4, 2012

Not long ago, a woman in Tacoma, Wash., received a suggestion from Facebook that she “friend” another woman. She didn’t know the other woman, but she followed through, as many of us have, innocently laying our cookie-crumb trails through cyberspace, only to get a surprise.

On the other woman’s profile page was a wedding picture — of her and the first woman’s husband, now exposed for all the cyberworld to see as a bigamist.

And so it goes in the era of what is called Big Data, in which more and more information about our lives — where we shop and what we buy, indeed where we are right now — the economy, the genomes of countless organisms we can’t even name yet, galaxies full of stars we haven’t counted, traffic jams in Singapore and the weather on Mars tumbles faster and faster through bigger and bigger computers down to everybody’s fingertips, which are holding devices with more processing power than the Apollo mission control. Big Data probably knows more about us than we ourselves do, but is there stuff that Big Data itself doesn’t know it knows? Big Data is watching us, but who or what is watching Big Data?

It is perhaps time to be afraid. Very afraid, suggests the science historian George Dyson, author of a recent biography of John von Neumann, one of the inventors of the digital computer. In “A Universe of Self-Replicating Code,” a conversation published on the Web site Edge, Mr. Dyson says that the world’s bank of digital information, growing at a rate of roughly five trillion bits a second, constitutes a parallel universe of numbers and codes and viruses with its own “physics” and “biology.”

There are things going on inside that universe that we don’t know about, he points out — except when it produces unpleasant surprises, as it did during the “flash crash” of the stock market in May 2010. And we had better find out what they are.

Where is Hari Seldon when we need him?

Unfortunately, he’s not real — yet. Hari Seldon, a mathematician and “psychohistorian,” who had figured out the laws governing history and society, was the central figure in Isaac Asimov’s magisterial “Foundation” trilogy. Set thousands of years in the future, the books follow Seldon’s and his followers’ attempts to preserve civilization during the impending collapse of the Galactic Empire.

There is something both spooky and grand about the idea that our lives are part of patterns and currents still invisible to us, like climate cycles yet undetected in the geological record. Or maybe not so grand: In Kurt Vonnegut’s novel “The Sirens of Titan,” all of human evolution and history has been manipulated from afar to deliver a widget about the size and shape of an old-fashioned beer can opener to a robot astronaut stranded by a broken spaceship on that Saturnian moon.

Nevertheless, if you could discover those principles, you might be able — dare I say it? — to rule the world.

Both Paul Krugman, the liberal Nobel laureate economist and Op-Ed columnist for The New York Times, and Newt Gingrich, the conservative former House speaker and presidential candidate, have admitted to being inspired as young men by the dream of being Hari Seldon.

Mr. Krugman told The New Yorker a couple of years ago that it was the lure of discovering such laws that made him want to be an economist.

You might think that the “physics” of these systems should not be a mystery, since we created them. But that does not mean we know what will come out of them. Alan Turing, the British mathematician and computer pioneer who was born 100 years ago this month, showed that even a simple routine or set of instructions left to run endlessly can produce complex results. It “can (indeed will) produce statements (and behavior) that we cannot necessarily understand,” wrote Mr. Dyson in an e-mail.

Surprises — what the complexity theorists call emergent properties — are part of the game. Do ants know they are in an anthill?

In the 1970 science fiction movie “Colossus: the Forbin Project” — a favorite among computer scientists, including Mr. Dyson — scientists create a supercomputer network. The first thing Colossus discovers after being turned on is that there is a similar network in the Soviet Union. They join forces and take over the world.

I wondered if such surprises occurred in the computer models used by the financial world. So I reached out to another Hari Seldon admirer, J. Doyne Farmer, a physicist and complexity theorist at the Santa Fe Institute in New Mexico and a founder of the Prediction Company, which is now owned by UBS, the giant Swiss bank.

Dr. Farmer said classical economics had failed miserably to provide the right data for us to understand ourselves. He and others have begun to develop so-called agent-based models of the economy, asking in effect how the seemingly random behavior of individual ants can give rise to anthills with all their pulsing purpose, form and intelligence.

It works great for ants, and it’s pretty to think that we might have something to learn about ourselves from our little six-legged friends as they carry off the crumbs from another picnic. Even if it means there is nothing more profound than a 22nd-century beer-can opener in our future
.

A version of this article appeared in print on June 5, 2012, on page D3 of the New York edition with the headline: Mystery of Big Data’s Parallel Universe Brings Fear, and a Thrill.