• Choose a category
• All
• Classic-Fiction

By Root 265 0

explosion of a new type of data. If you analyzed the flow of digital data in, say, 1980, only a smidgen of the world’s information had found its way into computers. Back then, the big mainframes and the new microcomputers housed business records, tax returns, real estate transactions, and mountains of scientific data. But much of the world’s information existed in the form of words—conversations at the coffee shop, phone calls, books, messages scrawled on Post-its, term papers, the play-by-play of the Super Bowl, the seven o’clock news. Far more than numbers, words spelled out what humans were up to, what they were thinking, what they knew, what they wanted, whom they loved. And most of those words, and the data they contained, vanished quickly. They faded in fallible human memories, they piled up in Dumpsters and moldered in damp basements. Most of these words never reached computers, much less networks.

That has all changed. In the last decade, as billions of people have migrated their work, mail, reading, phone calls, and webs of friendships to digital networks, a giant new species of data has arisen: unstructured data. It’s the growing heap of sounds and images that we produce, along with trillions of words. Chaotic by nature, it doesn’t fit neatly into an Excel spreadsheet. Yet it describes the minute-by-minute goings-on of much of the planet. This gold mine is doubling in size every year. Of all the data stored in the world’s computers and coursing through its networks, the vast majority is unstructured. Hewlett Packard, for example, the biggest computer company on earth, gets a hundred fifty million Web visits a month. That’s nearly thirty-five hundred customers and prospects per minute. Those visits produce data. So do notes from the company’s call centers, online chat rooms, blog entries, warranty claims, and user reviews. “Ninety percent of our data is unstructured,” said Prasanna Dhore, HP’s vice president of customer intelligence. “There’s always a gap between what you want to know about the customer and what is knowable.” Analysis of the pile of data helps reduce that gap, bringing the customer into sharper focus.

The potential value of this information is immense. It explains why Facebook, a company founded in 2004, could have a market value six years later of $50 billion. The company gathers data, most of it unstructured, from about half a billion people. Beyond social networks and search engines, an entire industry has sprung up to mine this data, to predict people’s behavior as shoppers, drivers, workers, voters, patients, even potential terrorists. As machines, including Watson, have begun to chomp on unstructured data, a fundamental shift is occurring. While people used to break down their information into symbols a computer could understand, computers are now doing that work by themselves. The machines are mastering human communication.

This has broad implications. Once computers can handle language, every person who can type or even speak becomes a potential programmer, a data miner, and an analyst. This is the march of technology. We used to have typists, clerks, legions of data entry experts. With the development of new tools, these jobs became obsolete. We typed (and spell-checked), laid out documents, kept digital records, and even developed our own pictures. Now, a new generation of computers can understand ordinary English, hunt down answers in vast archives of documents, analyze them, and come up with hypotheses. This has the potential to turn entire industries on their heads.

In the August meeting, Ferrucci told the audience the story of his recent medical odyssey and how a machine like Watson could have helped. Others suggested that Watson could man call centers, function as a brainy research assistant in pharmaceutical labs, or work as a whip-smart paralegal, with nearly instant recall of the precedents, both state and federal, for every case. They briefly explored the idea of Watson as a super question-answering Google. After all, it could carry out a much more detailed analysis of questions