Reinventing Discovery_ The New Era of Networked Science - Michael Nielsen [48]
Finding Meaning in All the World’s Knowledge
For nearly all of recorded history, we human beings have lived our lives isolated inside tiny cocoons of information. The most brilliant and knowledgeable of our ancestors often had direct access to only a tiny fraction of human knowledge. Then, in the 1990s and 2000s, over a period of just two decades, our direct access to knowledge expanded perhaps a thousandfold. At the same time, a second, even more important expansion has been going on: an expansion in our ability to find meaning in our collective knowledge. We see this expansion in Swanson’s use of Medline to find connections hidden in our collective medical knowledge, or the way Google and the CDC combined the CDC’s existing (but inadequate) knowledge of reported flu with Google’s search data, to figure out a better way of tracking the flu. We also see examples in our everyday lives, such as Google’s ability to answer our questions, finding just the right webpage, news article, scientific paper, or book. Tools such as Google and Medline redefine our relationship to knowledge, by giving us ways of finding previously hidden meaning, all the “unknown knowns” that are implicit in existing human knowledge, but that are not yet apprehended because of the massive scale of that knowledge. Earlier in this book we saw how collective intelligence can be amplified by restructuring expert attention, to take better advantage of the available expertise. In this chapter we’ll discuss a complementary approach to amplifying collective intelligence: to build tools that perform cognitive tasks directly, operating on knowledge itself, by searching for meaning and hidden connections in our collective knowledge.
The remainder of this chapter is in two parts. The first part tells the story of a project from astronomy called the Sloan Digital Sky Survey (SDSS). The SDSS is surveying the universe, much as early mapmakers surveyed the Earth, using a robotic telescope to explore the sky broadly, so far taking images of 930,000 galaxies. Those images aren’t just pretty pictures; they’re being mined by astronomers to answer all sorts of questions about our universe. We’ll learn how the SDSS has been used to find the biggest known structure in the universe, a giant chain of galaxies 1.37 billion light-years long; to discover new dwarf galaxies near our Milky Way galaxy; and to find a pair of orbiting black holes. But although these discoveries are fascinating in their own right, there’s a deeper reason we’re interested in the SDSS. That’s because although access to human knowledge has expanded enormously over the past two decades, a great deal of scientific knowledge isn’t yet publicly accessible, and a struggle is going on to make it more accessible. And so the first part of the chapter tells the story of the expansion of the information commons in science, using the SDSS as a concrete example to understand both the benefits and the challenges of that expansion. That concrete understanding prepares us for the second part of the chapter, where we broaden our focus to think about the big picture. What are the implications of making all the world’s knowledge openly available? And what new methods of discovery will it enable?
Exploring the Digital Universe
The largest known structure in the universe is a chain of