Total Recall - C. Gordon Bell [18]
Our aim was to preview and to help lay the groundwork for the Total Recall systems that are coming soon—very soon—and that by 2020 will be as commonplace as Web browsers and cell phones are today. A few early steps in the Total Recall revolution have already hit the market. These include Evernote, reQall, OneNote, Google’s Web history, and support for desktop search in operating systems. But as this book is being written these products remain small discrete solutions within a much larger puzzle.
HOW TO ORGANIZE AN E-MEMORY
Back in 2001 I could see we still had a lot of basic things to figure out about how to store and organize my data. We had just my sixteen gigabytes of documents and photographs loaded in my imperfect classification hierarchy of folders, and we had no good way to search them, sort them, annotate them, link them together into Bush’s “trails,” or analyze them for patterns and trends.
The files-and-folders method of organizing data is a fundamental feature of all modern operating systems such as Windows, Mac intosh, and Linux. File-and-folder hierarchies, even when stored digitally, suffer from the same basic limitation as libraries once did: Each book can exist only in one place, filed under one category. But an item might properly belong to several categories, or hundreds. A Brief History of Time is a physics book, but it’s also a book by Stephen Hawking, it was a best-seller, it talked about black holes, and it was published in 1988. You could easily come up with dozens of other attributes that would be perfectly legitimate criteria for tracking down A Brief History of Time and for sorting and grouping it with other books (and for that matter, for sorting and grouping it with other media of any kind: with lecture recordings, with songs, with articles, with pictures, with old news footage).
The MyLifeBits project ran into the problem like this: Logically, my Eagles folder should have been stored in both my Fun folder and my Animals folder, but in practice I had to make an arbitrary choice. And often, if I wanted to find some half-remembered piece of data again, I’d have to hunt for it in various folders, asking myself: If I were me, where would I file it right now?
Librarians have been familiar with this restriction for centuries. A copy of a book can only be on one shelf in just one section, often determined by the Dewey decimal system of book topics. So they created paper card catalogs, where a card was a surrogate for a book. Now the book—or at least its surrogate card—could be filed in more than one place. Dewey might have it placed in the physics section, but it could also be in the title card catalog, filed alphabetically by its title, as well as being filed alphabetically by author in the author card catalog. For your convenience, the Dewey subject index would be duplicated in the card catalog also, allowing you to flip through cards with your fingers rather than hiking down the aisles.
So here I was, with a system that was worse than a library with paper card catalogs. I was like a librarian who was not allowed to have a card catalog. Jim Gray, who is widely celebrated as a pioneer, even a founding father, of database design, shook his head over me.
“You need to use a database, Gordon. When are you going to listen to me?” he would ask.
I was resistant. “We don’t need no stinkin’ databases,” I’d reply.
My resistance stemmed from my first experiences with databases back in the 1970s. Back then databases were still new and much hyped—they were also space-hogging and hard to use. I knew they’d been improved in the time since, but I’d heard enough horror stories over the years to keep my prejudices well nourished. Also, I wasn’t clear on exactly what I wanted out of even a well-behaved one. But, it turned out Jim Gray was right—as usual.
A database is a program for storing and retrieving large collections of interrelated information. Modern databases let you very quickly retrieve all the records with a given attribute. You can rapidly sort, sift, and combine information