Reinventing Discovery_ The New Era of Networked Science - Michael Nielsen [64]
But some phenomena don’t have simple explanations. Think about the problem of translating Spanish into English. These languages contain a great deal of accidental complexity, as a result of all the contingencies in their historical genesis. To make high-quality translations we have no choice but to deal with all that complexity. In everyday life translators do this in part through a wealth of knowledge about the details of the languages, and in part through hard-to-describe intuition, built up over years of exposure to both languages. Any really precise explanation of how to translate from Spanish to English will necessarily be quite complex, and certainly won’t have the simplicity of the theory of evolution or the general theory of relativity.
Until recently, the complexity of the scientific explanations we use was constrained by the limitations of our own minds. Today, this is changing, as we learn how to use computers to build and then work with extremely complex models. To explain the change, let me give an example from the field of machine language translation. Starting around 1950, researchers began building computerized systems whose aim was to automatically translate from one language to another. Unfortunately, the early systems weren’t very good. They tried to do the translation using clever, relatively simple models based on the rules of grammar and other rules of language. This sounds like a good idea, but despite a lot of effort, it never worked very well. It turns out that human languages contain far too much complexity to be captured in such simple rules.
In the 1990s researchers in machine translation began trying a new and radically different approach. They threw out the conventional rules of grammar and language, and instead started their work by gathering an enormous corpus of texts and translations—think, say, of all the documents from the United Nations. Their idea was to use data-driven intelligence to analyze those documents en masse, trying to infer a model of translation. For instance, while analyzing the corpus the program might notice that Spanish sentences containing the word “hola” often have the word “hello” in their English translation. From this, the program would estimate a high probability that the word “hola” results in the word “hello” in the translated text, while the probability for English words unrelated to “hola” (“tiger,” “couch,” and “January,” for example) would be much lower. The program would also examine the corpus to figure out how words moved around in the sentence, observing, for example, that “hola” and “hello” tend to be in the same parts of the sentence, while other words get moved around more. Repeating this for every pair of words in the Spanish and English languages, their program gradually built up a statistical model of translation—an immensely complex model, but nonetheless one that can be stored on a modern computer. I won’t describe the models they used in complete detail here, but the hola-hello example gives you the flavor. Once they had analyzed the corpus and built up their statistical model, they used that model to translate new texts. To translate a Spanish sentence, the idea was to find the English sentence that, according to the model, had the highest probability. That high-probability sentence would be output as the translation.
Frankly, when I first heard about statistical machine translation I thought it didn’t sound very promising. I was so surprised by the idea that I thought I must be misunderstanding something. Not only do these models have no understanding of the meaning of “hola” or “hello,” they