• Choose a category
• All
• Classic-Fiction

By Root 406 0

don’t even understand the most basic things about language, such as the distinction between nouns and verbs. And, it turns out, my skepticism is justified: the approach doesn’t work very well—if the starting corpus used to infer the model contains just a few million words. But if the corpus has billions of words, the approach starts to work very well indeed. Today, this is the way the best machine translation systems work. If you’ve ever done a Google search that returned a result in a foreign language, you’ll notice that Google offers to “translate this page.” These translations aren’t done by human beings, or by special algorithms handcrafted with a detailed knowledge of the languages involved. Instead, Google uses an incredibly detailed statistical model of how to do translation. It’s far from perfect, but today it’s the best automated translation system around. Shortly after launching their translation service, Google easily won an international competition for English-Arabic and English-Chinese machines translations. What’s truly remarkable is that no one on the Google Translate team spoke Chinese or Arabic. They didn’t need to. The system could learn to translate by itself.

These translation models are in some sense theories or explanations of how to translate. But whereas Darwin’s theory of evolution can be summed up in a few sentences, and Einstein’s general theory of relativity can be expressed in a single equation, these theories of translation are expressed in models with billions of parameters. You might object that such a statistical model doesn’t seem much like a conventional scientific explanation, and you’d be right: it’s not an explanation in the conventional sense. But perhaps it should be considered instead as a new kind of explanation. Ordinarily, we judge explanations in part by their ability to predict new phenomena. In the case of translation, that means accurately translating never-before-seen sentences. And so far, at least, the statistical translation models do a better job of that than any conventional theory of language. It’s telling that a model that doesn’t even understand the noun-verb distinction can outperform our best linguistic models. At the least we should take seriously the idea that these statistical models express truths not found in more conventional explanations of language translation. Might it be that the statistical models contain more truth than our conventional theories of language, with their notions of verb, noun, and adjective, subjects and objects, and so on? Or perhaps the models contain a different kind of truth, in part complementary, and in part overlapping, with conventional theories of language? Maybe we could develop a better theory of language by combining the best insights from the conventional approach and the approach based on statistical modeling into a single, unified explanation? Unfortunately, we don’t yet know how to make such unified theories. But it’s stimulating to speculate that nouns and verbs, subjects and objects, and all the other paraphernalia of language are really emergent properties whose existence can be deduced from statistical models of language. Today, we don’t yet know how to make such a deductive leap, but that doesn’t mean it’s not possible.

What status should we give to complex explanations of thisype? As the data web is built, it will become easier and easier for people to construct such explanations, and we’ll end up with statistical models of all kinds of complex phenomena. We’ll need to learn how to look into complex models such as the language models and extract emergent concepts such as verbs and nouns. And we’ll need to learn how to cope with the fact that sometimes those emergent concepts will only be approximate. We’ll need, in short, to develop more and better tools for extracting meaning from these complex models.

With all that said, it still seems intuitive that simple explanations contain more truth than complex explanations. This prejudice against complex explanations in science is so ingrained that it even has a name: