Everything Is Obvious_ _Once You Know the Answer - Duncan J. Watts [24]
Recently, for example, my Yahoo! colleague Winter Mason and I conducted a series of Web-based experiments in which subjects were paid at different rates to perform a variety of simple repetitive tasks, like placing a series of photographs of moving traffic into the correct temporal sequence, or uncovering words hidden in a rectangular grid of letters. All our participants were recruited from a website called Amazon’s Mechanical Turk, which Amazon launched in 2005 as a way to identify duplicate listings among its own inventory. Nowadays, Mechanical Turk is used by hundreds of businesses looking to “crowd-source” a wide range of tasks, from labeling objects in an image to characterizing the sentiment of a newspaper article or deciding which of two explanations is clearer. However, it is also an extremely effective way to recruit subjects for psychology experiments—much as psychologists have done over the years by posting flyers around college campuses—except that because workers (or “turkers”) are usually paid on the order of a few cents per task, it can be done for a fraction of the usual cost.21
In total, our experiments involved hundreds of participants who completed tens of thousands of tasks. In some cases they were paid as little as one cent per task—for example, sorting a single set of images or finding a single word—while in other cases they were paid five or even ten cents to do the same thing. A factor of ten is a pretty big difference in pay—by comparison, the average hourly rate of a computer engineer in the United States is only six times the federal minimum wage—so you’d expect it to have a pretty big effect on how people behave. And indeed it did. The more we paid people, the more tasks they completed before leaving the experiment. We also found that for any given pay rate, workers who were assigned “easy” tasks—like sorting sets of two images—completed more tasks than workers assigned medium or hard tasks (three and four images per set respectively). All of this, in other words, is consistent with common sense. But then the kicker: in spite of these differences, we found that the quality of their work—meaning the accuracy with which they sorted images—did not change with pay level at all, even though they were paid only for the tasks they completed correctly.22
What could explain this result? It’s not completely clear; however, after the subjects had finished their work we asked them some questions, including how much they thought they ought to have been paid for what they had just done. Interestingly, their responses depended less on the difficulty of the task than on how much they had been paid to do it. On average, subjects who were paid one cent per task thought they should have been paid five cents; subjects who were paid five cents thought they should have been paid eight cents; and subjects who were paid ten cents thought they should have been paid thirteen cents. In other words, no matter what they were actually paid—and remember that some of them were getting paid ten times as much as others—everyone thought they had been underpaid. What this finding suggested to us is that even for very simple tasks, the extra motivation to perform that we intuitively expect workers to experience with increased financial incentives is largely undermined by their increased sense of entitlement.
It’s hard to test this effect outside of a laboratory setting, because workers in most real environments have expectations about what