Everything Is Obvious_ _Once You Know the Answer - Duncan J. Watts [90]
But let’s say that none of these attempts is effective. Perhaps the brand in question is just not appealing to particular demographics, or perhaps those people don’t respond to online advertising. Even in that event, however, the advertiser can at least stop wasting money advertising to them, freeing more resources to focus on the population that might actually be swayed. Regardless, the only way to improve one’s marketing effectiveness over time is to first know what is working and what isn’t. Advertising experiments, therefore, should not be viewed as a one-off exercise that either yields “the answer” or doesn’t, but rather as part of an ongoing learning process that is built into all advertising.21
A small but growing community of researchers is now arguing that the same mentality should be applied not just to advertising but to all manner of business and policy planning, both online and off. In a recent article in MIT Sloan Management Review, for example, MIT professors Erik Brynjolfsson and Michael Schrage argue that new technologies for tracking inventory, sales, and other business parameters—whether the layout of links on a search page, the arrangement of products on a store shelf, or the details of a special direct mail offer—are bringing about a new era of controlled experiments in business. Brynjolfsson and Schrage even quote Gary Loveman, the chief executive of the casino company Harrah’s, as saying, “There are two ways to get fired from Harrah’s: stealing from the company, or failing to include a proper control group in your business experiment.” You might find it disturbing that casino operators are ahead of the curve in terms of science-based business practice, but the mind-set of routinely including experimental controls is one from which other businesses could clearly benefit.22
Field experiments are even beginning to gain traction in the more tradition-bound worlds of economics and politics. Researchers associated with the MIT Poverty Action Lab, for example, have conducted more than a hundred field experiments to test the efficacy of various aid policies, mostly in the areas of public health, education, and savings and credit. Political scientists have tested the effect of advertising and phone solicitations on voter turnout, as well as the effect of newspapers on political opinions. And labor economists have conducted numerous field experiments to test the effectiveness of different compensation schemes, or how feedback affects performance. Typically the questions these researchers pose are quite specific. Should aid agencies give away mosquito nets or charge for them? How do workers respond to fixed wages versus performance-based pay? Does offering people a savings plan help them to save more? Yet answers to even these modest goals would be useful to managers and planners. And field experiments could be conducted on grander scales as well. For example, public policy analyst Randal O’Toole has advocated conducting field experiments for the National Park Service that would test different ways to manage and govern the national parks by applying them randomly to different parks (Yellowstone, Yosemite, Glacier, etc.) and measuring which ones work the best.23
THE IMPORTANCE OF LOCAL KNOWLEDGE
The potential of field experiments is exciting, and there is no doubt that they are used far less often than they could be. Nevertheless, it isn’t always possible to conduct experiments. The United States cannot go to war with half of Iraq and remain at peace with the other half just to see which strategy works better over the long haul. Nor can a company easily rebrand just a part of itself, or rebrand itself with respect to only some consumers and not others.24 For decisions like these, it’s unlikely that an experimental approach will be of much help; nevertheless, the decisions still have to get made. It’s all well and good for academics and researchers to debate the finer points of cause and effect, but our politicians and business leaders must often act in the absence of certainty.