Knocking on Heaven's Door - Lisa Randall [104]
The way to tell is to do the test many times. Each time you have a headache, flip a coin to decide whether to take an aspirin or not and record the result. After you do this enough, you can average out over all the different types of headaches you had and the varying circumstances in which you had them (maybe they go away faster when you’re not so sleepy) and use your statistics to find the right result. Presumably there is no bias in your measurement since you flipped a coin to decide and the population sample you used was just yourself so your result will correctly converge with enough self-imposed tests.
It would be nice to always be able to learn whether drugs worked with such a simple procedure. However, most drugs are treating more serious illnesses than headaches—perhaps even ones that lead to death. And many drugs have long-term effects, so you couldn’t do repeated short-term trials on a single individual even if you wanted to.
So usually when biologists or doctors test how well a drug works, they don’t simply study a single individual, even though for scientific purposes at least they would prefer to do so. They then have to contend with the fact that people respond differently to the same drug. Any medicine produces a range of results, even when tested on a population with the same degree of severity of a disease. So the best scientists can do in most cases is to design studies for a population as similar as possible to any given individual they are deciding whether or not to give the drug to. In reality, however, most doctors don’t design the studies themselves, so similarity to their patient is hard for them to guarantee.
Doctors might want instead to try to use pre-existing studies where no one did a carefully designed trial but the results were based simply on observations of existing populations, such as the members of an HMO. They would then face the challenge of making the correct interpretation. With such studies, it can be difficult to ensure that the relevant measurement establishes causality and not just association or correlation. For example, someone might mistakenly conclude that yellow fingers cause lung cancer because they noticed many lung cancer patients have yellow fingers.
That’s why scientists prefer studies in which treatments or exposures are randomly assigned. For example, a study in which people take a drug based on a coin toss will be less dependent on the population sample since whether or not any patient receives treatment depends only on the random outcome of a coin flip. Similarly, a randomized study could in principle teach about the relationships among smoking, lung cancer, and yellow fingers. If you were to randomly assign members of a group to either smoke or refrain from smoking, you would determine that smoking was at least one underlying factor responsible for both yellow fingers and lung cancer in the patients you observed, whether or not one was the cause of the other. Of course, this particular study would be unethical.
Whenever possible, scientists aim to simplify their systems as much as possible so as to isolate the specific phenomena they want to study. The choice of a well-defined population sample and an appropriate control group are essential to both the precision and accuracy of the result. With something as complicated as the effect of a drug on human biology, many factors enter simultaneously. The relevant question is then how reliable do the results need to be?
THE OBJECTIVE OF MEASUREMENTS
Measurements are never perfect. With scientific research—as with any decision—we have to determine an acceptable level of uncertainty.