Most people don’t care about clinical trials when it comes to treatment efficacy; they trust personal experience. If an infinitesimally diluted solution of nightshade seems to work for them, then this is their pudding and the proof is in it. Some puddings, however, are better than others, and the “it worked for me” pudding is perhaps the least consistent of them all.

Personal experience is simply an unreliable method of assessing the worthiness of a particular treatment against a specific illness. Why is that?

Imagine the following scenario: you have the flu. You’ve been sneezing and coughing for three days now and you’re hit with a fever. You’ve finally had it. You send your significant other to the drugstore to buy a homeopathic remedy, Oscillococcinum, which claims to eradicate flu-like symptoms. You pop a pill. The next day, you feel better. You pop another pill. The following day, even better. Pretty soon, your illness is gone. Oscillococcinum has now been proven to work against influenza.

Or has it?

Is it possible that you reached for the medication just as your symptoms hit their worst and that your immune system was perfectly capable of handling your influenza and would have behaved identically with or without the homeopathy?

Is it possible that nothing can be deduced of the efficacy of Oscillococcinum against influenza in this scenario?

Indeed, similar scenarios can be formulated based on this “personal experience” fallacy. Some illnesses are ill-diagnosed in the first place, yet other diseases vanish on their own for reasons that elude present-day science. Then there’s confirmation bias, our predilection for remembering the correlations we like to believe (e.g. when I took Oscillococcinum two years ago, my flu went away in a jiffy) and forgetting the correlations that disagree with our desires (e.g. when I took Oscillococcinum last year, my flu actually got worse for a few days… when I didn’t take Oscillococcinum three years ago, my flu went away on its own). Then there’s the different strains of flu every year, which our immune system may fight with a different intensity, and our unreliable memory, and a whole bag of biases and variables that make one human being a terrible laboratory.

The strength of human beings in medicine comes in numbers, and this is where the clinical trial shows its strength.

The randomized, double-blinded, placebo-controlled clinical trial is one of the crowning achievements of modern medicine. It is a system of proofing which removes most variables from the equation until one is looking at just the one: the efficacy of the treatment.

Take a large group of people. A large group of people. There is confidence in numbers.

Split this group into two subgroups randomly. You don’t want men on one side and women on the other; you don’t want health nuts at one end and couch potatoes at the other; you don’t want the Irish on one side and Cambodians on the other. You want to create homogenous subgroups so that they are roughly identical.

One subgroup will receive the treatment the efficacy of which is being assessed. The other subgroup will receive a placebo intervention: a sugar pill, an injection of saline, something that looks a lot like the treatment under trial but which is innocuous. If your studied substance comes in a syringe but the placebo is in pill form, whatever benefit you get from the former may simply be due to the placebo effect of getting jabbed, so the placebo must mimic the studied treatment in its form.

Can the participants know what they are receiving? No. They sign a consent form which states they will be randomly assigned to either the placebo arm or the new treatment arm, but they will not know what they will be receiving. This way, they can’t decide they must be feeling better because they received the new treatment.

Can the administrators of the treatment know what they are giving? No. This kind of trial is double-blinded, so neither the receiver nor the giver knows what’s being administered. Only the people running the trial behind the scenes know.

Obviously, modifications to this formula have to be made when a new treatment is meant to supersede a preexisting one. We don’t withhold cancer treatment to test a new drug: this would be unethical. In these cases, the placebo arm becomes “standard of care + placebo” and the new treatment arm becomes “standard of care + new treatment”. The cold objectivity of science has to be tempered with the humanness of morality.

In science, a sample size is represented by the letter n. When faced with anecdotal evidence, a health scientist will often say, “But that’s n = 1”, and rightly so. Personal experience easily drives our decisions but it is far from being a good epistemology for determining treatment efficacy.

(Feature picture by Caravaggio)