Scientists caution against hasty conclusions when trying alternative remedies

A new study suggests that people tend to continue an unproven treatment when there’s only an initial marginal improvement in symptoms and otherwise no evidence to tell if it’s a potential coincidence.

"I've noticed many of my patients take unnecessary vitamins, pills, or alternative remedies with little evidence to inform their choice, leading to a lot of distraction, wishful thinking, and wasted money,” says Donald Redelmeier, senior author of the study, a staff internist and senior scientist at Sunnybrook Health Sciences Centre, and professor at the University of Toronto’s Temerty Faculty of Medicine.

“Perhaps even more concerning is a false belief that leads to a missed diagnosis that later becomes incurable,” Redelmeier adds.

The study, published in the journal JAMA Network Open, explores ‘post-hoc bias’ — a tendency in reasoning that causes many patients to continue taking dubious, or unreliable, treatments. The bias encourages a popular misconception: that because one action preceded another later event, the first must have caused the second since it occurred in sequence.

But medical science, the researchers note, stresses that the order of two events does not prove a cause-and-effect since coincidences are frequent. The implication for medical care is that a patient who improved after a treatment is not necessarily a patient who improved because of the treatment.

Instead, other potential explanations include withdrawal from an adverse activity, added rest, or the body’s own healing powers.

The researchers ran multiple experiments, using hypothetical clinical scenarios administered by randomized survey, to pharmacists and members of the community to test bias across a variety of clinical cases.

The scenarios described a patient with fatigue, or another vague symptom, who feels a bit better after trying a vitamin, shampoo, sugar pill, or other treatment.

“We found that most respondents suggested continuing the treatment indefinitely even though the change in symptoms might be pure random chance," says Redelmeier, who is affiliated with the department of medicineat Temerty Medicine, and the Institute for Health Policy, Management and Evaluation at U of T’s Dalla Lana School of Public Health.

"The post-hoc bias can play tricks on patients that can eventually lead to serious disappointments, and for health-care workers, it can ultimately lead to shortfalls in care."

While attributing a patient’s initial improvement of symptoms — or the lack of — to a treatment is an intuitive and quick approach, say the researchers, the study reinforces the need for patients to be cautious about the post-hoc bias.

Clinicians also need to continuously re-assess patients and resist the bias to assume that a patient who improves after a treatment is not necessarily due to the treatment itself, Redelmeier says. Similarly, a lack of improvement in symptoms can lead patients to discontinue treatment too soon.

“An awareness of post-hoc bias will not make it disappear, however we suggest patients and clinicians need to think twice and stay mindful of alternative explanations,” Redelmeier adds.

The study was supported by the Alfred P. Sloan Foundation, a Canada Research Chair in Medical Decision Sciences, Canadian Institutes of Health Research, the PSI Foundation of Ontario, and the National Science Foundation. Donald Redelmeier is also a senior scientist at ICES.