Recently, while debating a respected colleague regarding a
shift in practice towards treatment X, despite a lack of
evidence showing its superiority, my colleague said: “But we know that practice
always runs ahead of the evidence”. He was implying that the evidence would one
day catch up and justify the practice. I wondered if medical practice really was
running ahead of the evidence, or whether it was running away from it.
When clinical practice does run away from the evidence, we
tend to spend our time gathering evidence to support the current practice,
instead of using an objective evaluation of the current evidence to inform
future practice. This is known as putting the cart before the horse. It is also
known as Confirmation Bias.
Confirmation bias
Like the ghosts in The Sixth Sense, many clinicians and
researchers only see what they want to see. I am not referring to a conscious process, where a deliberately biased argument is created (case-building) such as in a debate. In medical practice it is usually subconscious, where the selection of patients, measurement tools, outcomes and time-points tends to favour the beliefs of the researcher, and the researcher’s biases are then confirmed.
Even if the evidence is not clearly supportive, some may interpret the
results as confirming their pre-conceived beliefs. This is also confirmation
bias.
Confirmation bias is seeking, interpreting or recalling evidence
in a way that is favourable to existing beliefs, expectations or hypotheses.
And it is one of the most problematic aspects of human reasoning.
The bottom line
Many clinicians are running their practice on confirmation
bias, or less. I think trends (read: fashions) in current practice should slow
down and wait until the evidence catches up. The two can then travel
together, preferably with the evidence doing the driving.
For a very long and boring paper on confirmation bias, read this
article.
I mostly agree. On the other hand, historically thinking,evidence is closer to medical practise now, perhaps more than ever.
ReplyDeleteThink of medical practice at the 1800's, 1900 's even 1950's. How much of it was evidence based? Most of it was based on empiricism, confirmation bias or authenticity. I think we made a huge progress from that point by adopting evidence-based studies. That's how the thing goes: one thinks of a new treatment as suitable (proof of principle) applies it to some patients, gets some positive feedback (empiricism/confirmation bias) and when the meme spreads some researchers subject it to clinical trials. Although that doesn't seem the right way, waiting the ideas to come up only by people in research centers would probably slow down progress and innovation.
However, I understand your considerations especially when I look of how many people (patients and doctors) turn into "alternative" therapies that not only are unproven but even lacking proof of principle (i.e homeopathy).
Thanks, your take on medical history is correct, but many do not realise that aspect of the history of medicine, and are repeating the past mistakes.
DeleteGenerating hypotheses and then testing them, is what it is all about, and many of the hypotheses come out fo what we see (experience). But we often miss the 'testing' step and accept hypotheses as correct, based on experience.
We are certainly doing better than we were, but that doesn't mean there aren't some big holes in the evidence for our current practice.
I think moriakos hit that nail on the head. It is all about prior plausibility and basic science (i.e. deeper model). Apply the test treatment with the explanation rooted in what is plausible based on science.
ReplyDeleteI think a big problem we are running into (recently addressed on SBM) is that the "evidence-base" is outrunning the science-base. Too much focus placed on practical research and not enough on learning more of the basic science.
Thanks Nick, but regarding the balance between basic science and clinical research, I often argue the opposite: that basic science tends to get the lion's share of funding as it is seen as more 'scientific'.
DeleteClinical research, however, provides information on what really works. Based on lab studies, I can justify almost any treatment you can think of that has been tried in the past. Clinical research tells us whether or not it works, and often shows us that it doesn't. Clinicians often skip this step, and base their practice only on the lab evidence and their own prior probabilities (based on their own confirmation-biased experience).
I agree there is a balance needed, but the reason I brought up that point is because of all the 'alt-med' evidence (acupuncture, etc) that has really no scientific plausibility, but a lot of 'evidence.'
ReplyDeleteI think what you say about clinicians is true, but are they keeping up on the literature, is there literature to support their treatment, and are they keeping patient's informed of the extent of our knowledge regarding that treatment?
Yes, in general you are right. Alt med does practice as you say: without plausibility and with a lot of 'evidence', but not with a lot of GOOD evidence, i.e. comparative/placebo clinical trials.
DeleteYour contrast between alt med and conventional medicine is true, but I guess my point is that just because something is coming from conventional medicine, and the doctors thinks it works, it does not mean that there is good clinical evidence behind it, even though it might have biological plausibility.
In my opinion, biological plausibility is a requirement; it does not provide evidence of effectiveness on its own.
Sorry, I am in agreement with you on evidence.
ReplyDeleteMy question is: what do you think is contributing to the problem you stated (cart before the horse)?
Putting the cart before the horse is a product of confirmation bias, and other biases. The solution to biased thinking and biased practice is to remove or minimise the bias. This is done by the use of scientific methods, which specifically work to reduce bias and lead us to a more reliable and accurate assessment of the truth.
DeleteThat sounds simple, but the problem with implementing that is clinicians often have a poor understanding of biases and scientific methods - of how to search for, collect and assess available evidence based on its scientific quality. And they are humans so they fall back on what is easy and what 'sounds' right: what they see and what others tell them, and tradition and fashion.
Clinicians are often not up to date with the literature, and when they read the literature, they often do not have the critical appraisal skills to be able to assess the quality of a single study, let alone compare all the available evidence.
This is not an exaggeration or a cheap attack at my colleagues. It is a system-based problem. I spend a considerable amount of time trying to learn critical thinking and the scientific method myself, but I cannot keep up to date with the literature and do not have time to properly appraise all of the treatments I use.
The easiest thing for me to do is to practice medicine the way everybody else does, without actively seeking, assessing or appraising the scientific evidence, and operating on patients when things are not clear. If I do that, I will not get sued or criticised and I will be well paid.
Bottom line: it is the path of least resistance.
So do you think there needs to be more critical thinking/appraisal type classes in pre-med and/or med school?
ReplyDeleteYes, and I would go even further. It needs to be part of any science-based course, and should be a part of general knowledge provided in school.
DeleteUniversities pay lip service to this. They rarely offer courses in critical thinking, but are happy to run courses on alternative medicine.
I received little or no training in this at university.
I'm unsure of the culture of college/universities in Australia (I believe that is where you are from because of the .au), but here in the USA, it seems to slowly getting away from actual education (well not about science and the other goodies supposed to be taught in college).
ReplyDeleteUniversities are hell bent on providing lavish dorms, diverse atmospheres, clubs upon clubs, the best sports to watch, and spa/health centers to attract the customer, I mean student, to put their loan or Mom + Dad's money down.
When it became big business the educational institutions became less about education and more about attraction.
Then high school, boy oh boy. Teaching to the test has seriously destroyed any critical thinking en masse. Some lucky few (like me) will have parents that send them to private institutions (a pretty penny mind you). Not that this is a guarantee that we will get a solid critical thinking education.
Because here is the rub, critical thinking and education are one in the same. We teach the basics and the how's/why's so the kids grow up thinking, trying, failing, trying again, etc and learn to become critical appraisers with life in general. Well that is the hope, but reread my post to see the state we are in.