The favourable results of treatments can fade over time (the Decline Effect, see later post) for many reasons. Often the initial enthusiasm (bias) of the proponents is not replicated in later studies (see previous blog). But sometimes it can be shown that as the scientific validity of the studies improve, the effect decreases. This is because studies with better scientific methods will (by definition) have less bias (causes of error) and therefore the results will provide a better estimation of the truth. Vertebroplasty (injecting osteoporotic vertebral fractures with cement) is a recent (and controversial) case in point.
The decline in effectiveness of this procedure was documented in an editorial in The Spine Journal (The vertebroplasty affair: the mysterious case of the disappearing effect size). Initial reports of vertebroplasty, without controls (comparison patients) were very positive. Some comparative studies were also positive (but less so). There was still some apparent benefit when some non-placebo randomised clinical trials (RCTs) were done. But by the time the blinded placebo-controlled RCTs were done (here and here), the effect was gone.
The RCTs were scrutinised in a published report* by the North American Spine Society, and so they should be, as there is no such thing as a perfect study. And that is what science is all about: putting your findings out there for scrutiny. The authors addressed the criticisms, and even went to the extent of publishing a systematic review in the British Medical Journal, combining their data in order to answer the criticism about not having enough acute (fresh) fractures.
It is disappointing that many are dismissive of the RCTs, but are less critical of the unblinded, uncontrolled or non-placebo controlled studies. Unfortunately some people trust their eyes (seeing their patients getting better after giving the treatment) and make the logical error of post hoc ergo propter hoc (after this, therefore because of this). This human trait is a common logical error when assessing treatments for conditions that are self-limiting (get better anyway), like vertebral fractures.
My answer to the promoters of vertebroplasty is to show me the randomised trials that demonstrate a benefit. And if they finally produce one, we should scrutinise that paper, and not accept it on face value. To quote Rachelle Buchbinder, one of the lead authors of the two RCTs:
“Vertebroplasty appears to confer no benefit over placebo but poses some risk … The onus is on the proponents of the procedure to perform further high-quality randomized placebo-controlled trials”
And I didn’t even get time to report the complications associated with this procedure. If you are interested, click here for a glimpse.
*I realise that many of you may not have full access to these articles. If you don’t, I am sorry but I am not allowed to reproduce them for you.
When I was in residency, the radiology group at the hospital sent out a pie chhart of patient satisfaction scores to all of the doctors in the hospital group and residents. They did this after the study (ies) came out that sham = vertebroplasty. The pie chart was of patient satisfaction, which was 95%. I found it laughable.ReplyDelete
Healthcare reform isn't sexy, but eliminating all of this garbage/expensive/worthless "medicine" would be quite helpful. Unfortunately, people call that rationing.
Additionally, I'm confounded when we hammer no traditional medicine for not being EbM but don't do it to ourselves when we don't have the proper evidence. We just like it because we "see" the effects and "know" the mechanism of action (restoring vertebral height, lowering blood glucose as much as possible, etc) must work.
Thanks Michael, my sentiments exactly. There is a double standard in medicine that most practitioners seem blind to, because it fits their worldview, that they are right and anything non-traditional is wrong. They feel that medicine is right because it is BASED on science, not because particular treatments have necessarily been scientifically proven. Poor quality buildings can be constructed on good foundations. Even science fiction is BASED on science.Delete