“You don’t need a randomised trial to prove that parachutes
work” is a common refrain from doctors who believe that their operation works,
despite a lack of good scientific evidence from controlled trials. They see it
as an argument-stopper, but it is actually an example of argument by analogy,
and a poor one at that. And at least one surgeon has been burned by putting the
parachute analogy in writing.
Arguing by analogy is common, but it can be overdone. The
strength of the argument lies in the similarities between the two situations,
and there are very few similarities between jumping out of a plane with or
without a parachute, and surgery. Unless the risk of death without surgery is
immediate and high (about 100%), and the risk of death with surgery is very low (less than 1%), then yes, you don’t
need a randomised trial to convince me, just prove those numbers to me and you
have won the argument.
A better analogy would involve pointing out that every
surgical procedure that has been compared to sham surgery has been shown to be
ineffective, and that treatments subjected to randomised trails usually show
much weaker benefits than previously thought from less rigorous studies (see
these, previous, posts, here, and here).
The example I will use comes from Dr J Cooper, a thoracic
surgeon who in 1995 published
good results in a series of patients who had undergone a new operation for
emphysema, which involved removing 20-30% of each lung (LVRS: lung volume
reduction surgery).
The operation became popular and was being done fairly widely, but due to doubts (from others) about the effectiveness a large randomised trial to compare
LVRS to medical therapy (the NETT trial) was proposed, and funded by Medicare,
the National Institute of Health and the Agency for Healthcare Research and
Quality.
Dr Cooper
then pulled out the parachute analogy in an editorial
from 2001 in which he sees the evidence of benefit from LVRS as overwhelming,
and rejects the need for the NETT trial.
In fact, Dr Cooper was still publishing great results from (uncontrolled)
case series in 2003 when
the results of the NETT trial were published in the
New England Journal of Medicine. There was no difference in the rate of death
between the two groups at two years. The topic has been covered in a Cochrane
review from 2009 where they also note that there is no difference in deaths
at 2 years. There is an improvement in lung function and quality of life over
that period, but this can be partly explained by the fact that surgery tends to
weed out the sicker patients early: the death rate by two years might be the
same, but your odds of dying within 90 days is more than 6 times higher if you
have surgery.
The arguments persist, mainly looking at what patients might
benefit most and what patients do worse, in order to refine the indications for
surgery. It sounds a bit like Desperately Seeking Subgroups (an upcoming post
on this site) but to be fair, that is the discussion we should be having – we
should not be attempting to shut down the debate by using inappropriate arguments
by analogy.
Addit: 1 July 2012: See the Healthcare etc. blog post on the parachute analogy here
Addit: 1 July 2012: See the Healthcare etc. blog post on the parachute analogy here
Has the parachute analogy been used in the defence of other surgical procedures without an evidence based?
ReplyDeleteThe line of defence adopted in the article by Dr Cooper seems almost ridiculous after the publication of the NETT trial. "One cannot argue against evidence-based medicine, but the nature of the evidence and the manner in which it is obtained is highly relevant to surgical innovation. Randomized clinical trials are appropriate for some but not all circumstances. Furthermore, participation in clinical trials, be they randomized or otherwise, can be a potential source of conflict between the interest of the investigator and the interest of the subjects."
Thanks Abhinav,
DeleteThis is the only published case of using the parachute analogy of which I am aware. However, in conversations and discussions, it gets used regularly. The reason I wrote the post was because of how often I hear it invoked.
John Cunningham wrote a humorous article on it in the ANZ J Surg (http://onlinelibrary.wiley.com/doi/10.1111/j.1445-2197.2010.05638.x/full), but the impression I got from that was that he was making fun of epidemiologists rather than using it in defence of any particular procedure.
Indeed I was Ian.
ReplyDeleteThe parachute analogy was merely a diving board to the argument that intention to treat analysis is not always appropriate for surgical interventions. There you go - an analogy about an analogy.
John
Would not needing to ask the research question: "is it better to stop or not to stop a major bleed occurring during a surgical procedure" be a fair parachute analogy, do you think?
ReplyDeleteYes, there may be situation where it is reasonable to use the parachute analogy, although I think that using the parachute analogy takes it outside of medicine and takes it to an extreme that is not truly comparable.
DeleteAs the Healthcare etc. blog points out, parachute analogy basically refers to a NNT of 1; the implication is that without it you die, and with it you live, something virtually never seen in medicine. Take your example of stopping bleeding, as it is the title of a blog I am currently working on. There are many examples of stopping bleeding that sound logical but do not translate into a change in mortality, so instead of a NNT of 1, you actually have a NNT closer to infinity. One example (as I don't want to spoil the upcoming blog): splenectomy was routinely performed for splenic laceration in trauma because the perceived mortality without surgery was 90%. Now, 90% of splenic ruptures are treated non-operatively because it is not necessary, and there are long term and short term complications associated with the procedure.
Stay tuned.