Sunday, 1 July 2012

The parachute analogy


“You don’t need a randomised trial to prove that parachutes work” is a common refrain from doctors who believe that their operation works, despite a lack of good scientific evidence from controlled trials. They see it as an argument-stopper, but it is actually an example of argument by analogy, and a poor one at that. And at least one surgeon has been burned by putting the parachute analogy in writing.

 Arguing by analogy is common, but it can be overdone. The strength of the argument lies in the similarities between the two situations, and there are very few similarities between jumping out of a plane with or without a parachute, and surgery. Unless the risk of death without surgery is immediate and high (about 100%), and the risk of death with surgery is very low (less than 1%), then yes, you don’t need a randomised trial to convince me, just prove those numbers to me and you have won the argument.

A better analogy would involve pointing out that every surgical procedure that has been compared to sham surgery has been shown to be ineffective, and that treatments subjected to randomised trails usually show much weaker benefits than previously thought from less rigorous studies (see these, previous, posts, here, and here).

The example I will use comes from Dr J Cooper, a thoracic surgeon who in 1995 published good results in a series of patients who had undergone a new operation for emphysema, which involved removing 20-30% of each lung (LVRS: lung volume reduction surgery).

The operation became popular and was being done fairly widely, but due to doubts (from others) about the effectiveness a large randomised trial to compare LVRS to medical therapy (the NETT trial) was proposed, and funded by Medicare, the National Institute of Health and the Agency for Healthcare Research and Quality.

Dr Cooper then pulled out the parachute analogy in an editorial from 2001 in which he sees the evidence of benefit from LVRS as overwhelming, and rejects the need for the NETT trial.
In fact, Dr Cooper was still publishing great results from (uncontrolled) case series in 2003 when the results of the NETT trial were published in the New England Journal of Medicine. There was no difference in the rate of death between the two groups at two years. The topic has been covered in a Cochrane review from 2009 where they also note that there is no difference in deaths at 2 years. There is an improvement in lung function and quality of life over that period, but this can be partly explained by the fact that surgery tends to weed out the sicker patients early: the death rate by two years might be the same, but your odds of dying within 90 days is more than 6 times higher if you have surgery.

The arguments persist, mainly looking at what patients might benefit most and what patients do worse, in order to refine the indications for surgery. It sounds a bit like Desperately Seeking Subgroups (an upcoming post on this site) but to be fair, that is the discussion we should be having – we should not be attempting to shut down the debate by using inappropriate arguments by analogy.

Addit: 1 July 2012: See the Healthcare etc. blog post on the parachute analogy here

5 comments:

  1. Has the parachute analogy been used in the defence of other surgical procedures without an evidence based?

    The line of defence adopted in the article by Dr Cooper seems almost ridiculous after the publication of the NETT trial. "One cannot argue against evidence-based medicine, but the nature of the evidence and the manner in which it is obtained is highly relevant to surgical innovation. Randomized clinical trials are appropriate for some but not all circumstances. Furthermore, participation in clinical trials, be they randomized or otherwise, can be a potential source of conflict between the interest of the investigator and the interest of the subjects."

    ReplyDelete
    Replies
    1. Thanks Abhinav,
      This is the only published case of using the parachute analogy of which I am aware. However, in conversations and discussions, it gets used regularly. The reason I wrote the post was because of how often I hear it invoked.
      John Cunningham wrote a humorous article on it in the ANZ J Surg (http://onlinelibrary.wiley.com/doi/10.1111/j.1445-2197.2010.05638.x/full), but the impression I got from that was that he was making fun of epidemiologists rather than using it in defence of any particular procedure.

      Delete
  2. Indeed I was Ian.
    The parachute analogy was merely a diving board to the argument that intention to treat analysis is not always appropriate for surgical interventions. There you go - an analogy about an analogy.
    John

    ReplyDelete
  3. Would not needing to ask the research question: "is it better to stop or not to stop a major bleed occurring during a surgical procedure" be a fair parachute analogy, do you think?

    ReplyDelete
    Replies
    1. Yes, there may be situation where it is reasonable to use the parachute analogy, although I think that using the parachute analogy takes it outside of medicine and takes it to an extreme that is not truly comparable.
      As the Healthcare etc. blog points out, parachute analogy basically refers to a NNT of 1; the implication is that without it you die, and with it you live, something virtually never seen in medicine. Take your example of stopping bleeding, as it is the title of a blog I am currently working on. There are many examples of stopping bleeding that sound logical but do not translate into a change in mortality, so instead of a NNT of 1, you actually have a NNT closer to infinity. One example (as I don't want to spoil the upcoming blog): splenectomy was routinely performed for splenic laceration in trauma because the perceived mortality without surgery was 90%. Now, 90% of splenic ruptures are treated non-operatively because it is not necessary, and there are long term and short term complications associated with the procedure.
      Stay tuned.

      Delete