tag:blogger.com,1999:blog-6193043695356712843.post166341821234888748..comments2023-09-02T23:55:45.583+10:00Comments on Doctor Skeptic: The replication problemDr Skeptichttp://www.blogger.com/profile/09376469049519802493noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-6193043695356712843.post-28158473811091231172014-09-21T06:21:28.197+10:002014-09-21T06:21:28.197+10:00Thanks for the link. I like the term they used: Wi...Thanks for the link. I like the term they used: Wishful Science. I think that explains a lot of what we see in research and practice, where "wishful thinking" and science get a bit mixed up.Dr Skeptichttps://www.blogger.com/profile/09376469049519802493noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-41732051266345901412014-09-21T03:22:36.726+10:002014-09-21T03:22:36.726+10:00Excellent article! There was a recent NPR segment ...Excellent article! There was a recent NPR segment on the toll these sloppy studies take on patients in clinical trials. http://www.npr.org/blogs/health/2014/09/15/344084239/patients-vulnerable-when-cash-strapped-scientists-cut-corners<br />http://solowomenathomeandabroad.blogspot.com/https://www.blogger.com/profile/00116246897231606697noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-62627116594629019252014-09-19T02:01:39.825+10:002014-09-19T02:01:39.825+10:00Thanks and yes, sloppy science is everywhere, just...Thanks and yes, sloppy science is everywhere, just ask any decent economist. But that doesn't mean that science is bad, it is just done badly. And interpreted badly. There is still good science in medicine, our priority should be to improve the quality and to be able to detect the bad science.Dr Skeptichttps://www.blogger.com/profile/09376469049519802493noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-25256152715851131042014-09-19T02:01:23.399+10:002014-09-19T02:01:23.399+10:00Thanks and yes, sloppy science is everywhere, just...Thanks and yes, sloppy science is everywhere, just ask any decent economist. But that doesn't mean that science is bad, it is just done badly. And interpreted badly. There is still good science in medicine, our priority should be to improve the quality and to be able to detect the bad science.Dr Skeptichttps://www.blogger.com/profile/09376469049519802493noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-66396424762194181022014-09-18T23:56:15.267+10:002014-09-18T23:56:15.267+10:00As a layman this is why I have so little faith in ...As a layman this is why I have so little faith in medical practitioners. However, it should be noted that the sloppy science applies not only to medicine but to other areas of interest of daily life. It is almost a regression to belief in magic sometimes. X waved the wand once and it produced this, and this was funded by Y. Never a follow-up to X and his Y funded study. "It" become accepted protocal. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-62030990076326477882014-09-18T20:47:22.945+10:002014-09-18T20:47:22.945+10:00Thanks John and I agree: exploring why similar stu...Thanks John and I agree: exploring why similar studies show different results can be very revealing. The Nature studies were a little different as they tried to replicate lab studies precisely and it appears that, despite a public perception that lab research is somehow more 'pure', it appears that lab research is often a lot less rigorous than clinical research (less blinding, randomisation etc.), possibly because of the high methodological and ethical standards now required of clinical research. See my recent (June) post for a similar comment on animal research.<br />And I definitely don't want to give the impression that all research should be disregarded, only that a lot of it is biased, and that bias in research needs to be reduced (which, I believe it is, slowly) and to be detected and considered when reading research.Dr Skeptichttps://www.blogger.com/profile/09376469049519802493noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-17204901878604791492014-09-18T20:29:19.597+10:002014-09-18T20:29:19.597+10:00Only 16% of highly cited articles were contradicte...Only 16% of highly cited articles were contradicted later, as reported in the study by Ioannidis. Is that surprising, given the variability in populations, selection criteria, outcome measurements, etc that occur in clinical studies, even those which are considered identical by Ioannidis? It's hardly news. I'm surprised it's not more. One of the things about looking at these contradicting studies is often examining the subtle differences between these "identical" studies as way of determining if these differences are responsible for the outcomes. I often find it's more valuable looking at contradicting studies this way, rather than throwing up one's arms in the air and declaring that all research is flawed.John Cunninghamhttps://www.blogger.com/profile/09311583398118105706noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-7283723722531465512014-09-18T20:22:49.982+10:002014-09-18T20:22:49.982+10:00Excellent point. Most of the failure to reproduce/...Excellent point. Most of the failure to reproduce/replicate is for studies with positive/surprising results, like antibiotics curing back pain. I think those studies HAVE to be replicated. Also, in my own field, I find that 'negative' research is not taken up or accepted, and therefore a replication study could strengthen the evidence base and be more likely to effect a change in practice.<br />An example of what you are saying would be PRP injections, where the evidence is overwhelmingly negative, but people keep doing more research, and occasionally studies come up positive. I guess I have less of a problem with that (part of my No-such-thing-as-bad-data philosophy) except, as you say, that it is a waste of resources.Dr Skeptichttps://www.blogger.com/profile/09376469049519802493noreply@blogger.comtag:blogger.com,1999:blog-6193043695356712843.post-37900196156949003762014-09-18T20:08:54.042+10:002014-09-18T20:08:54.042+10:00What about too many reproducibility studies that l...What about too many reproducibility studies that lead to research waste? There are times when we do not need any more repeated studies. It most often occurs when people are trying to prove that something works when the previous studies have shown they don't. In this instance it is important to consider whether a repeated study is worthwhile and what it would add. One can look to systematic reviews and meta-analysis to determine how large a study would need to be to overturn the results of previous studies.Anonymousnoreply@blogger.com