By Colin Wright on October 29, 2011 | Discuss
This is a follow-up to my previous post investigating a study from the Mayo Clinic in collaboration with the University of Minnesota claiming that external qigong, a form of ancient Chinese medicine, is an effective treatment for chronic pain. My critique apparently got on the nerves of at least one person, Drew Hempel, qigong enthusiast and woo extraordinaire, who offered his assurance regarding the validity of the study and its methodology. Sadly, it’s not assurance that I am after—it’s evidence. However, maybe I was wrong; maybe the study was academically rigorous and its conclusions actually sound. After all, I am only an undergraduate (despite the fact that, in a recent blog post, Hempel incorrectly described me as a “university senior biologist”), and I admittedly only read the abstract.
Mr. Hempel has posted on internet blogs and forums statements such as the following:
Last fall there was a new study done by doctors from one of the top rated hospitals in the world — the Mayo Clinic in Minnesota. The study proved the existence and the efficacy of external qi (paranormal energy) healing transmission. . . O.K. I want to emphasize the implications of this study. This is ground-breaking official proof of something that undermines the very foundation of science.
Such extraordinary claims require even more extraordinary evidence, and Hempel believes, along with many, many others, that this evidence exists in a study performed at the Spring Forest Qigong center in Minnesota, published in The American Journal of Chinese Medicine.
After Hempel’s criticisms of my post and his request that I not “give up so easily” in my search for truth (I suggest Hempel do the same), I decided to check whether or not my university subscribed to the specific journal in order to obtain the full text of External Qigong for Chronic Pain (2010), the study that had supposedly demonstrated the efficacy of qigong. Much to my surprise, they do, and I found it. While reading the study, my initial criticisms based on the abstract alone became more and more cemented. I am now–more than ever–convinced that the study is absolutely bunk from the top down. The flaws are numerous, and I have included them below in point form, followed by a more in-depth criticism regarding the methodology behind each.
1. Flawed sampling method.
2. Lack of adequate controls.
3. Subjectivity in data collection.
4. Reliance on anecdotal evidence.