A cherry picking tale, and an update on spinal manipulation science

Spinal manipulation: underwhelming effects on back pain since 1895. (Illustration by Gary Lyons)

Spinal manipulative therapy (SMT) has long enjoyed a solid reputation for the treatment of so-called “mechanical” low back pain, and is widely used by physical therapists as well as chiropractors. Even skeptics tend to give it a pass, regarding it as the one thing chiropractors offer that we don’t really have a complaint with, much the way we used to think maybe acupuncture had something going for it, until better evidence crushed our optimism. For instance, just recently Dr. David Gorski — as tough-minded a quackery critic as they come — casually mentioned that “no one argues that … manipulative therapies can’t help back pain.” Actually, skeptics who’ve been following the SMT science are now arguing exactly that.

Recently in a Facebook discussion, a chiropractor answered just such a skeptical critic (not me) like so:

“I do not have a pile of citations to give you, however if you dig a little, you will find research to support spinal manipulative therapy.”

Yes, unfortunately, you can always “find research to support” anything you like, and I commented to that effect. Indeed, that is exactly how I used to work, before I knew better. Probably until at least 2005, I was still much more of a massage therapist than a science writer. (I quit massage more than a year ago.) I thought like a therapist, had therapist thoughts, and therapist theories. I would bring those ideas to my desk and look for scientific papers in PubMed that backed me up — pretty much the definition of “cherry picking” — and then (groan) fancy myself to be quite the smarty pants. I really hadn’t the faintest clue that scientific papers could be so incredibly misleading, or how trivial a single experimental result was — the sound of one hand clapping.

Ironically — and fortunately for my writing career — although my efforts were certainly amateurish, I was already doing much more research than most massage therapists ever do. Most are too busy and/or lack the inclination to “go there,” don’t know what PubMed is (vast database of medical science), and have no idea what a p-value is (learn it from stick people) or what regression to the mean means.

Probably thanks mostly to Carl Sagan (“What would Carl Sagan do?”) I actually noticed when I awkwardly encountered evidence that clashed with my hypothesis. Well-intentioned and naive cherry picking isn’t always a total disaster. If you do it with a shred or two of integrity. You might actually pick some good cherries, and you might care about it when you find one that tastes funny. I started to give less weight to little studies when I noticed that they were often contradicted by better ones. I began to prefer the findings of more credible reviews and larger RCTs.

Like this one …

The latest and greatest in evidence about spinal manipulative therapy

BMJ Clinical Evidence — an excellent organization that publishes EBM reviews (much like the World HQ for EBM, The Cochrane Collaboration) — updated their acute low back pain review recently (sorry, link behind paywall — a clever Google search may give you a link that will work from the search results). Here’s what they had to say about spinal manipulative therapy:

One systematic review (149 people) and one subsequent RCT (101 people) added at this update. The review and RCT found no significant difference between spinal manipulation and placebo or usual care in pain. One further study added in harms which reports on adverse effects after spinal manipulation. Categorization of spinal manipulation changed from ‘Likely to be beneficial’ to ‘Unknown effectiveness.’

This is nothing new to anyone who has actually been following the SMT evidence: scientific optimism about spinal manipulative therapy has been steadily eroding for a decade. There was never any evidence of serious efficacy (it always failed the “impress me” test), and larger and better experiments have conclusively shown that it is either completely ineffective, or so unreliable and trivially effective on average that it really just doesn’t deserve to be an expensive therapy with a good reputation any more. It just isn’t all it’s cracked up to be.

When presented with that evidence, the chiropractor threw my words back in my face: “Just like you said, you can always find research to support anything you like.” Oh, the irony!

What I meant, of course, is that you can always find crappy, old, tiny studies in lame journals with results that (at least superficially) seem to support your point, which is specifically why I cited a large high quality meta-analysis in a great journal — because it’s much harder to find one of those that supports your pet theory (not impossible, but much more difficult).

This logic was obviously completely lost on him. He also decried the “fairly small sample size” of 149, having apparently read only the first five words of the abstract and failing to realize that the number referred to an addition to a much larger pool of over 1000 subjects — a complete failure to comprehend what any high school junior could have gotten from the abstract.

Well done, chiropractor sir! Science sniffed suspiciously, totally misunderstood, and dismissed! Ideological mission accomplished. This is truly cherry picking blindness at its finest.

A sadly representative sample

This kind of low-quality response, heavily filtered through a thick layer of ideological bias, has been distressingly typical of my extensive experience debating chiropractors over the years, both public and private. And this fellow was a good deal more polite and rational-sounding than most: he at least paid lip service to the value of science, even if he had little working knowledge of it. Sadly, many chiropractors openly attack science itself, as though science is personally to blame for bad hospital food and arrogant doctors.

The last time I went to a chiropractor for back pain — quite a few years ago now — it was something like $90 out of pocket for the first visit, and then $45 for each subsequent visit, and I also had to assertively decline the on-going $500+/year commitment that she “strongly recommended” — so strongly that she felt it was necessary to try to frighten me with some tough talk about how careless and perilous it would be not to take proper care of my back, like neglecting oral hygiene.

And yet now the science is very clear that, in all likelihood, the benefit I seemed to get from the treatment at the time — modest — was largely nonspecific effects and placebo, and would have been matched by virtually any other treatment I felt optimistic about and paid good money for.

Funny story for balance: another chiropractor I saw in the same period refused to treat me, stating that the evidence did not support spinal manipulation in my case — the same case that need long-term SMT according to the other chiropractor. That was actually the first time I’d ever encountered the idea that the evidence might not support the main treatment offered by chiropractors … and I heard it from a chiropractor, an insider, and not a skeptic in a pub. After that I investigated, found, and read a book largely critical of chiropractic (highly recommended), also by a chiropractor, Sam Homola. So I was actually introduced to chiropractic criticism by a couple of chiropractors. Oh, the irony again.

If you’re keen to learn more about SMT, a good source is my book about low back pain, which includes an extremely detailed and readable review of the science and concepts of spinal manipulative therapy, along with many other low back pain therapies and treatments: the good, the bad, and the ugly.

Post banner image courtesy of Photo Extremist via Flickr under Creative Commons

8 Responses to “A cherry picking tale, and an update on spinal manipulation science”

  1. locklin says:

    Man is this a tough paper to get my hands on. Great article, but it would have been nice to have a citation. For anyone else looking for this paper the PubmedID is 21549023.

  2. Thanks, Locklin, much obliged. I was unable to find a PubMed record at the time of publication. I think it probably did not exist yet (note that the PubMed record describe it as a 2011 paper and the PubvMed record is “in process.”) That said, the link to the source was valid — it’s just that the content was behind a paywall, which is an unfortunate but routine limitation that an author can do nothing about.

    Of course, in the olden days it was impossible to verify most citations without extraordinary efforts!

    • locklin says:

      Unfortunately, if you don’t have access, that pay wall link doesn’t even give you citation information, not even the title! What happened to giving the title/abstract for free eh? jees’ what money grubbers. Unfortunately my University doesn’t have an up-to-date subscription for Clin Evid. The CBI Health Group puts pdf’s of many of it’s papers on it’s website (including the 2007 version of this paper). Hopefully they will add this paper at some point.
      http://www.cbi.ca/web/cbi-health-group/about-cbi/cbi-resource-centre/research

  3. Ethan says:

    Some libraries and universities have access to those online journals, if PubMed doesn’t get it soon.

Trackbacks/Pingbacks

  1. [...] imagine my surprise when I read today about a study that shatters what I thought I knew about SMT, or Spinal Manipulation Therapy, the fancy name for chiropractic. Now, admittedly I haven’t [...]

  2. [...] them to crack your spine because it hurts will result in a resounding “Yes!” in most (but not all) [...]

  3. [...] A cherry picking tale, and an update on spinal manipulation science « Alternative Medicine &la…Spinal manipulative therapy (SMT) has long enjoyed a solid reputation for the treatment of so-called “mechanical” low back pain, and is widely used by physical therapists as well as chiropractors. Even skeptics tend to give it a pass, regarding it as the one thing chiropractors offer that we don’t really have a complaint with, much the way we used to think maybe acupuncture had something going for it, until better evidence crushed our optimism. For instance, just recently Dr. David Gorski — as tough-minded a quackery critic as they come — casually mentioned that “no one argues that … manipulative therapies can’t help back pain.” Actually, skeptics who’ve been following the SMT science are now arguing exactly that. [...]


  • Paul Ingraham

    Paul Ingraham is a former Vancouver massage therapist who quit his job in alternative medicine due to frustrations with anti-scientific attitudes and practices, which led to a legal scuffle. He now makes his living writing about science-based treatment for common injuries and pain problems, and has published hundreds of articles and several ebooks at SaveYourself.ca (see skeptics reading guide). He is also an editor for ScienceBasedMedicine.org.