‘More junk getting through to the keeper*’

The whole idea of the peer review process prior to publication is to weed out the junk, so it does not get published. One thing that the alternative therapies have in common is that their journals let a lot of junk science through. Too many studies get published in those journals that should never see the light of day, let alone been conducted so badly in the first place. There are ethical issues at stake in this and the editors of those journals would do well to apprise themselves of publication ethics. Institutional ethics committees or review boards also have a responsibility to prevent bad science from even getting off the ground.

What spurred that little rant was this publication today on ‘The effect of reflexology on the quality of life with breast cancer patients‘ published in the journal, Complementary Therapies in Clinical Practice. They do not get much worse than this one.

It was a study that supposedly randomized 60 people with breast cancer into two groups; one group the control and one group getting reflexology; the aim being to see how it affected their quality of life and symptoms. Sounds good on the surface, but:

They ended up with exactly 30 in each group and reading what they did, they did not randomise – even though they said they did! They just allocated them to different groups depending on the day of the week. The assumption of the statistical tests used are that proper randomization is used. Epic Fail. How can the reviewers and editor not see that randomisation did not take place? Regardless of the results of the study, stop there as without proper randomisation the results are meaningless and can not be trusted. As the study was approved by the “Ethical Committee of the Health Science Institute¬†of Ataturk University”, then that ethics committee needs to look at its decision-making process, as all the work that went into this study and the voluntary participation of the participants was wasted.

I have already blogged about reflexology studies almost always ending up with the exact same number in each group when they are supposed to be randomised and this study just confirms that problem.

And while we could stop there as the data can not be trusted, they then went on and did 38 within groups t-tests! Seriously? DId the authors, peer reviewers and editor of the journal not see an alarm bell go of with that?

  • it was a within groups analysis rather than a between groups analysis
  • do that many t-tests, just by chance you will get a statistically significant result
  • no hints of a Bonferroni correction because of the multiple tests.

The author’s conclusion of¬†Reflexology was found to reduce the symptoms experienced by breast cancer patients, while at the same time increasing the functional and general health status simply can not be supported by the data from this study … yet it still made it through to the keeper.

Having said that, a damn good foot massage will probably make anyone with a chronic illness feel better; it is that, that is NOT reflexology and that is NOT supported by the results of this study.

Please sign up for my newsletter when a new content is posted:

*for those not familiar with the metaphor: “through to the keeper”; it comes from cricket when the batsman does not even attempt to play at the ball and lets it pass through to the wicket keeper unchallenged.


My Online Courses

Time for some shameless self-promotion. I have been running my online Clinical Biomechanics Boot Camp since last year (you can enrol and start it at any time). It has been very popular and its kept me busy. Now, if I was not busy enough I have launched another two courses in the same format. One on Running Shoes (more here) launches next week and one on Critical Thinking and Skepticism (more here) launches tomorrow.

Here are the promotional videos on each one:

If you enroll before Monday 4th September in the Running Shoe one, there is a 20% discount. Use the code PRELAUNCH at checkout for the discount.

Please sign up for my newsletter when a new content is posted: