…please read and understand the study first. Don’t embarrass yourself by just commentting based on the what you think the title of the study means.
For example, this recent study was published. The study investigated outcomes of clubfoot treatment to see if immediate or delayed treatment affected outcomes. Of the 176 cases they reviewed, the age at presentation did not affect the outcome (except for the issue of cast slippage).
I shared this study on my social media platforms and so did many others. It was surprising the number of comments in response that the treatment for clubfoot must begin as soon as possible, which is not what the study showed. I can only assume that the commenters are responding to the title of the study and not actually bothering to read it, let alone understand it. Fail.
Reflexology is just made up mythology. There is no known physiological link between parts of the foot and organ systems in the body, let alone any involvement in these disease processes. Every single meta-analysis and systematic review of all the clinical trials of it have concluded the same thing: it does not work. It is no better than a placebo. The only clinical trials that show it works are in low quality, low or no impact factor journals and have serious methodological flaws. The most common methodological flaw is the lack of a control group. Most of those studies were not even on reflexology, but were on nothing more than a damn good foot massage. Everyone, regardless of what medical condition they have is probably going to feel better after a damn good foot massage!
Over on Podiatry Arena, there is a thread: Reflexology is not an effective treatment for any medical condition that started over 7 years ago and continues to be added to with more research and reading it you can see the obvious issues with the overall poor quality of research on reflexology that claims it works. None of the studies that are of high quality show that it works. That pattern is so typical and so familiar to anyone who follows the pseudosciences.
One thing that i did start to notice in that thread and commented on several times is that the number of times that a clinical trial on reflexology (and you see the same pattern in other alternative and complementary medicine studies) is the authors claim that they used randomization of the subjects to the different groups, yet they end up with the exact same number in each group. Randomization is the cornerstone of the randomized controlled trial as it is how you ensure that the subjects in each group are nearly identical in characteristics.
When you randomize subjects to different groups, you almost never get the same number in each group. Very occasionally you do (by random chance). Just pick up any selection of high quality journals and look at the randomized trials in them and you will almost never see one in which the numbers in each group are the same.
Now have a look at that thread on Podiatry Arena (all 6 pages of it). Its is amazing how many clinical trials that claimed that they used randomization end up with the exact same number in each group. You see the same pattern in a lot of clinical trials on alternative medicine topics. One can only conclude that the researchers did not do what they claim they did. They did not randomize them and used some other method of allocation to the groups, which further undermines the credibility of clinical trials on reflexology. Fail.
Restless legs syndrome is a common problem, too often without a satisfactory solution. Because of this, there is plenty of bad advice being given and many different treatments options available; many of which are underpinned by testimonials and anecdotes and with poor or no science or data. When extraordinary claims are made, then extraordinary evidence is needed to support the claims.
Recently a press release was put out for a study on a new product, The Restiffic foot wrap for use in restless legs syndrome. At last count, Google could find the press release carried on 27 websites. One thing all the websites had in common was that they just parroted the press release and demonstrated a total lack of critical thinking skills. I have already raised some issues with the press release elsewhere, but to reiterate:
1. The press release was based on an uncontrolled study in a low impact factor journal. The study did claim to have control group, when it did not. It did have a reference group from unrelated studies on a totally different population that they wrongly did a statistical analysis on. What the study really was is uncontrolled single arm analyzed with a within groups analysis. These types of uncontrolled studies tend to massively overestimate effect sizes and we have no idea if all the effect obtained was not due to placebo or natural history. The claims in the press release and claims made by the company are not supported by the evidence. A properly controlled study is needed to base the claims on.
The reason you need a control group with the same characteristics as the intervention group is that in the statistical analysis of the data you subtract the placebo effect of the control group from the treatment effect in the intervention group to get what is the real effect of the intervention. That is why uncontrolled studies massively overestimate effect sizes (like the above study). I know they claimed to have a “control” group, but they did not. They just used a reference group from a meta-analysis of other studies to compare their outcome to. The placebo effect in restless legs syndrome is strong, so some of the effect, or a large part of it, or all of the effects in this study in question could have been due to placebo. We will not know as they did not have a properly constituted control group.
2. The company behind the product have been claiming that the product is FDA approved, when it is not. The product has simply been cleared as safe by the FDA and to make the claims that it is approved is likely to get them in trouble with the FDA as their guidelines are clear. The company does appear to have backed way from these claims recently.
I have nothing against this product and have full empathy with those who have restless legs syndrome and the lengths they often go to, to get relief. I just do not like seeing them targeted in marketing by products that offer a “cure” based on no science or nonsense.
The Restiffic Foot Wrap may turn out to be an effective product and I hope it does for those who have restless legs syndrome. The company should be applauded for supporting the above study which a lot of restless legs syndrome approaches are not subjected to, but more is needed. I just object to it being marketed implying that it is FDA approved and the strength of the claims as to how effective it is based on an uncontrolled study.
Kuhn, P., Olson, D., & Sullivan, J. (2016). Targeted Pressure on Abductor Hallucis and Flexor Hallucis Brevis Muscles to Manage Moderate to Severe Primary Restless Legs Syndrome The Journal of the American Osteopathic Association, 116 (7) DOI: 10.7556/jaoa.2016.088
In my takedown analysis in the last post of two papers on morton’s neuroma, I pointed out that both studies were analysed wrong and that this should have been pulled up in the pre-publication peer review process and it wasn’t. Also, in my other blog I pulled up a number of studies on the same issue.
Understanding the difference between a ‘within group’ analysis vs a ‘between groups’ analysis can be confusing, so I put this video together describing them. Hopefully you can see wht the two studies reviewed here and the multiple studies reviewed here got it wrong.
The treatment group showed a significant difference before and after extracorporeal shockwave therapy (P < .0001). The sham group did not have a significant difference after 12 weeks (P = .1218).
and as such concluded
Conclusions: Extracorporeal shockwave therapy is a possible alternative to surgical excision for Morton’s neuroma
Given that impressively sounding title and the p values you could not help but be impressed, except that the way the results are stated in that quote above should have let off a huge alarm bell that the JAPMA editorial and peer review process had failed. The whole point of having a control group is that you do a statistical test comparing the outcomes between the intervention and control groups (ie a between groups analysis). What this study did was a within groups analysis which is not how you analyse a trial comparing two (or more) groups.
The study actually posted all the raw data in table, so I reanalyzed their data using the right analysis and published that as a letter to the editor of JAPMA. Others (here and here) also expressed the concerns about how the study was analysed. It turns out that the authors own data actually showed that shock wave therapy did not work for mortons neuroma!
JAPMA did not do the right ethical thing and retract the publication. They should have. They did not even link the abstract or paper on the website to the letters-to-the editor pointing out about the wrong analysis. They should have. If you read that paper and do not know about the issues of between groups vs within groups analysis, then without going quite a way out of your way to find the letters-to-the editor, you would believe that shock wave works for mortons neuroma. That is a problem.
These results suggest that ESWT may reduce pain in patients with Morton’s neuroma.
Guess what? JAPMA’s peer review and editorial processes let us down again
Guess what? The authors did a within groups analysis rather than the correct between groups analysis (the exact same wrong analysis as the 2009 paper above)
Guess what? They did not actually publish the mean and standard deviations of the outcome values in the paper (another failure of the editorial and peer review process to not get the authors to include that), but if you look at the outcomes in figure 3 (if you have access to the paper), notice that if you compare them between the two groups, they are about the same (you can only guess what the values are from the graph as they did not publish them).
Guess what? This study’s own data actually showed that shock wave therapy for mortons neuroma does not work, which is the opposite of what they conclude.
You think they would have learnt from the first one above, wouldn’t you?, but, no they repeated the same error again in the second one.
Am I going to write to the Journal another letter to the editor? No, can’t be bothered. The first example above shows that it does not work.
JAPMA should have retracted the first paper and now it should retract this second paper.
This is a serious ethical issue and JAPMA editorial staff would do no worse than appraise themselves on publication ethics. Anyone not familiar with the between groups vs within groups analysis will mistakenly believe the conclusions of these studies, when both studies show that shock wave therapy does not work for neuroma’s. What is the ethics of using shock wave therapy on neuromas based on these two papers that should not have made it through the editorial and peer review processes?
I have made no secret of my views on the editorial and peer review processes at JAPMA and have previously called them on another publication on my other blog on running. Guess what: one of the issues was the same as above, ie within group vs between group analysis. JAPMA’s Impact Factor has been languishing around 0.5 for a while now while all other foot and podiatry related journals have gone up. There is a reason for that.
Snake oil is a remedy that is sold for a condition that does not work any better than a placebo. How many remedies that are ‘sold’ for plantar fasciitis don’t work better than a placebo? There is probably a lot.
The biggest problem with plantar fasciitis is that the natural history of it is to generally get better on its own. Just look at the placebo groups in the clinical trials to see how many do get better on their own without treatment. The only reason to treat it is that is hurts and can take a while to get better on its own. The problem with any treatment given, the question is that did the treatment work or was the plantar fasciitis just get better on its own at the same time as the treatment was started. that is why we need to rely on good randomized prospective clinical trial to sort out the treatments that don’t work better than a placebo vs those that do work better than a placebo.
For this reason there are so many plantar fasciitis snake oil salesmen on the web. You can see plenty of eBooks and secret sauce treatments that “the doctors don’t know about” for $19.95. All they have supporting them are plenty of testimonials that you have no idea if they are real or not. There are certainly no clinical trials supporting them.
It gets really scary when you surf around the blogosphere and see how ‘pronation’ and ‘overpronation’ is being pontificated on by pseudo-experts and giving advice when they clearly have no idea what they are talking about. This is very common in the running community and is a flawed basis for the prescription of running shoes. There is so much of what is written about it that is pseudoscience, yet is accepted as fact by those who are not familiar with the whole body of evidence on the topic. There is a lot of cherry picking of flawed studies to make a point and and ignoring of what the more powerful meta-analyses and and systematic reviews are showing.
The most recent example of this is the recent study that came with a press release that was widely picked up on running websites and forums about ‘busting the pronation myth‘ in which the authors claimed to have shown that pronation was not a risk factor for injury in runners. Those with agendas widely trumpeted that press release and running websites parroted from that press release. There was very little critical appraisal of the actual research that underpinned the press release.
Basically the study was a prospective risk factor study. They used the Foot Posture Index (FPI) and found that those runners with a higher FPI did not get more overuse injuries. BUT if you actually read the paper:
1. They eliminated all those wearing foot orthotics from the study. The assumption would be that if they are wearing foot orthotics, then they are probably a pronator at risk for injury. So if you eliminate all those pronators who are at risk for and injury, you are left with the pronators who are not at risk for an injury! So, of course they will then find that ‘pronation’ is not a risk factor! I can’t believe how dumb this was to do and how all those parroting the study did not see this as an issue.
2. They also used an FPI of 7 as the cut off point of the neutral vs pronated category. While this is based on normalized data, there would hardly be any clinician familiar with the FPI that would accept that. This means that most of the feet in their ‘neutral’ group were probably pronated!
3. Another study came out around the same time that showed the exact opposite of this study. Why did those parroting the above study not mention this one? This was cherry picking at its worse.
At the end of the day, there is more than enough studies on ‘overpronation’ and injury risk to combine all the data from all the good ones into a meta-analysis and systematic review. The most recent one to do that concludes that ‘overpronation’ is a small, but statistically significant risk factor for injury. This data is much more powerful than individual studies and should be the one that we focus on to avoid the use of cherry picking one study or another to promote an agenda.