How Much of Orthodox Medicine Is Evidence Based?
This is an interesting letter from the British Medical Journal
Scientific heavyweights deplore the NHS money wasted on unproved and disproved treatments used by practitioners of complementary and alternative medicine (CAM), [1, 2] but Lewith, a CAM proponent (see previous letter), is cited elsewhere as saying that the BMJ reckons that 50% of the treatments used in general practice aren’t proved, and 5% are pretty harmful but still being used. 
His data were taken from the BMJ Clinical Evidence website. A pie chart (see it below in the Evidence-based Practice posting) indicates that, of about 2500 treatments supported by good evidence, only 15% of treatments were rated as beneficial, 22% as likely to be beneficial, 7% part beneficial and part harmful, 5% unlikely to be beneficial, 4% likely to be ineffective or harmful, and in the remaining 47% the effect of the treatment was “unknown.”
Read more on this topic
The text says, “The figures suggest that the research community has a large task ahead and that most decisions about treatments still rest on the individual judgements of clinicians and patients.” On 9 October 2007 the situation had changed—but not for the better. Treatments rated “beneficial” had decreased from 15% to 13%. The associated text is unchanged.
Acute low back pain is a common and well investigated condition. BMJ Best Treatments reports that back pain affects 70-85% of all adults, and each year almost half of us get back pain that lasts at least a day. There are 18 treatments for acute low back pain which have been tested by randomized controlled trials (RCTs), of which two (11%) were graded “beneficial” and 13 (72%) “unknown.”
The accompanying table shows all of the 18 treatments for acute low back pain and their rated effect. According to this table, a condition that is extremely common, and for which many treatments have been intensively researched, has an even higher than average proportion of treatments that are labelled “unknown” efficacy, or in other places “need further study.” There must be some mistake.
The solution to the mystery is that the label “unknown” does not mean, “We have no knowledge of the effect of this treatment because it has not been tested in an RCT.”
Astonishingly, it means, “We have tested this treatment in several RCTs, but on balance there is currently no convincing evidence that it is effective for this condition.”
So, the actual efficacy of these 13 other “MEDICAL” treatments for acute back pain are NOT “unknown”… they have just never been demonstrated” in any of the trials that have been conducted. That is vastly different than what this Pie Chart suggests!
Lewith’s interpretation of the pie chart is highly misleading. The research community has been commendably diligent, but of course RCTs often fail to find that certain treatments are effective.
Euphemisms such as “unknown” or “needs more study” for the inefficacy of such treatments may soothe the feelings of proponents of those treatments that have so far failed to show efficacy, but it does an injustice to the researchers who obtained these data, and misleads both practitioners and patients about the extent to which orthodox medicine is evidence based.
John S Garrow vice chairman, HealthWatch, The Dial House, Rickmansworth WD3 7DQ email@example.com
Competing interests: None declared.
- Kamerow D.
Wham, bang, thank you CAM.
BMJ 2007;335:647. (29 September.)
- Colquhoun D.
What to do about CAM?
BMJ 2007;335:736. (13 October.)
- Cope J.
The great debate.
Healthwriter 2007 Apr:1-3