List All News Items
Subscribe to FCER's The Week in Chiropractic
Donate to FCER
(Publisher's Note: A research
project reviewed by HealthMall.com deals with a review of RCTs on manual
therapy for asthma. The article may be seen at
following is a response from Dr. Anthony L. Rosner, FCER Director of
Research and Education.)
above-referenced article is no longer available from the
Response to Manual
Therapy for Asthma Review1
Hondras' recently published systematic review of
randomized clinical trials  addressed to manual therapy
represents a sincere effort to summarize those investigations in what is
commonly regarded as the gold standard of clinical research. That said,
however, one has to remain particularly vigilant against accepting
randomized clinical trials at face value, particularly in those instances
involving physical interventions, in which the complete blinding of
practitioners [and most likely patients as well] in the traditional RCT
design is all but impossible.
Consider the following pieces of evidence which suggest
that randomized clinical trials can be misinterpreted or even
One of the randomized
clinical trials regarding the use of chiropractic in managing asthma which
was published in The New England Journal of Medicine 
and most likely accepted as one of two trials examining chiropractic in
Hondra's study is fatally flawed by the following considerations,
presented in detail as an object lesson so as to more fully acquaint the
reader with the problems of properly designing and interpreting a
randomized clinical trial:
Lack of validity of
the sham procedure:
With over 20 commonly used techniques and 100 procedures
overall described for chiropractic, there is understandably a great deal
of controversy as to what constitutes a proper sham or mimic treatment.
Furthermore, with applications to no less than three regions of the
patient having been described in the Balon study [gluteal, scapular, and
cranial], there is a high probability that the sham procedure is invasive
and overlaps to a large extent with the maneuvers chosen with the actual
manipulation. This suspicion is strongly supported by a recently published
clinical trial in a leading pediatrics journal to the effect that massage
compared to a noncontact placebo produces significant improvements in lung
functional tests, asthma symptoms, and stress indicators in two separate
cohorts of children. 
The problem of sham procedures in the Balon study is
compounded by the fact that nearly a dozen chiropractors had to be trained
to perform such a procedure with no indication of standardization. The
effect of all this is to minimize or obscure the therapeutic effect that
might be observed in an actual adjustment.
Masking of possible effects by
The fact that all patients have been medicated may be
necessary from an ethical point of view, but it would be expected to mask
the beneficial effects that might have been observed from spinal
manipulation. The reader must be cognizant of the fact that this trial
reports little or no benefits in addition to standard
Vagueness of interaction with the
The nature of personal interaction with the patient is
ill-defined at best, dubious at worst No indication is given as to how the
practitioner such as might be seen in the clinic interacts with the
patient except to administer a satisfaction questionnaire. This leads to
the additional intrigue as to how eligible patients as young as 7 years of
age are to competently answer such questions as those pertaining to
"feeling at ease, the skill and the ability of the chiropractor, and
overall quality of care" that were administered in the
Improvements over baseline values
in both treatments:
The fact that there was significant improvement by
intervening with the patients is demonstrated by the declines at 2 months
and 4 months of both daytime symptom scores and the number of puffs per
day of a beta-agnonist, in addition to small increases of peak expiratory
flow rates and pediatric quality of life scores in both groups. Such is to
suggest that even in this trial there was significant improvement in the
patients enrolled. What is not clear is which form(s) of intervention
[global and/or manual] elicited responses. What is not shown by the data
is that contact with the chiropractor fails to provide additional benefits
in addition to medication in the management of childhood asthma. It is
simply an outmoded concept to assume that simply the presence or absence
of cavitation constitutes the difference between chiropractic and no
Lack of complete representation of
Given the fact that the human diurnal cycle lasts 24
hours, I am mystified by the lack of data representing nighttime symptoms.
In effect, we have been shown only half the complete picture in this
Balon's study reflects the challenges and problems of
properly designing a clinical trial which involves more than simply
ingesting pills which can be fully masked. In the application of manual
therapies, practitioners cannot be blinded. The result in single-blind
clinical investigations such as represented by the Balon study is that the
authors rely solely upon the patients' incorrect answers to validate their
ignorance as to what type of treatment they received. There is no
allowance for the nuances of emotion or expectations of the therapist
which are conveyed to the patient.
Possible masking of effects
by Type II error:
Even with its questionable design, the Balon study
appears to demonstrate a tendency toward improvement in activity,
symptoms, emotions, and overall quality of life in the manipulated as
compared to the sham treated group. Statistical significance could not be
demonstrated, however, presumably because the experimental groups employed
in the trial were too small. Obscuring of significant results by improper
experimental design or interpretation is known as a Type II
Another highly visible
clinical trial comparing three interventions in the management of acute
low-back pain  suffered from poor design  and
inappropriate statistical procedures.  Worse, it implied that a
single intervention represented chiropractic care such that its clinical
relevance was highly questionable.
Indeed, the Royal College of General Practitioners in a
very recent systematic review of the literature designed to update the
CSAG Guidelines of the United Kingdom  has concluded that this
trial neither adds nor detracts from the evidence base regarding
appropriate interventions for low-back
A meta-analysis has
shown that contrasting interpretations can be obtained, depending upon
which of 25 scales used to distinguish between high- and low-quality
trials is actually employed. 
A review of clinical
trials comparing two antifungal agents has indicated that the apparent
advantages of one of the instruments could have been obtained by
manipulations of the design of most of the trials, in which the competing
agent was inappropriately administered. 
The weight of evidence
produced by clinical trials may be overcalculated due to the fact that the
clinical trials are overrepresented as duplicate, "sausage" publications
by the same authors. [11–14]
attached to clinical trials create a misleading profile of high- and
low-quality studies if they place too much emphasis upon sham procedures
which we already know will seriously compromise controlled studies
involving physical methods such as spinal manipulation if they are not
true placebos. In other instances, the mere utterance of such terms as
"blinded" or "randomized" in the title of the paper cited may be
sufficient to glean points in the rating of clinical trials--even though
such terms are never defined or qualified. The proper remedy in this
instance would be to demote the trial ratings if such terms are
inappropriately used. 
The point to realize here is that RCTs are subject to
misinterpretation and outright abuse. Their generalization from a
fastidious, defined laboratory setting is problematical. It is sometimes
forgotten that the source of randomized clinical trials remains the sound,
well-documented observations in the clinical setting. This has led no less
an epidemiologist than David Sackett to conclude that there are
essentially two pillars of sound clinical evidence, only one of which is
experimentally derived from the RCT: 
"External clinical evidence can inform, but can never
replace, individual clinical expertise, and it is this expertise that
decides whether the external evidence applies to the individual patient at
all and, if so, how it should be integrated into a clinical
In light of these many arguments, I would maintain that
reviews of clinical research should place far greater emphasis upon cohort
studies and case series in its research goals rather than assume
categorically that they provide inferior guidance to clinical
decision-making than RCTs. It should be quite clear from this discussion
that a well-crafted cohort or case series is far more informative than a
flawed or corrupted RCT.
That said, one must then interpret such systematic
reviews as Hondras' effort with extreme caution on the basis that one or
more of its basic component RCTs is seriously flawed, such that the entire
review might then have incorrectly evaluated the best clinical evidence
Anthony L. Rosner, Ph.D.
June 27, 2000
Hondras MA, Linde K, Jones AP.
Manual therapy for asthma.
Cochrane Database Systematic Review 2000; 2: CD001992.
Balon J, Aker PD, Crowther ER, Danielson C, Cox PG, et. al.
A comparison of active and simulated chiropractic manipulation as
adjunctive treatment for childhood asthma.
New England Journal of Medicine 1998; 339: 1013-1020.
Field T, Henteleff T, Hernandez M, Martinez E, Mavunda K, Kuhn C, Schanberg S.
Children with asthma improved pulmonary functions after massage therapy.
Journal of Pediatrics 1998; 32(5): 854-858.
Cherkin, DC, Deyo, RA, Battie, M, Street, J, and Barlow, W.
A Comparison of Physical Therapy, Chiropractic Manipulation, and Provision
of an Educational Booklet for the Treatment of Patients
with Low Back Pain
New England Journal of Medicine 1998 (Oct 8); 339 (15): 1021-1029
Back pain, science, politics and money.
The Chiropractic Report November 1998; 12(6).
Freeman M, Rossignol A.
A critical evaluation of the methodology of a low back pain clinical trial:
A case study in misleading statistics.
Journal of Manipulative and Physiological Therapeutics 2000; 23(5): in press.
Back pain. Report of a Clinical Standards Advisory Group Committee on back pain.
May 1994, London: HMSO.
Royal College of General Practitioners,
unpublished update of CSAG Guidelines [reference 2], 1999.
Juni P, Witschi A, Bloch R, Egger M.
The hazards of scoring the quality of clinical trials for meta-analysis.
Journal of the American Medical Association 1999; 82(11): 1054-1060.
Johansen HK, Gotzsche PC,
Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis.
Journal of the American Medical Association 1999; 282(18): 1752-1759.
Fair conduct and fair reporting of clinical trials.
Journal of the American Medical Association 1999; 282(18): 1766-1768.
Multiple publication of reports of drug trials.
European Journal of Clinical Pharmacology 1989; 36: 429-432.
Huston P, Moher D.
Redundancy, disaggregation, and the integrity of medical research.
Lancet 1996; 347: 1024-1026.
Tramer MR, Reynolds DJM, Moore RA, McQuay
Impact of covert duplicate publication on meta-analysis: A case study.
British Medical Journal 1997; 315: 635-640.
Editorial: Evidence-based medicine.
Spine 1998; 23(10): 1085-1086.
Return to PREVIOUSLY PUBLISHED