FROM:
J Manipulative Physiol Ther 2003 (Jan); 26 (1): 47 ~ FULL TEXT
Wenban AB
Private practice of Chiropractic, Barcelona, Spain.
OBJECTIVE: To calculate the proportion of care delivered in a chiropractic practice supported by good-quality clinical trials.
DESIGN: Retrospective survey.
METHODS: Data were collected from patient files relating to 180 consecutive patient visits in a suburban chiropractic practice in northern Spain. Each patient's presenting complaint was paired with the chiropractor's chosen primary intervention. Based on a literature review (Medline, Mantis, and nonautomated searches of local medical libraries), each presenting complaint-primary intervention pairing was categorized according to the level of supporting evidence as follows: Category I, intervention based on good quality clinical trial evidence; Category II, intervention based on poor-quality or no clinical trial evidence. To distinguish between good- and poor-quality clinical trials, studies were critically appraised and assigned quality scores.
RESULTS: Of the 180 cases surveyed, 123 (68.3%) (95% CI, 61.5%-75.1%) were based on clinical trials of good methodologic quality (Category I). Only 57 (31.7%) (95% CI, 24.9%-38.5%) of the cases were based on poor-quality or no clinical trial evidence (Category II).
CONCLUSION: When patients were used as the denominator, the majority of cases in a chiropractic practice were cared for with interventions based on evidence from good-quality, randomized clinical trials. When compared to the many other studies of similar design that have evaluated the extent to which different medical specialties are evidence based, chiropractic practice was found to have the highest proportion of care (68.3%) supported by good-quality experimental evidence.
From the Full-Text Article:
Discussion
The retrospective nature of this survey helped to prevent the occurrence of observation bias; that is, the chiropractor who rendered the care was not aware that her case notes would be reviewed for the purposes of this study, thus blinding her to the intent of the study and protecting the integrity of the case notes as they pertained to the quality of intervention.
However, a number of problems arise out of basing this analysis solely on the primary presenting condition and the primary intervention applied. This study is based on a very simplified model of clinical practice, whereas, in reality, clinical practice is a complex encounter. Often, patients have more than just 1 presenting complaint (comorbidity), and chiropractors often resort to using more than 1 intervention with the same patient. Moreover, many questions yet to be asked of the clinical encounter are not amenable to the RCT format, such as “What do practicing chiropractors consider valid and useful sources of evidence for clinical practice?” and “Do chiropractors deem the methods taught under the banner of evidence-based practice (EBP) relevant and useful in answering those questions that arise out of clinical encounters with their patients?”
One author has criticized other studies similar in design to this study, by stating, “In measuring what is most readily measurable, they reduce the multidimensional doctor–patient encounter to a bald dichotomy and may therefore distort rather than summarize the doctor's overall performance.” [66] This criticism must certainly be considered in this study and chiropractic in general, where the clinical encounter is multidimensional, and where many practitioners see themselves delivering care, as opposed to treatment, from a wellness perspective, which in the health/disease continuum is always viewed relative to the pursuit of an individual's optimum potential.
It has been suggested that “at the very least, future attempts to answer the question ‘How evidence based is my practice?’ should include some measure of how competing clinical questions were prioritized for each case and how the evidence obtained was specified to reflect the needs and choices of the individual patient.” [67] Although this is a legitimate concern, no attempt was made in this study to measure or assess such qualitative information. Regardless, this study lends further weight to the belief that chiropractic can be evaluated with methods as rigorous, or more so, as those used to evaluate specialties of medicine.
Probably the weakest point is this study's methodology, a weakness that exists in all the similar, previous studies that have examined the extent to which medical specialties are “evidence based” simply because 1 supportive RCT is located in the literature. These studies are further weakened by their failure to search for and take into account nonsupportive RCTs. These flawed benchmarks were set by the authors who designed and carried out the first of these types of studies21 and have been perpetuated by a number of subsequent authors, [22, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36] myself included. The use of integrative studies, including systematic reviews, are recommended as a critical step in improving the quality of this type of study in the future.
The proponents of EBP have stated that when seeking answers to questions about the effectiveness of therapy, “we should avoid the non-experimental approaches, since these routinely lead to false-positive conclusions about efficacy.” [68] Therefore, in keeping with that statement, I made an effort to exclude RCTs of poor methodologic quality (quality score <50), and did not accept, or search for, convincing nonexperimental studies as evidence of intervention effectiveness. In contrast, similar previous studies that asked how evidence based are a number of medical specialties, 1 of which was coauthored by a leading proponent of EBP, [21] did not distinguish between good- and poor-quality RCTs, and accepted nonexperimental studies as proof that an intervention, and ultimately a medical specialty, was evidence based. It appears contradictory to me that the proponents of EBP promote 1 very high standard when teaching EBP (ie, in questions about therapy, we should avoid nonexperimental approaches) but then lower the bar when it comes to assessing the extent to which their own medical specialties are evidence based. The study by Gill et al, [22] assessing the proportion of interventions in general medical practice that were evidence based, is a case in point. That study's inclusion criteria for what counts as evidence in support of an intervention's effectiveness were set such that they included nonexperimental studies. The study authored by Nordin-Johansson et al [36] went even further when, in studying the extent to which internal medicine was evidence based, they allowed the inclusion of the consensus opinions of national expert panels to count toward calling internal medicine evidence based.
As a result of the very loose criteria they employed, Gill et al [22] were able to claim in the conclusion to their study that 81% of interventions used in general practice are evidence based. However, had the authors of that study set their inclusion criteria in accordance with the industry standard as promulgated by the proponents of EBP, thereby only allowing RCTs to count as evidence of an intervention's effectiveness, the proportion of interventions used in general practice that could be claimed to be evidence based would have dropped to only 31%. Very probably, had the authors of that article taken the time to appraise critically the supportive RCTs they located, and to exclude RCTs of poor quality, the proportion of care provided in general practice may have been less than 31%. Interestingly, chiropractic, which has come under ongoing criticism for its lack of research, fares comparatively well (see Table 6). Of the care provided by the chiropractor in this study, 68.3% was deemed evidence based when examined with a similar, if not more stringent, methodology than that used in assessing the extent to which a number of medical specialties are evidence based.
The Cochrane collaboration model [69] of compiling systematic reviews of evidence argues that, when they exist, most weight should be given to carefully controlled trials. However, this approach is inclined to provide answers to questions that are easily addressed with existing research methodologies. It does not necessarily address all the needs posed by clinical practice. For example, in chiropractic, to date, much of the research has been directed toward treating individuals who are not well. In this study, which involves chiropractic patients, 27.2% of patients received care for indications that were not supported by RCTs of good methodologic quality. Those conditions may provide researchers of a biomedical inclination with fruitful areas for future research into the effectiveness of chiropractic care. If, however, the purpose of chiropractic is to optimize health, as suggested by the Association of Chiropractic Colleges paradigm, [70] then an equally appropriate research focus might be to explore why and how chiropractic care can best support well individuals in maintaining a disease-free state and attaining optimal well-being.
One author, [71] after examining the medical and chiropractic literature in relation to the role of the chiropractor within the broader health care system, has suggested that studies [72-74] performed to date “reflect a much broader scope of practice for chiropractic than is suggested by the epidemiology of the patient complaints.” In light of such preliminary findings, it may be that the chiropractic profession and public would benefit from further exploration of the role of chiropractic care from a wellness perspective. Even the chiropractor, from whom the data for this study was collected, contends that the care delivered was not directed specifically toward the resolution of a patient's chief complaint or symptoms. Instead, the chiropractor's stated intention was “to optimize the individual's inherent healing capacity.” Preliminary studies do support the observation that some quality-of-life measures do improve in patients who receive ongoing chiropractic care, [75, 76] but whether such improvements accrue from an optimized healing capacity due to periodic spinal adjustment awaits further research.
Because this study involved performing a thorough but far from exhaustive literature search, only a more extensive investigation of the literature will reveal whether further studies exist that support the care delivered by the chiropractor in this study. Furthermore, since the literature review for this study was performed, a number of published RCTs [77, 78] support interventions that formed part of this study and may therefore result in a greater proportion of chiropractic practice being deemed evidence based should this type of study be repeated.
Eleven of the 14 previous studies of this type were structured such that they considered positive, convincing, nonexperimental studies worthy of qualifying an intervention and discipline as “evidence based.” In contrast to that approach, I attempted to raise the standard on what qualified for “evidence-based” status, as did Geddes et al, [25] who examined to what extent psychiatric care was evidence based, and Michaud et al, [30] when they similarly examined internal medicine. This was achieved by acknowledging interventions to be “evidence based” only if supported by at least 1 relevant randomized clinical trial. Additionally, this study went 1 step beyond all similar, previous studies by critically appraising those RCTs that were found and excluding those that did not achieve a quality score of 50 points or more. Despite this more rigorous approach, it should be remembered that individual studies are rarely ever conclusive and, as mentioned earlier in this discussion, the use of integrative studies, including systematic reviews, are recommended as a critical step in improving the quality of this type of study in the future.[79] Other researchers [27] that previously used a design similar to this study did incorporate systematic reviews into Category I evidence but were able to locate only 2 reviews of relevance.
Further use of this research method in examining the extent to which a given area of health care is evidence based may require a more thorough evaluation of the methodologic quality of the RCTs used in support of the care delivered. In this study, quality scores were used in an attempt to improve the quality of permissible evidence. However, the validity of such quality assessment scales has recently been criticized, with one author stating, “Perhaps the most insidious form of subjectivity masquerading as objectivity . . . is ‘quality scoring’” and “I wholeheartedly condemn quality scores because they conflate objective study properties, such as study design, with subjective and often arbitrary quality weighting schemes.” [80] Furthermore, a recent meta-analysis of studies using different quality assessment scales concluded that the use of summary scores are, at best, problematic. [81] A number of the other studies that have used this research design have been criticized as follows: “Apart from anything else, they were undertaken in specialized units and looked at the practice of world experts in Evidence Based Medicine; hence, the figures arrived at can hardly be generalized beyond their immediate setting.” [66] This study does not suffer from such restrictive shortcomings, because it was carried out in a typical chiropractic practice, and the practitioner involved had only a basic grounding in EBP. However, generalization of these findings to other chiropractic practices must await, at a minimum, supportive findings from larger similar surveys. I am presently conducting further studies using an evolved version of this pilot study methodology to examine the extent to which chiropractic practice might be considered evidence based in a number of different countries.
In chiropractic, as has been observed in CAM, [82] there is a strong emphasis on tailoring care to the individual patient. This creates problems for the incorporation of EBP into chiropractic, because the actions of the proponents of EBP, despite what they may claim, reveal that research designs other than the RCT format are considered to be of little value in questions concerning the efficacy of an intervention. [83] The paradox of the clinical trial is that it may be able to assess whether an intervention works under artificially specified controlled conditions, but in no way can it assess who will benefit from a given intervention. Therefore, although accurate decision making in a clinical setting may benefit from evidence derived from RCTs, the successful application of that evidence to the individual patient before us requires evidence from both qualitative and quantitative research. [84]
The importance of research paradigms, other than the quantitative, have previously been discussed in the chiropractic literature. [85, 86] I concur with those authors, in that chiropractic needs to make a balanced investment in quantitative, qualitative, and emergent research paradigms if it is to optimize the health of its science, art, and philosophy.
Read the
FULL TEXT Article now.
