Sophisticated Research Design in
Chiropractic and Manipulative Therapy;
“What You Learn Depends on How You Ask.”

Part A. Quantitative Research: Size Does Matter

This section is compiled by Frank M. Painter, D.C.
Send all comments or additions to:     Frankp@chiro.org

FROM:   Chiropractic Journal of Australia 2016; 44 (2): 1–21 ~ FULL TEXT

OPEN ACCESS

---

Lyndon G. Amorin-Woods, BAppSci(Chiropractic), MPH

Senior Clinical Supervisor;
Murdoch University Chiropractic Clinic
School of Health Professions,
Discipline of Chiropractic Murdoch University South Street campus,
90 South Street, Murdoch,
Western Australia 6150

Enjoy Part 2:   Qualitative Research: Quality vs. Quantity    Enjoy Part 3:   Mixed Methods: “Why Can’t Science And Chiropractic Just Be Friends?”

---

Many chiropractors remain sceptical of evidence-based practice (EBP) and some may view it as an attack on the profession which they feel must be resisted. A counter-argument is centred on the primacy afforded quantitative methodology as epitomised by the randomised controlled trial (RCT). This defensive posture may be mitigated by recognising the role complex research has played in the legitimisation of the profession. The pre-eminence of the randomised controlled trial (RCT), considered by many as the gold-standard of evidence, has led some authors to go so far as to functionally disregard all evidence that is not an RCT. However, it is readily apparent the RCT is not always the most appropriate study design to gather evidence, especially in the CAM health sector. This paper discusses the role of sophisticated design in quantitative chiropractic research, presenting examples sequentially through the traditional quantitative hierarchy and concludes that optimal methodology depends on the research question. Research design must allow for the various dimensions of the (chiropractic) clinical encounter, and may be sophisticated at all levels, but must above all, be contextual. The ‘best available’ or most relevant evidence depends on what one needs for a specific purpose. A critical caution is the proviso that care must be exercised not to draw inappropriate conclusions such as causation from descriptive studies.

INDEX TERMS: Chiropractic; Evidence-Based Practice; Quantitative Evaluation; Research Design

---

INTRODUCTION

Proponents and detractors of evidence-based practice (EBP) in chiropractic, in common with the rest of healthcare, generally adopt antithetical positions characterised more by dogmatic convictions than by genuine debate. Some consider RCT evidence as the gold standard of sophisticated evidence, while others are highly critical. [1] The principal proposition of this paper will be that sophisticated research designs have an important role in generating new knowledge at all ‘levels’ of the hierarchy and should not be avoided because of the challenge presented by complexity. It is my view that a sequential analysis of the various study designs in clinical and health system research demonstrates that different designs have each added a unique dimension to the corpus of knowledge concerning chiropractic, manual therapy, spinal pain complementary medicine and human well-being. A study may reside ‘lower’ on the evidentiary hierarchy, but this certainly does not preclude it from being complex, sophisticated or valuable.

It is pertinent to acknowledge that the very delineation of evidence into strata, a so-called hierarchy, has itself been extensively critiqued; for example Goldberg observed EBM’s rigid hierarchy of evidence to be “the culprit of its objectionable epistemic practices”. [2] Of late, the re-emerging stature of qualitative research mirrors the journey of science and philosophy through significant paradigms or worldviews, including; positivist, post-positivist, critical theory, constructivist and participatory/advocacy. [3] Quantum physics and complexity theory further add to the recognition that science, and especially health science, is still subject to the dictum; “absence of proof does not equal proof of absence.”

Modern epidemiology and healthcare research are part of a human tradition of inquiry that stretch back centuries, deep into antiquity at least until Ibn Sina (Avicenna) in 1,025 AD, who may possibly have conceived the first recorded ‘RCT’ design. [4, 5] From the beginning of time, philosophers and deep thinkers have intuitively tried to establish association, causation and effect, albeit in a culturally evolutionary nomenclature and context. From the ancient healer, to the modern clinician, in all cultures, humans have been engrossed in studying and understanding links between their surroundings and disease, and finding ways to relieve ill health and live productive lives.

This tradition of inquiry accelerated post ‘Enlightenment’ in western civilization with advances in public health through various acknowledged and distinct public health eras namely; ‘health protection’, (antiquity-1830’s), ‘miasma control’ (1830-80’s), ‘contagion control’ (1880’s-1930’s), ‘preventive medicine’ (1940-60’s), ‘primary health care’ (1970-80’s), ‘health promotion’ (1980-2000), and finally ‘population health’ (2000-present). [6] Scientists have always attempted to understand the aetiology and course of disorders and ailments, often driven by a geopolitical imperative. [7] As an aside, the ‘modern’ RCT design is generally acknowledged to have originated in the early 20th century, not in health care but in agriculture, via work done on experimental design by Sir Ronald Fisher. [8] Fisher’s, seminal works related to crops, which lend themselves particularly well to randomisation and outcome determinations. [9]

Table 1

In the late 1950s, Sir Austin Bradford Hill formally articulated theoretical principles and methodology for ensuring the reliability of results of clinical research (Table 1). These ‘criteria’ subsequently became the basis of the protocols for RCT design. [10, 11] Bradford Hill participated as the statistician in what is believed to be the first medical randomised controlled trial of the modern era; conducted in the 1940s to examine the effectiveness of streptomycin as treatment for tuberculosis. [12]

From these beginnings has grown what is known as Evidence-Based Medicine (EBM), which has morphed through various iterations including ‘Evidence-Based Practice’ (EBP) and of late, ‘Evidence-Informed Practice.’ In chiropractic, along with other healthcare professions, the advocates of evidence-based practice often find themselves in conflict with those that disagree with the ‘reductionist’ approach. [13-15] Some even associate RCT’s with a so-called ‘cult of scientism’. [16] In recent years, there has been increasing scientific discourse regarding the relative value of studies other than RCT’s especially when applied to chiropractic, particularly in the context of good RCT design. [17-19] For example, Rosner eloquently articulated the view that well conducted ‘lower level’ studies are often of more value than flawed RCT’s, presenting historical examples of notable case studies, “from heart transplants to Freudian psychology, (that) never went on to become RCT’s in their own right”. [20] Proponents of the chiropractic subluxation paradigm thusly criticise those who assert there is no evidence at all to support the construct as being (intentionally) blinded to evidence other than RCT’s, since the theory does not lend itself as readily to investigation via RCT as does for example, pharmaceutical research. [21]

Figure 1 The traditional pyramid of evidence [22, 23] Figure 2 Figure 3 Figure 4

      Design of Quantitative Epidemiological Studies

Evidence based medicine has historically has almost been a metaphor for quantitative methodology. Quantitative evidence is even stratified into various levels; a so called ‘pyramid’. These strata will be considered in a sequential manner in this paper; however it is pertinent to acknowledge firstly that there exist quite a few variations on this ‘pyramid’, depending on the intended use of the evidence, i.e.; for diagnosis or therapy. [22, 23] (Figures 1-3 and Table 2).
Quality of evidence

High Quality Evidence:   one can be very confident the effect estimate presented lies very close to the true value, thus there is a very low probability of further research completely changing the presented conclusions.

Moderate Quality Evidence:   one can be confident that the presented effect estimate lies close to the true value, but it is also possible that it may be substantially different, so further research may completely change the conclusions.

Low Quality Evidence:   one is not confident in the effect estimate and the true value may be substantially different so further research is likely to change the presented conclusions completely.

Very low quality Evidence:   there is no confidence in the estimate and it is likely that the true value is substantially different so new research will probably change the presented conclusions. [24]

      Guidelines and Recommendations

The strength of a recommendation reflects the extent of confidence that desirable effects of an intervention outweigh undesirable effects. Strong recommendations mean that most informed patients would choose the recommended management and that clinicians can structure their interactions with patients accordingly (most people in the same situation would want the recommended course of action and only a small proportion would not). Weak recommendations mean that patients’ choices will vary according to their values and preferences, and clinicians must ensure that patients’ care is in keeping with their values and preferences (most people in the same situation would want the recommended course of action, but many would not). Strength of recommendation is determined by

1) the balance between desirable and undesirable consequences of
     alternative management strategies

2) quality of evidence

3) variability in values and preferences, and

4) resource use [cost]. [27, 28]

Most guideline panels have used letters and numbers to summarise their recommendations, but they use them differently. This is potentially confusing, thus a binary system is often preferred. [28, 29]

      Cause and Effect

The search for proximate cause of ill health; the ‘evidence’, is a somewhat fraught subject. At the outset of any discussion about evidence, it is important to reflect that the term ‘causation’ is itself sometimes poorly understood. To be a ‘cause’ the factor must directly influence the occurrence of a disease or condition; this means it must be more than simply an observed association. A cause is a preceding event, condition or characteristic that is necessary for, and results in, a given outcome at a specific time.

Epidemiologists recognise four types of causal relationships:

• Necessary and sufficient (uncommon; e.g. ingesting sufficient poison)

• Necessary but not sufficient (multiple factors required, e.g., cancer)

• Sufficient but not necessary (e.g., radiation exposure)

• Neither necessary nor sufficient (e.g., most chronic diseases). [30]

‘Post hoc ergo propter hoc,’ "after this, therefore because of this." It is also imperative to bear in mind; association does not equal cause; in reality, few conditions have a single cause; usually there are multiple influences interacting in complex ways. Exposure to a causal agent does not necessarily always cause a condition to develop. A more practical concept is the contemporary concept of ‘probabilistic causation’; where a cause is defined as “a factor that increases that probability (or chance) that an effect will occur”. [30] Even with respect to infectious diseases, medical epidemiologists recognise a triad that is in a state of flux – environment, host and agent - and the distributions, effects and type of infectious disease fluctuate as each of these factors vary. Diseases are rarely said to be ‘caused by’ factor ‘x’. Conversely one must bear in mind lack of association certainly disproves causality.

      Aims

This paper aims to identify examples of complex quantitative chiropractic and manipulative therapy research at each level of the traditional quantitative hierarchy.

---

METHODS

Five databases were searched in addition to ‘Google Scholar’ in December 2012. These databases included Medline, Web of Science, EMBASE, SportDiscus, and The Cochrane Library. The search criteria used were “chiropractic research” OR “manipulative therapy research” OR “complex research” AND ‘‘evidence hierarchy’’ AND “quantitative” AND ”the particular type of evidence” for example; “randomised controlled trial.” The examples are chosen somewhat arbitrarily, primarily on the criteria of their usefulness as examples of the study designs.

---

RESULTS

Following is a sequential analysis of the traditional hierarchy of quantitative evidence along with selected chiropractic examples of each. Examples were chosen as being representative of study design; no evaluation is made on study quality.
Note: Quotation marks indicate abstracts are quoted directly

      Clinical [Anecdotal] Expert Opinion

The epistemology of evidence-based medicine categorises expert opinion as the lowest form of evidence, superseded even by methodologically flawed clinical research, however it is an inescapable reality that expert opinion is likely to remain as the bedrock on which clinical practice is built; “randomized clinical trials can tell us which treatment is better, but they cannot tell us for whom it is better”. [31, 32]

Despite its low ranking in the evidence hierarchy, anecdotal information still exerts a disproportionally powerful influence on clinical thinking and behaviour, consequently one must be careful to distinguish between the use of anecdotes to convey ideas and influence behaviour, and their validity in making causal inferences. [32 ] In health care, recommendations by a respected local peer have been shown to be a more powerful force for change in clinical practice than evidence-based consensus guidelines published nationally. [33] Since chiropractors have been accused of extrapolating causation from anecdotal clinical experience, a better understanding of the evidentiary hierarchy may mitigate this tendency.

      In-Vitro Studies

Basic science, also known as in-vitro (meaning literally ‘in glass’) or bench research typically resides in Level III (or Level D recommendation) of the evidentiary hierarchy. [26] Animal studies and human in-vitro research do have a distinct advantage; they permit simplification of the system under study. [34] However, it can be challenging to extrapolate from the laboratory results to the functioning person or from animals to humans. Investigators, (and those using the research findings) must thus be careful to avoid over-interpretation or generalisation of their results. [35]

Chiropractic example:   Song [36], Spinal manipulation reduces pain and hyperalgaesia after lumbar intervertebral foramen inflammation in the rat. “This study showed that Activator-assisted Spinal Manipulative Therapy (ASMT) can significantly reduce the severity and shorten the duration of pain and hyperalgesia caused by lumbar IVF inflammation. Manipulation of a specific spinal segment may play an important role in optimising recovery from lesions involving IVF inflammation”.

      Descriptive Studies

As the name suggests, these studies describe disease or risk factor frequency in relation to person, place and time. They do not include researcher-imposed treatments and thus cannot establish causal relationships. Important uses of descriptive studies include trend analysis, health-care planning, and hypothesis generation. A frequent error in reports of descriptive studies again is overstepping the data; studies without a comparison group allow no inferences to be drawn about associations, causal or otherwise. Hypotheses about causation from descriptive studies are often subsequently tested in rigorous analytical studies. [37]

      Case Reports and Case Series

The value of case reports and case-series has sometimes been understated and maligned; their historical significance however should certainly not be overlooked; all chiropractors will be familiar with the (whether apocryphal or not) case of one Harvey Lillard. Case reports are sometimes the initiating vector for creating suspicion and generating hypotheses, for example; in 1961, a single case of a 40-year-old woman who developed a pulmonary embolism, 5 weeks after beginning to use an oral contraceptive pill, triggered suspicion the drug was the cause [38], and it was a case of Karposi’s Sarcoma and subsequent case-series that alerted authorities to the arrival of HIV/AIDS in Los Angeles. [39]

Chiropractic examples:

Case study: Lafond.   Rehabilitation program for traumatic chronic cervical pain associated with unsteadiness: a single case study.   “This case report indicates that an 8-week rehabilitation program combining therapeutic exercises with spinal manipulative therapy may have had an effect on improvement of postural control in a trauma Chronic Neck Pain patient with unsteadiness”. [40]

Case series: BenEliyahu.   Magnetic resonance imaging and clinical follow-up: study of 27 patients receiving chiropractic care for cervical and lumbar disc herniations. “This prospective case series suggests that chiropractic care may be a safe and helpful modality for the treatment of cervical and lumbar disc herniations”. [41]

Figure 4

      Correlation and Ecological Studies (Figure 4)

Ecological (or cross-sectional) studies analyse populations rather than individuals. The ecological study is thus susceptible to what is termed the ecological fallacy; i.e. associations noted at aggregate level do not extrapolate to the individual. Ecological studies can be a ‘snapshot’ (such as the National Health Survey conducted by the Australian Bureau of Statistics) and may be of a specific time window, e.g. a calendar year, or a fixed point.

Ecological studies can be easily confused with cohort studies, especially if different cohorts are located in different places. The difference is that in the case of ecological studies there is no information available about the individual members of the populations compared; whereas in a case control/cohort study the data pair exposure/health is known for each individual. [37]

Notwithstanding their limitations, ecological studies are certainly useful because they can be carried out easily, quickly and relatively inexpensively using data that are generally already available. [37] If interesting and strong associations are observed the results of ecological studies can provide the opportunity for later studies designed to build on the initial observations. Unlike case-control studies, they can be used to calculate not only an odds ratio (OR), but also absolute risks (AR) and relative risks (RR) from prevalence’s risk rates (PRR). They may be used to describe some feature of the population, such as prevalence of an illness, or they may support inferences of cause and effect. [37]

Chiropractic example:   Smith.   Chiropractic Health Care in Health Professional Shortage Areas in the United States. “Chiropractic providers render a substantial amount of care to underserved and rural populations. Health policy planners should consider the full complement of providers available to improve access to care”. [42]

      Analytic Studies

Figure 5

Case Control (Figure 5)   Case-control studies are used to identify factors that may contribute to a health condition by comparing retrospectively subjects who have that condition (the 'cases') with patients who do not have the condition but are otherwise similar (the 'controls'). [37]

Chiropractic examples:   Cassidy.   Risk of Vertebrobasilar Stroke and Chiropractic Care, Results of a Population-Based Case-Control and Case-Crossover Study.   “The increased risks of VBA stroke associated with chiropractic and Primary Care Practitioner (PCP) visits are likely due to patients with headache and neck pain from VBA dissection seeking care before their stroke, there was no evidence of excess risk of VBA stroke associated chiropractic care compared to primary care”. [43]

      Cohort Studies (Figure 6)

Figure 6

Cohort studies are similar to randomised controlled trials in that they can compare outcomes in groups that did and did not receive an intervention. The main difference is that allocation of individuals is not by chance. They can be thought of as natural experiments in which outcomes are measured in the real world. [44] They can evaluate even large groups of diverse individuals, follow them for long periods, and provide information on a range of outcomes, including rare adverse events for which RCT’s are not suitable. [45] The focus of any appraisal of a cohort study is on an individual comparison between an intervention group and a comparison group in a defined population (44). Cohort studies are however, known to be vulnerable to selection bias since allocation is not by chance as in a randomised controlled trial. [44]

      Retrospective Cohort

A retrospective [historic] cohort study generally means to look back in time and examine a patient's health history or lifestyle. It is a research study in which the records of groups of individuals who are alike in many ways but differ by a certain characteristic are compared for a particular outcome. In retrospective cohort studies, a risk ratio or odds ratio gives an assessment of relative risk (RR). [37]

Chiropractic examples:   Miller.   Adverse Effects of Spinal Manipulative Therapy in Children Younger Than 3 Years: A Retrospective Study in a Chiropractic Teaching Clinic.   “This was a 3-year retrospective study of 781 paediatric case files, a total of 5,242 chiropractic treatments. There were no serious complications resulting from chiropractic treatment. This study shows that for the population studied, chiropractic manipulation produced very few adverse effects and was a safe form of therapy in the treatment of patients in this age group”. [46]

      Prospective Cohort

A prospective cohort study is a cohort study that follows over time a group of similar individuals who differ with respect to certain factors under study, to determine how these factors affect rates of a certain outcome. [37] The prospective study is important for research on the aetiology of disorders in humans because for ethical reasons people cannot be deliberately exposed to suspected risk factors in controlled experiments. Prospective cohort studies are typically ranked higher in the hierarchy of evidence than retrospective cohort studies. [47]

Chiropractic examples:   Cifuentes.   Health Maintenance Care in Work-Related Low Back Pain and Its Association With Disability Recurrence.   “In 894 cases followed 1 year using workers’ compensation claims data, for work-related non-specific LBP, the use of health maintenance care provided by physical therapist or physician services was associated with a higher disability recurrence than in chiropractic services or no treatment”. [48]

      Randomised Controlled Trial (Figure 7)

Figure 7

The RCT occupies the top level of the traditional hierarchy of evidence adopted with minor variations internationally (e.g.; NHMRC Australia). Study subjects, after assessment of eligibility and recruitment, but before the intervention, are randomly allocated to receive 1 or other of the alternative interventions. Random allocation in real trials is complex, but conceptually, the process is like tossing a coin. After randomisation, the 2 (or more) groups of subjects are followed in exactly the same way, and the only differences between the intervention(s) they receive should be those intrinsic to the treatments being compared. The most important advantage of proper randomisation is that it minimizes allocation bias, balancing both known and unknown prognostic factors, in the assignment of treatments. [49] Since they are expensive and recruiting patients can be difficult, randomised controlled trials are generally short term and used to determine efficacy in selected populations under strict conditions. [37]

The Consolidated Standards of Reporting Trials (CONSORT) Statement is intended to improve the reporting of a randomised controlled trial (RCT), enabling observers to understand a trial's design, conduct, analysis and interpretation, and to assess the validity of its results. [50]

Chiropractic examples:   Bishop.   The Chiropractic Hospital-based Interventions Research Outcomes (CHIRO) Study: a randomized controlled trial on the effectiveness of clinical practice guidelines in the medical and chiropractic management of patients with acute mechanical low back pain.   “This was the first reported randomised controlled trial comparing full Clinical Practice Guideline-based (CPG) treatment, including spinal manipulative therapy administered by chiropractors, to family physician–directed Usual Care (UC) in the treatment of patients with Acute Mechanical-LBP. Compared to family physician-directed UC, full CPG-based treatment including Chiropractic SMT is associated with significantly greater improvement in condition-specific functioning”. [51]

      Systematic Review and Meta-Analysis

A systematic review (SR) is a literature review focused on a research question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question. [52] One of the principal sources of pre- appraised data is the Cochrane Collaboration; the methodology for SR’s are stated by the collaboration; a systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods selected to minimise bias. The key characteristics of a systematic review are:

• a clearly stated set of objectives with pre-defined eligibility criteria for studies;

• an explicit, reproducible methodology;

• a systematic search that attempts to identify all studies that would meet the eligibility criteria;

• an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias;

• a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarise the results of independent studies. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies. [53] Reporting standards for SR’s are articulated in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses [PRISMA] statement. [54]

Chiropractic examples:   Hawk (2007)   Chiropractic care for non-musculoskeletal conditions: a systematic review with implications for whole systems research.   “Databases searched included PubMed, Ovid, Mantis, Index to Chiropractic Literature, and CINAHL. All randomized controlled trials (RCTs) were evaluated using the Scottish Intercollegiate Guidelines Network (SIGN) and Jadad checklists; a checklist developed from the CONSORT (Consolidated Standards of Reporting Trials) guidelines; and one developed by the authors to evaluate studies in terms of Whole Systems Research (WSR) considerations”. [55]

      Evidence-Based Guidelines (EBG’s)

Jurisdictions [e.g.; Health Departments] refer to systematic reviews and metaanalyses; including those by the Cochrane collaboration, to evaluate the best available evidence when formulating evidence-based guidelines. For inclusion in a guideline there must be good evidence that each procedure recommended is effective; the benefits must outweigh the risks; the cost must be reasonable compared to its expected benefits; and finally, the recommended actions must be practical and feasible. Guidelines are intended to enable, guide, motivate, or sometimes force physicians and other types of providers to deliver certain types of care; however, they do not directly determine the care provided to a particular patient”. [27]

Example relevant to chiropractic:   Goertz.   Institute for Clinical Systems Improvement [ICSI; Minnesota and Wisconsin] Health Care Guideline; Adult Acute and Subacute Low Back Pain. “Spinal manipulative therapy should be considered in the early intervention of low back pain” [Moderate evidence, strong recommendation]. [56]

---

DISCUSSION

Evidence-based practice is founded on the premise of utilising the best available evidence in clinical decision making taking into account the patients’ values, goals and expectations and the practitioners expertise and scope of practice. [57] Often it is not necessary or appropriate for that best evidence to be an RCT. It is openly acknowledged and articulated by eminent thinkers such as Diamond and Robinson that some central questions especially in the natural and social sciences can't be answered by controlled laboratory experiments; many manipulative experiments, while theoretically possible, would be considered immoral or illegal. Other methods of observing, describing, and explaining phenomena must be devised (58).

It is also openly acknowledged that in some areas of healthcare [among them surgery, interventional pain management, public health and healthcare delivery] that the bulk of evidence of clinical intervention and/or or policy effectiveness has been derived from non-randomised designs [59], thus it appears somewhat incongruous that the canon of ‘high level evidence’ the RCT be exclusively applied to chiropractic and SMT as the sole criteria justifying clinical management. [13]

---

CONCLUSION

Complexity certainly presents challenges for those designing and conducting clinical trials, however this is by no means a challenge unique to chiropractic and the challenge should not be viewed as overly daunting. Controlled trials and other sophisticated research designs have been conducted within the chiropractic and CAM sector for many decades. There is now an impressive body of literature exploring the management of spinal pain, which after all constitutes well over 90% of chiropractic practice. [60] There remain however, many important areas that require further investigation, for example; chiropractic for paediatric patients, dose-response, supportive/maintenance therapy, non-musculoskeletal conditions, subgroups and cost effectiveness.

---

ACKNOWLEDGEMENTS

This paper is based on a series of lectures given during the PUBH4403 unit of the MPH by coursework at the School of Population Health, University of WA by David Preen & Jane Heyworth 2012.

---

REFERENCES:

1. Ioannidis J.
   Why most published research findings are false.
   PLoS Med. 2005;2(8):e124.

2. Goldenberg M.
   Iconoclast or Creed? Objectivism, pragmatism, and the hierarchy of evidence.
   Perspectives in Biology and Medicine. 2009;52(2):168-87.

3. Creswell J.
   Qualitative inquiry and research design.
   Choosing among five approaches.
   2nd ed: Sage Publications, Thousand Oaks.; 2007.

4. Tschanz D.
   The Arab Roots of European Medicine.
   Saudi Aramco World. 1997;48(3):20-31.

5. Ibn Sina (Avicenna).
   The Canon of Medicine (The Qanun) Persia. 1025.

6. Awofeso N.
   What’s New About the “New Public Health”?
   Am J Public Health. 2004(94):705-9.

7. Huber V.
   The Unification of the Globe by Disease?
   The International Sanitary Conferences on Cholera 1851-1894.
   The Historical Journal. 2006;49(2):453-76.

8. Barahona C.
   Randomised Control Trials for the Impact Evaluation of
   Development Initiatives: A Statistician’s Point of View.
   ILAC Working Paper 13, Rome, Italy:
   Institutional Learning and Change Initiative. 2010.

9. Fisher RA.
   Statistical Methods for Research Workers.
   Edinburgh and London: Oliver & Boyd; 1932.

10. Medical Research Council.
    A Framework for development and evaluation of RCTs for
    complex interventions to improve health.
    London: Medical Research Council. 2000.

11. Medical Research Council.
    Developing and evaluating complex interventions: new guidance.
    London: Medical Research Council. 2008.

12. Medical Research Council.
    Streptomycin treatment of pulmonary tuberculosis.
    BMJ (Clinical research ed). 1948;2:769-82.

13. Bronfort G, Haas M, Evans R, Leiniger B, Triano J.
    Effectiveness of Manual Therapies: The UK Evidence Report
    Chiropractic & Osteopathy 2010 (Feb 25); 18 (1): 3

14. Haneline M, Meeker WC., editors.
    Introduction to Public Health For Chiropractors:
    Jones And Bartlett Publishers; 2011.

15. Villanueva-Russell Y.
    Evidence-Based Medicine and its Implications for the Profession of Chiropractic.
    Social science and medicine. 2005;60:545-61.

16. Esteb W.
    RCT and the Cult of Scientism 2010. Available from:
    http://www.patientmedia.com/blog/2010/05/rct_and_the_cult_of_scientism.html

17. Benson K. HA.
    A comparison of observational studies and randomized, controlled trials.
    New England Journal of Medicine. 2000;342(25):1878-86.

18. Concato J., Nirav-Shah, Horwitz RI.
    Randomized, controlled trials, observational studies and the hierarchy of research designs.
    The New England journal of medicine. 2000;342(25):1887-92

19. Rosner A.
    Fables or Foibles: Inherent Problems with RCTs
    J Manipulative Physiol Ther 2003 (Sep); 26 (7): 460–467

20. Rosner A.
    Evidence-Based Medicine: Changing with the Tides 2009. Available from:
    http://www.chiroaccess.com/Articles/Evidence-Based-Medicine-
    Changing-with-theTides.aspx?id=0000098
    [2/04/2011].

21. Kent C.
    An Analysis of the General Chiropractic Council's Policy on Claims Made for the
    Vertebral Subluxation Complex.
    Foundation for Vertebral Subluxation. 2010(June 15):1-8.

22. SUNY Downstate Medical Center.
    Evidence Based Medicine Course. A Guide to Research Methods:
    The Evidence Pyramid: Medical Research Library of Brooklyn; 2001. Available from: http://phpartners.org/tutorial/04-ebph/2-keyConcepts/4.2.7.html

23. Health Sciences Library.
    Evidence-Based Practice Tools Seattle, WA: University of Washington; 2011. Available from:
    http://libguides.hsl.washington.edu/ebptools.

24. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al.
    GRADE guidelines: 3. Rating the quality of evidence.
    J Clin Epidemiol. 2011;64(4):401-6.

25. Oxford Centre for Evidence-based Medicine:
    University of Oxford; 2011. Available from:
    http://www.cebm.net/index.aspx?o=1025

26. Ebell M, Siwek J, Weiss B, Woolf S, Susman J, Ewigman B, et al.
    Strength of Recommendation Taxonomy (SORT): a patient-centered approach to
    grading evidence in the medical literature.
    Am Fam Physician. 2004;69:549-57.

27. Eddy DM.
    Evidence-Based Medicine: A Unified Approach.
    Health Affairs. 2005;24(1):9- 17.

28. Guyatt G, Oxman A, Kunz R, Falck-Ytter Y, Vist G, Liberati A, et al.
    Rating quality of evidence and strength of recommendations:
    Going from evidence to recommendations.
    BMJ. 2008;336(7652):1049-51.

29. Schunemann H, Best D, Vist G, Oxman A.
    Letters, numbers, symbols and words:
    how to communicate grades of evidence and recommendations.
    CMAJ. 2003;169:677-80.

30. Parascandola M, Weed DL.
    Causation in epidemiology.
    Journal of Epidemiology and Community Health. 2001;55(12):905-12.

31. Tonelli M.
    In defense of expert opinion.
    Acad Med. 1999;74(11):1187-92.

32. Enkin MW, Jadad AR.
    Using anecdotal information in evidence-based health care:
    Heresy or necessity?
    Annals of Oncology. 1998;9(9):963-6.

33. Lomas J, Enkin M, Anderson G, et al.
    Opinion leaders vs. audit and feedback to implement practice guidelines.
    JAMA. 1991;265:2202-7.

34. Vignais P, Vignais P.
    Discovering Life, Manufacturing Life:
    How the experimental method shaped life sciences.
    Berlin: Springer; 2010.

35. Rothman SS.
    Lessons from the living cell: the culture of science and the limits of reductionism.
    New York: McGraw-Hill; 2002.

36. Song, XJ, Gan, Q, Cao, J-L, Wang, Z-B, and Rupert, RL.
    Spinal Manipulation Reduces Pain and Hyperalgesia After
    Lumbar Intervertebral Foramen Inflammation in the Rat
    J Manipulative Physiol Ther. 2006 (Jan); 29 (1): 5–13

37. Gordis L.
    Epidemiology. 4th ed.
    Philadelphia, PA: Saunders (Elsevier); 2008.

38. Jordan W, Anand J.
    Pulmonary embolism.
    Lancet. 1961 Nov 18;278(7212):1146-7.

39. Centers for Disease Control (CDC).
    Pneumocystis pneumonia--Los Angeles.
    MMWR Morb Mortal Wkly Rep. 1981 Jun 5;30(21):250-2.

40. Lafond D, Champagne A, Cadieux R, Descarreaux M.
    Rehabilitation Program for Traumatic Chronic Cervical Pain
    Associated With Unsteadiness: A Single Case Study
    Chiropractic & Osteopathy 2008 (Nov 17); 16 (1): 15

41. BenEliyahu DJ.
    Magnetic Resonance Imaging and Clinical Follow-up:
    Study of 27 Patients Receiving Chiropractic Care
    for Cervical and Lumbar Disc Herniations
    J Manipulative Physiol Ther 1996 (Nov); 19 (9): 597—606

42. Smith M, Carber L.
    Chiropractic Health Care in Health Professional Shortage Areas
    in the United States.
    American Journal of Public Health. 2002;92(12):2001-9.

43. Cassidy JD, Boyle E, Cote P, et al.
    Risk of Vertebrobasilar Stroke and Chiropractic Care: Results of a Population-based
    Case-control and Case-crossover Study
    Spine (Phila Pa 1976) 2008 (Feb 15); 33 (4 Suppl): S176–183

44. Rochon P, Gurwitz J, Sykora K, Mamdani M, Streiner D, Garfinkel S, et al.
    Reader's guide to critical appraisal of cohort studies: 1.
    Role and design.
    BMJ (Clinical research ed). 2005;330(7496):895-7.

45. Grimes DA, Schulz KF.
    Bias and causal associations in observational research.
    Lancet. 2002;359(248-52).

46. Miller JE, Benfield K.
    Adverse Effects of Spinal Manipulative Therapy in Children Younger Than 3 Years:
    A Retrospective Study in a Chiropractic Teaching Clinic
    J Manipulative Physiol Ther 2008 (Jul); 31 (6): 419–423

47. Euser A, Zoccali C, Jager K, Dekker F.
    Cohort Studies: Prospective versus Retrospective.
    Nephron Clin Pract. 2009;113:c214-c7.

48. Cifuentes M, Willetts J, Wasiak R.
    Health Maintenance Care in Work-Related Low Back Pain and Its Association
    With Disability Recurrence
    J Occupational and Environmental Medicine 2011 (Apr); 53 (4): 396–404

49. Moher D, Hopewell S, Schulz KF, Montori V, Gřtzsche PC, Devereaux PJ, et al.
    CONSORT 2010 explanation and elaboration: updated guidelines for reporting
    parallel group randomised trials.
    Br Med J. 2010; 340:c869.

50. CONSORT GROUP.
    The CONSORT Statement 2010. Available from:
    http://www.consort-statement.org/

51. Bishop PB, Quon JA, Fisher CG, Dvorak MF.
    The Chiropractic Hospital-based Interventions Research Outcomes (CHIRO) Study:
    A Randomized Controlled Trial on the Effectiveness of Clinical Practice Guidelines
    in the Medical and Chiropractic Management of Patients with Acute Mechanical Low Back Pain
    Spine J. 2010 (Dec); 10 (12): 1055-1064

52. Centre for Evidence Based Medicine.
    What is EBM? 2009. Available from:
    http://www.cebm.net/?o=1913

53. Green S, Higgins JP, Alderson P, Clarke M, Mulrow CD, Oxman AD.
    Part 1 Cochrane Reviews:
    The Cochrane Collaboration; 2011 [updated 2011]. Available from:
    http://handbook.cochrane.org/

54. Moher D, Liberati A, Tetzlaff J, Altman DG (2009)
    Preferred Reporting Items for Systematic Reviews and Meta-Analyses:
    The PRISMA Statement
    Int J Surg 2010; 8 (5): 336–341

55. Hawk C, Knorsa R, Lisi A, Ferrance RJ, Evans MW.
    Chiropractic Care for Nonmusculoskeletal Conditions: A Systematic Review
    With Implications For Whole Systems Research
    J Altern Complement Med. 2007 (Jun); 13 (5): 491–512

56. Goertz M, Thorson D, Bonsell J, Bonte B, Campbel lR, Haake B, et al.
    Institute for Clinical Systems Improvement.
    Adult Acute and Subacute Low Back Pain. Updated. ICSI. November 2012.

57. Sackett DL., Rosenberg WMC., Muir Gray JA., Haynes RB., Richardson WS.
    Evidence-Based Medicine:
    What It Is and What It Isn't
    British Medical Journal 1996 (Jan 13); 312 (7023): 71–72

58. Diamond J, Robinson JA.
    Natural Experiments of History. Cambridge, MA:
    Harvard University Press; 2010.

59. Manchikanti L, Boswell M, Giordano J.
    Evidence-Based Interventional Pain Management: Principles, Problems,
    Potential and Applications.
    Pain Physician. 2007;10:329-56.

60. Amorin-Woods LG, Beck RW, Parkin-Smith GF, Lougheed J, Bremner AP.
    Adherence to Clinical Practice Guidelines Among Three Primary
    Contact Professions: A Best Evidence Synthesis of the Literature
    for the Management of Acute and Subacute Low Back Pain
    J Can Chiropr Assoc 2014 (Sept); 58(3): 220–237

Return to SUBLUXATION

Return to RESEARCH ARTICLES

Since 7-13-2016