Sophisticated Research Design in
Chiropractic and Manipulative Therapy;
“What You Learn Depends on How You Ask.”

Part C: Mixed Methods: “Why Can’t Science
And Chiropractic Just Be Friends?”

This section is compiled by Frank M. Painter, D.C.
Send all comments or additions to:     Frankp@chiro.org

FROM:   Chiropractic Journal of Australia 2016; 44 (2): 1–21 ~ FULL TEXT

OPEN ACCESS

---

Lyndon G. Amorin-Woods, BAppSci(Chiropractic), MPH

Senior Clinical Supervisor;
Murdoch University Chiropractic Clinic
School of Health Professions,
Discipline of Chiropractic Murdoch University South Street campus,
90 South Street, Murdoch,
Western Australia 6150

Enjoy Part 1:   Quantitative Research: Size Does Matter    Enjoy Part 2:   Qualitative Research: Quality vs. Quantity

---

Many commentators have recognised the limitations and inapplicability of the traditional quantitative pyramid hierarchy especially with respect to complementary and alternative (CAM) health care, observing the way Evidence-based Practice [EBP] is sometimes implemented is controversial, not only within the chiropractic profession, but in all other healthcare disciplines, including medicine itself. A phased approach to the development and evaluation of complex interventions can help researchers define the research process and complex interventions may require use of both qualitative and quantitative methods. The chiropractic profession has little to fear from evidence-based practice; in fact it should be used productively to improve patient care, clinical outcomes and the standing of the profession in the eyes of the public, other health professions and legislators.

Keywords Evidence-Based Practice; Mixed Methods; Research Design

---

INTRODUCTION

Many scientists have recognised the limitations and inapplicability of the traditional quantitative pyramid hierarchy especially with respect to complementary and alternative (CAM) health care, including chiropractic. Over the last decade some authors have suggested refinements of the model, for instance; in the place of an evidence hierarchy, Jonas [1] suggested the construction of an “evidence house” with “rooms” for different types of information and purposes and later presented a refined circular model. [1]

Jonas [1] observed:

“... the best evidence may be observational data from clinical practice that can estimate the likelihood of a patient's recovery in a realistic context. [2] Patient's illnesses are complex physical, psychological, and social experiences that cannot be reduced to single, objective measures. [3] Personal experience of illness might sometimes be captured only through qualitative research. [4] The “best” evidence thus may be the meaning that patients give to their illness and recovery. At other times, the “best” evidence may come from laboratory studies. [5] Arranging types of evidence in a “hierarchy” obscures the fact that sometimes the best evidence is not objective, not additive, and sometimes not even clinical”. [6]

This ‘house’ model evolved to be depicted by Walach et al as being circular instead of hierarchical. [7] This model was derived from the experience and history of evaluation methodology in the social sciences. [8, 9] Again, rather than postulating a single "best method" this view acknowledges that there are optimal methods for answering specific questions, and that a composite of all methods constitutes best scientific evidence. Sometimes there may exist nothing but expert opinion on a clinical question; thus, it becomes ‘best evidence.’ However when making decisions with respect to patient care, it is imperative to use the highest-level evidence available. When evidence is lacking, clinical decisions may still be made on biological plausibility, cost-effectiveness, feasibility and avoidance of harm. [10-12]

In chiropractic science, a frequent objection to RCT’s is that the chiropractic clinical encounter is so complex that it is impossible to identify controls as valid comparisons, especially that of designing a valid placebo for chiropractic spinal manipulative therapy (CSMT). [13-16] Complexity certainly presents challenges for those designing and conducting clinical trials however, this is by no means a challenge unique to chiropractic and should not be viewed as overly daunting. Clinical interventions in the fields of nursing, mental health and psychology are also inherently complex, but this has not prevented this sector from being extensively investigated via RCT’s. [17, 18]

Scientists in all the health care disciplines recognise that clinical encounters are composed of a kaleidoscope of constituent parts, which are able to act both independently and interdependently. These include client/patient behaviours, goals and expectations, and practitioner characteristics as well as locale, demographics, and personality; of both patients and clinicians. [19, 20] All these criteria are certainly characteristic of chiropractic practice.

Certainly controlled trials and other sophisticated research designs have been applied to chiropractic (and SMT) for many decades. There is now an impressive body of literature exploring the management of spinal pain which after all constitutes the vast majority of chiropractic practice. [21-23] There are more than 2500 controlled trials of treatments for back and neck pain listed in the Cochrane database, 32 Cochrane systematic reviews of randomised trials, 13 national clinical guidelines and 2 international guidelines. [24-27]

Other researchers went to some lengths to illustrate what may be to some, surprising limitations of RCT’s;

1) RCT’s usually lack external validity since they tend to employ such strict inclusion and exclusion criteria that the participants are not representative of the general population;

2) RCT’s may actually increase health risks in the general population since people with complex health problems are not usually participants in RCT’s;

3) The premise that RCT’s are the only form of evidence capable of providing an unbiased estimate of treatment effects is false [28];

4) The averaged results derived from RCT’s often offer insufficient or even incorrect guidance on how to approach a specific case [29]; and

5) The excessive expense of RCT’s leads to vulnerabilities in the quality of evidence. [30]

The payment of participants creates incentive to ‘bury’ negative reports and bias toward areas where commercial funding is possible (i.e. pharmaceuticals) and a brief intervention time to evaluate efficacy. [31]

In the later part of the last decade, as inadequacies in traditional RCT’s have become increasingly debated, a literal plethora of innovative designs have been proposed, summarised articulately by Rosner in a treatise on challenges faced by EBM. [32] These include; Pragmatic trials, in which the intervention is intended to represent “real-world” care; Factorial designs, comparing single modalities to a combination of modalities; Preference trials, in which participants with no treatment preferences are randomised as usual, but those with distinct preferences in their care receive their favoured treatment; and n-of-1, a single patient trial with multiple crossovers between a treatment period and a placebo or standard treatment period. [33]

Rosner goes on to summarise important RCT design evolution; ‘Randomised Encouragement’ design: to encourage adherence to the trial protocol, participants are either given strong incentives that are outside usual practice or are allowed to choose or decline a specific treatment to which they have been assigned [34]; Pragmatic clinical trials (PCT’s), which ask practical questions about the risks, benefits, and costs of intervention as they would occur in routine clinical practice [35]; Whole systems research (WSR), which uses observational studies and includes qualitative as well as quantitative research. [36] Finally, Patient-oriented evidence that matters (POEM) which focuses only on what is important useful information at the point of care. [37] There is a growing movement to employ these methodologies [38], aiming to examine all the influential variables (including physiological, psychological, spiritual, social, and personal preference/utility) and to recognise that each patient is unique with respect to these dimensions. [39, 40]

Then there is the known limitations phenomenon of A+B vs. B studies; where A+B is likely to show superior outcomes even if A is a placebo. There is however, emerging thought that intentionally choosing usual-care controls, even when a satisfactory placebo exists, may allow capture of non-specific therapeutic benefits that are common to all interventions. This is especially true if the primary intent is to explore the value of an intervention for improving the lives of patients; that is, a ‘pragmatic trial’. [41, 42]

Campbell et al recognised and outlined a phased approach to the development and evaluation of complex interventions to help researchers define clearly where they are in the research process and significantly, noted evaluation of complex interventions requires use of both qualitative and quantitative evidence. [43] Foster described the key features of high-quality randomised controlled trials;

1) sample representative of the patient group;

2) randomisation of individuals;

3) appropriate control or comparison treatment (sometimes a placebo);

4) concealment of the intervention from both patients and practitioners; and

5) an intention-to-treat analysis.

The methodological challenges posed by studying ‘complex’ interventions thus include the need to define the various components of the intervention including their anticipated specific and non-specific effects, determine the characteristics of patients that may respond to a multi-modal intervention and ensure consistent and highquality delivery of the treatment programme. [19, 44]

      Mixed Methods Research (MMR)

What is Mixed Methods Research?   Creswell [45] defines mixed methods research (MMR) as;

“… a method, that focuses on collecting, analysing and mixing both quantitative and qualitative data in a single study or series of studies. Its central premise is that the use of quantitative and qualitative approaches in combination provides a better understanding of research problems than either approach alone (45, p5)”.

Mixed methods research (MMR) thus provides many strengths that offset the weaknesses of either quantitative or qualitative designs, the key strengths being: (a) the voices of participants are directly 'heard' and recorded in qualitative research, (b) all of the potential tools of data collection are all available to the researcher when both methods are used, and (c) the researcher is enabled to utilise all the various types of data available to answer the research question. [46, 47] A prominent example is ‘Pragmatism’, with its focus on ‘what works’ which allows the researcher to move beyond philosophical questions about mixing or combining methods and allows for an integrated methodology for the health and social sciences. [48, 49]

      Aim

The aim of this paper was to identify various mixed methods models of research design and discuss their applicability to complex and sophisticated research in the complementary and alternative space, particularly chiropractic and manipulative.

---

METHOD

Five databases were searched in addition to ‘Google Scholar’ in December 2012. These databases included Medline, Web of Science, EMBASE, SportDiscus, and The Cochrane Library. The search criteria used were “complex research” OR “mixed methods” AND “chiropractic research” OR “complementary” AND “alternative” AND ‘‘model.’’

---

RESULTS

Various designs of mixed-methods design were identified broadly fitting four major types according to a typology described by Creswell and Plano-Clarke. [45] Note: examples are presented as representing study design and no evaluation is made of study quality. Abstracts are sometime quoted verbatim where indicated.


      The Four Major Types of Mixed Methods Designs

      1.   Triangulation Design

Figure 1a Figure 1b Figure 1c Figure 1d Figure 2a Figure 2b Figure 2c Figure 3a Figure 3b Figure 3c Figure 4a Figure 4b Figure 4c

The triangulation design is the most common and well-known MMR design, which was previously known as the ‘concurrent triangulation design’. [50] Researchers implement both methods during the same timeframe and with equal weight. The purpose is to obtain different but complimentary data on the same topic. The intent is to bring together the differing strengths of quantitative methods (large sample size, trends, generalisation) with those of qualitative methods (small n, details, in-depth perspectives). This design is used when a researcher wants to directly compare and contrast quantitative statistical results with qualitative findings or to validate or expand quantitative results with qualitative data. The traditional model of triangulation mixed methods design is the convergence model where integration occurs during the interpretation phase (Figures 1 a-d). [51]

CAM Example;

Kopansky-Giles [52], Teaching an inter-professional approach to the management of musculoskeletal problems in primary care – a pilot study.   “Pre-and-post program semi-structured focus groups with students were conducted to explore satisfaction with the program, and perceptions of program impact on the acquisition of collaborative competencies in MSK care. Key informant interviews with teachers were conducted to determine program content and the different educational approaches to be used. Inductive thematic analysis and triangulation of data sets was utilized to evaluate qualitative data. [52]


      2.   Embedded Design

The embedded design, is characterised by having one dominant method, whereas the other data set provides a secondary or supportive role where priority is given to the quantitative methodology, and the qualitative data set is subservient. [51] Experimental Embedded Design is the most commonly used variant of the embedded design and has qualitative data embedded within an experimental design (such as a true experiment or a quasi-experimental design) and the qualitative data set is subservient within that methodology. A variant is the ‘Correlational Embedded Design’ where researchers collect qualitative data as part of their study (Figures 2 a-c).

CAM Example;

GP attitudes and self-reported behaviour in primary care consultations for low back pain. A national UK-based survey of GPs and physiotherapists with an embedded qualitative study.   “This study qualitatively examined the attitudes and self-reported behaviour of GPs in relation to guideline adherence for patients with LBP”. [53]


      3.   Explanatory Design

These are two phase mixed-method designs; the purpose of which is to use qualitative data to help explain or build upon initial quantitative results (Figures 3 a-c). Sequential explanatory design consists of two phases, beginning with the quantitative phase and then the qualitative phase, which aims to explain or enhance the quantitative results. There are two main variants—the follow-up explanatory model and the participant selection model. Within the follow-up explanatory model, the researcher identifies specific quantitative findings, such as unexpected results, outliers or differences between groups that need further exploration using qualitative methodology, whereas the qualitative phase has priority in the participant selection model, and the purpose of the quantitative phase is to identify and purposefully select participants. [51]

CAM Example;

Ritenbaugh [54], Developing a patient-centered outcome measure for complementary and alternative medicine therapies.   “Patients receiving complementary and alternative medicine (CAM) therapies often report shifts in well-being that go beyond resolution of the original presenting symptoms. The authors undertook a research program to develop and evaluate a patient-centered outcome measure to assess the multidimensional impacts of CAM therapies, utilizing a novel mixed methods approach that relied upon techniques from the fields of anthropology and psychometrics”. [54]

      4.   Exploratory Design

The exploratory design is a sequential design where the first phase, qualitative, helps in the development of the quantitative phase. [51] These are two phase studies with the intent of using results of the first method (qualitative) to help develop or inform the second method (quantitative), based on premise that an exploration is needed – for, for example: measures or instruments are not available, the variables are unknown or there is no guiding framework or theory. Applications include; Instrument Development Model, Taxonomy Development Model (Figures 4 a-c).

CAM Example;

Jones-Harris [55], Are chiropractors in the UK primary healthcare or primary contact practitioners? a mixed methods study.   “A sequential study of exploratory design was used; this model is characterised by an initial phase of qualitative data collection and analysis that precedes and informs the quantitative phase of data collection and analysis. In this study, interviews with members of chiropractic teaching faculty were used to inform the development of a questionnaire used to survey the opinions of chiropractors in the UK.” [55]

---

Discussion

The way EBP is sometimes implemented is controversial, not only within the chiropractic profession, but in all other healthcare disciplines, including medicine itself. [56] Criticisms are well-documented in the healthcare literature, eloquently summarised for example by Mykhalovskiy and Weir. [57] RCT’s have long had strident critics, notably within mental health, nursing and the social sciences; fields not entirely dissimilar to chiropractic with respect to complex clinical encounters. Vocal among these are for example;

McManus [58],   “contrary to the wilder claims of their advocates, randomised controlled trials should not be construed as gold standards, the highest forms of knowledge on which evidence based medicine can be based. They are perhaps the smallest units of knowledge, consisting only of crude empiricism devoid of theory” [58] ...

and Rolfe [59];   “Therapeutic relationships are not merely more or less complex examples of a general and measurable intervention; they are unique instances that need to be understood in very different ways. That is not to say... that the RCT has no place ( in nursing), but that the RCT should know its place”. [59]

As the debate regarding the application of EBP in chiropractic continues, it is perhaps timely to reflect that if health care were to ubiquitously hold all treatments and interventions for any condition, in any profession, to ‘Level 1’ or high level evidence [or recommendation level ‘A’], that is in effect a proposal to revert most of healthcare back to the dark ages. Those who criticise the (lack of) scientific (RCT) basis of chiropractic would do well to reflect on the absence of (RCT) clinical trials of, for example; dental root canals and spinal surgery.

Alcantara [60] points out;

“the RCT design minimizes or excludes the impact of the doctor-patient relationship (i.e., the non-specific effects) on outcomes, while in chiropractic (as with other alternative therapies), the non-specific therapeutic effects of the clinical encounter is embraced and considered an important aspect to wellness care”. [60]

It is an unavoidable observation that research evidence and EBP is often ostensibly crafted to support political or policy agendas rather than to ensure equitable access for patients to care or to drive evidence-based healthcare services. For example, the scientific evidence for the treatment of low back pain is published in national and international clinical guidelines, yet adoption of these guidelines in clinical practice by relevant disciplines including physiotherapy and medicine is nonetheless very poor. [61-64] Consequently, there is a major drive to educate patients and practitioners about evidence-based management of spinal pain, with a view to translate research into practice. [24, 65] One can rest assured however; there will be an even greater focus on evidence-based treatment and practice in the emerging healthcare milieu of the future, with cost containment and streamlining of care practices being at the centre of attention. [66] The bottom line is this – EBP has gained favour across the majority of healthcare disciplines and organisations, and is here to stay for the foreseeable future. [67]

Some chiropractors may continue to rail against, and even fear of EBP, but this will do little to promote and advance the profession. This attitude is also misplaced since as this series points out, a substantial amount of research evidence already exists that supports many of the measures chiropractors use in day-to-day practice. [68, 69] Correspondingly, there is also an marked absence of evidence contrary to the chiropractic paradigm, even in areas for which there is not yet compelling supportive data, such as for non-musculoskeletal conditions, wellness/well-being and paediatric care. There is a marked difference between a question having been asked and so results are known, and the (right) question not been asked in the right way thus results not being available. For example; for some time it has been thought neck manipulation is no more effective than mobilisation (and was less risky) [70], however this belief is rapidly being superseded by emerging high level evidence. [71, 72] Thus, extant evidence should rather be used to promote and improve the professions’ profile using an EBP approach. It is, therefore worth reminding readers at this stage that EBP is not purely a research exercise; that is only one part of the EBP cycle, but rather a continuous process of; a) accessing and synthesising relevant healthcare, b) acting on the evidence by incorporating the evidence into practice, and c) evaluating if actions taken have had the desired effect in practice. [73-76]

I am thus of the view that the chiropractic profession should further embrace EBP; using the positive aspects of this approach to the advantage of the profession and setting aside unnecessary defiance of an approach that is already the most significant influencing factor in healthcare decision-making of our time. In other words, generate research evidence that can influence healthcare policy by demonstrating that a) chiropractic has a definite role to play within the boarder healthcare system, b) that chiropractors are amenable and able to work within multidisciplinary settings, and c) that chiropractors have effective and cost-effective care to offer patients. Translational research and the gathering of practice based evidence is understandably assuming an increasing importance. [65]

There are certainly those critical of the slower than ideal trend in increasing rigour of (CSMT) RCT’s, however an exciting part of contemporary research and EBP is that research designs can now measure the effectiveness of complex interventions – interventions where the outcome is a result of the interplay and effect of the various management components. [43, 77-79] This confirms that there are indeed research designs, in the form of pragmatic practice-based clinical trials that can test complex treatments. This type of research does not seek to explain the effect of an individual treatment, but rather the effect of a “package” of care and the patients’ “care journey”. [80] This is the type of research that can be embraced by the chiropractic profession to influence healthcare policy and clinical practice, using the EBP approach. Investigators should not be daunted by the challenge, rather embrace new and creative approaches to evaluating for example; ‘in situ’ clinical encounters such as in PBRN’s. [81, 82]

Chiropractic scientists are indeed rising to the challenge. Those who doubt the disproportionate influence of chiropractic researchers may have been somewhat surprised when The Spine Journal announced its "Top 25 Hottest Articles"; the most downloaded articles for 2011. Thirteen of the 25 articles have at least 1 author who was a chiropractor. In addition, 10 of the 25 include content relating to spinal manipulation or chiropractic,; 6 of which included the terms in their title. [83] A positive spin-off of evidence-based practice is that it can influence the evolution of the chiropractic profession by challenging many traditional practices, it questions use of interventions that are merely the preference of the practitioner and it helps evaluate the role of chiropractic within a contemporary healthcare landscape. As with all healthcare disciplines, the goal is to improve by, if necessary, adopting new ways of doing things and to set aside those practices that clearly are not effective or in the best interest of patient subgroups.

"If we are serious about coming to know something, then our research methods will have to be adapted to the nature of the phenomenon that we are trying to understand”. [84]

Landmark recent publications have built on evolving knowledge over the last decade to provide all health practitioners with guidance on how they can design and evaluate complex studies. [19, 85, 86] Chiropractic scientists have embraced this trend and this paper argues that the chiropractic profession should adopt such thinking more widely.

---

CONCLUSION

The examples presented in this series illustrate, what can be achieved by relatively few dedicated chiropractic research scientists with meagre resources. Complex challenges and limited funding are not, therefore, a reason to avoid sophisticated research. In fact, the complexity of chiropractic care and the ubiquitous lack of financial resources, make it even more important to use rigorous methods to conduct research. This series of papers has attempted to illustrate that a major objection to sophisticated research in chiropractic – that chiropractic is too complex to be studied, can be overcome by careful attention to methodology and detail; and utilisation of appropriate design. It is emphasised that the strength of inventive chiropractic research in the authors’ opinion lies in the combination of both qualitative and quantitative research, and in the gathering, replication and application of research findings in the ‘real world’ of clinical practice.

---

ACKNOWLEDGEMENTS

I would like to acknowledge Gregory F. Parkin-Smith, Anthony L. Rosner, and Randy W Beck for review of the manuscript

---

REFERENCES:

1. Jonas W  
   The Evidence House: How to Build an Inclusive
   Base for Complementary Medicine
   Western Journal of Medicine 2001 2001 (Aug); 175 (2): 79–80

2. Pincus T.
   Analyzing long-term outcomes of clinical care without randomized controlled clinical trials: the consecutive patient questionnaire database.
   Adv Mind Body Med 1997;13:3-31

3. Cassel E.
   The nature of suffering and the goals of medicine.
   N Engl J Med 1982;306(11):639-45

4. Feinstein A.
   "Clinical Judgment" revisited: the distraction of quantitative models.
   Ann Intern Med 1994;120(9):799-805

5. Yue Q, Bergquist C, Gerden B.
   Safety of St. John's wort.
   Lancet 2000;355:576-7

6. Linde K, Jonas W.
   Evaluating complementary and alternative medicine: the balance of rigor and relevance.
   In: Jonas WB, Levin JS, eds.
   Essentials of complementary and alternative medicine.
   Philadelphia: Lippincott Williams & Wilkins; 1999

7. Walach H, Falkenberg T, Fonnebo V, Lewith G, Jonas W.
   Circular instead of hierarchical: methodological principles for the evaluation of complex interventions.
   BMC Med Research Methodol 2006;6(1):29

8. Shadish WRJ., Cokk TD., Leviton LC., editors.
   Foundations of program evaluation.
   Theories of practice. Newbury Park, CA: Sage; 1991

9. Wittmann WW, Walach H.
   Evaluating complementary medicine: Lessons to be learned from evaluation research.
   In: Lewith G, Jonas WB, Walach H., editors.
   In Clinical research in complementary therapies: principles, problems, and solutions.
   London: Churchill Livingston; 2002:93-108

10. Triano, JJ.
    What Constitutes Evidence For Best Practice?
    J Manipulative Physiol Ther. 2008 (Nov); 31 (9): 637–643

11. Amorin-Woods L, Parkin-Smith G.
    Clinical Decision-making to Facilitate Appropriate
    Patient Managementin Chiropractic Practice:
    'The 3-questions Model'
    Chiropractic & Manual Therapies 2012 (Mar 14); 20: 6

12. Leboeuf-Yde C, Lanlo O, Walker BF:
    How to Proceed When Evidence-based Practice Is Required
    But Very Little Evidence Available?
    Chiropractic & Manual Therapies 2013 (Jul 10); 21 (1): 24

13. Martel J, Dugas C, Dubois JD, Descarreaux M.
    A Randomised Controlled Trial of Preventive Spinal Manipulation With and Without
    a Home Exercise Program for Patients With Chronic Neck Pain
    BMC Musculoskelet Disord. 2011 (Feb 8); 12: 41

14. Hancock MJ., Maher CG., Latimer J., McAuley JH.
    Selecting an appropriate placebo for a trial of spinal manipulative therapy.
    Aust J Physiother 2006;52(2):135-8

15. Kawchuk GN, Haugen R, Fritz J.
    A True Blind for Subjects Who Receive
    Spinal Manipulation Therapy
    Arch Phys Med Rehabil 2009 (Feb); 90 (2): 366–368

16. Hawk, C, Long, CR, Reiter, R, Davis, CS, Cambron, JA, and Evans, R.
    Issues in Planning a Placebo-controlled Trial of Manual Methods: Results of a Pilot Study
    J Altern Complement Med 2002; 8 (1) Feb: 21–32

17. Richards DA, Hamers JPH.
    RCTs in complex nursing interventions and laboratory experimental studies.
    Int J Nurs Stud 2009;46(4):588-92

18. Buchan H.
    Gaps between best evidence and practice: causes for concern.
    Med J Australia 2004;180:S48-S9

19. Medical Research Council.
    Developing and evaluating complex interventions: new guidance.
    London: Medical Research Council. 2008

20. Medical Research Council.
    A Framework for development and evaluation of RCTs for complex interventions to improve health.
    London: Medical Research Council. 2000

21. Sorensen LP, Stochkendahl MJ, Hartvigsen J, Nilsson NG.
    Chiropractic Patients in Denmark 2002: an expanded description and comparison with 1999 survey.
    J Manipulative Physiol Ther 2006;29(6):419-24

22. Sibbritt D, Adams J.
    Back pain amongst 8,910 young Australian women: a longitudinal analysis of the use of conventional providers, complementary and alternative medicine (CAM) practitioners and self-prescribed CAM.
    Clin Rheumatol 2010;29(1):25-32

23. Zhang A.
    Complementary and alternative medicine use in Australia: A national population-based study.
    Melbourne: RMIT University Research Repository; 2006

24. Department of Health WA.
    Spinal pain model of care.
    Perth: Health Networks Branch, Department of Health,
    Western Australia Perth, Western Australia: Health WA; 2009 [27th June 2015].
    Available from:
    http://www.healthnetworks.health.wa.gov.au/modelsofcare/docs/
    Spinal_Pain_Model_of_Care.pdf

25. Koes B, van Tulder M, Ostelo R, Burton A, Waddell G.
    Clinical Guidelines for the Management of Low Back Pain
    in Primary Care: An International Comparison
    SPINE (Phila Pa 1976) 2001 (Nov 15); 26 (22): 2504–2513

26. Koes BW, van Tulder M, Lin CW, Macedo LG, McAuley J, Maher C.
    An Updated Overview of Clinical Guidelines for the Management of Non-specific Low Back Pain
    in Primary Care
    European Spine Journal 2010 (Dec); 19 (12): 2075–2094

27. Foster N.
    Barriers and progress in the treatment of low back pain.
    BMC Me. 2011;9(1):108

28. Concato J., Shah N., Horwitz RI.
    Randomized, controlled trials, observational studies, and the hierarchy of research designs.
    N Engl J Med 2000;342:1887-92

29. Deisboeck TS.
    Personalizing medicine: a systems biology perspective.
    Mol Syst Biol 2009;5:249

30. Pharmaceutical Research and Manufacturers of America.
    Pharmaceutical Industry Profile 2010

31. Kaplan BJ., Giesbrecht G, Shannon S, McLeod K.
    Evaluating treatments in health care: The instability of a one-legged stool.
    BMC Med Research Methodol 2011;11:65

32. Rosner AL.
    Evidence-based Medicine: Revisiting the Pyramid of Priorities
    J Bodywork & Movement Therapies 2012 (Jan); 16 (1): 42–49

33. Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A.
    Complementary and alternative medicine whole systems research: beyond identification of inadequacies of the RCT.
    Complement Ther Med 2005;13 206e12

34. West SG, Duan N, Peguegnat W, Gaist P, Des Jarais DC, Holtgrave D, et al.
    Alternatives to the randomized controlled trial.
    Am J Public Health 2008. ;98(8):1359e66

35. Tunis SR, Stryer DB, Clancy CM.
    Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.
    JAMA 2003;290(12):1624e32

36. Hawk C, Khorsan R, Lisi AJ, Ferrance RJ, Evans MW.
    Chiropractic Care for Nonmusculoskeletal Conditions: A Systematic Review
    With Implications For Whole Systems Research
    J Altern Complement Med. 2007 (Jun); 13 (5): 491–512

37. White B.
    Making evidence-based medicine doable in everyday practice.
    Fam Pract Manag 2004;51:e58

38. Ritenbaugh C., Aickin M., Bradley R., Caspi O., Grimsgaard S., Musial F.
    Whole systems research becomes real: new results and next steps.
    J Altern Complement Me. 2010;;16((1):):131-7

39. Guastello S, Koopmans, M., Pincus, D. (eds)., .
    Chaos and Complexity in psychology: the theory of nonlinear dynamical systems:
    Cambridge University Press; 2008

40. Verhoef M., Lewith G., Ritenbaugh C., Thomas K., Boon H., Fønnebø V.
    Whole systems research: moving forward.
    Focus Altern Complement Ther 2004;9(2):87-90 [1]

41. Avins A, Cherkin D, Sherman K, Goldberg H, Pressman A.
    Should we reconsider the routine use of placebo controls in clinical research?
    Trials 2012;13(44)

42. Schwartz D, Lellouch J.
    Explanatory and pragmatic attitudes in therapeutical trials.
    J Chronic Dis 1967;20:637-48

43. Campbell M., Fitzpatrick R., Haines A., Kinmonth AL.
    Framework for design and evaluation of complex interventions.
    British Med J 2000;321:694-6

44. Foster N.
    Do RCTs provide the answer? The current state of play for musculoskeletal conditions and what is on the horizon.
    Clin Chiropr 2010;13(1):99-102

45. Creswell J, Plano Clark V.
    Designing and conducting mixed methods research.
    Thousand Oaks, CA: Sage Publications 2007

46. Creswell JW, Plano Clark VL.
    The nature of mixed methods research. designing and conducting mixed methods research. 2nd ed.
    Thousand Oaks, CA: Sage; 2011:1-18

47. Borkan JM.
    Mixed methods studies: a foundation for primary care research.
    Ann Fam Med 2004;2(1):4-6

48. Morgan DL.
    Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods.
    J Mix Methods Res 2007;1(1):48-76

49. Johnson RB, Onwuegbuzie AJ.
    Mixed methods research: a research paradigm whose time has come.
    Educ Res 2004;33(7):14-2.

50. Creswell J.
    Research design: qualitative, quantitative and mixed methods approaches.
    2nd ed. Thousand Oaks, CA: Sage Publications 2003

51. Doyle L, Brady A-M, Byrne G.
    An overview of mixed methods research.
    J Res Nurs 2009;14(2):175-185

52. Kopansky-Giles D, Peranson J, Reeves S. P05.68.
    Teaching an interprofessional approach to the management of musculoskeletal problems in primary care - a pilot study.
    BMC Compl Alt Med 2012;12(Suppl 1):P428

53. Corbett M, Foster N, Ong BN.
    GP attitudes and self-reported behaviour in primary care consultations for low back pain.
    Fam Pract 2009;26(5):359-364

54. Ritenbaugh C, Nichter M, Nichter M, Kelly K, Sims C, Bell I, et al.
    Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format.
    BMC Compl Alternative Med 2011;11(1):135

55. Jones-Harris A.
    Are chiropractors in the uk primary healthcare or primary contact practitioners?: a mixed methods study.
    Chiropr Osteop 2010;18(1):28

56. Feinstein AR., Horwitz RI.
    Problems in the "evidence" of "evidence-based medicine".
    Am J Med 1997;103:529-35

57. Mykhalovskiy E., Weir L.
    The problem of evidence-based medicine: directions for social science.
    Soc Sci Med 2004;59:1059-69

58. McManus C.
    Engineering quality in health care.
    Qual Health Care 1996;5(3):127

59. Rolfe G.
    Complexity and uniqueness in nursing practice: Commentary on Richards and Hamers.
    Int J Nurs Stud 2009;46 1156-1158

60. Alcantara J.
    The chiropractic care of children: an open response to Chiropractic & Manual Therapy’s thematic series on pediatric chiropractic.
    J Pediatric Maternal Fam Health 2011

61. Armstrong MP., McDonough S., Baxter GD.
    Clinical guidelines versus clinical practice in the management of low back pain.
    int J Clin Pract 2003;57(1):9-13

62. Di Iorio D., Henley E., Doughty A.
    A survey of primary care physician practice patterns and adherence to acute low back problem guidelines.
    Arch Fam Med. 2000;9(10)

63. Scott NA., Moga C., Harstall C.
    Managing low back pain in the primary care setting: The know-do gap.
    Pain Res Manage 2010;15(6):392

64. Amorin-Woods LG, Beck RW, Parkin-Smith GF, Lougheed J, Bremner AP.
    Adherence to Clinical Practice Guidelines Among Three Primary
    Contact Professions: A Best Evidence Synthesis of the Literature
    for the Management of Acute and Subacute Low Back Pain
    J Can Chiropr Assoc 2014 (Sept); 58(3): 220–237

65. Green LW.
    Making research relevant: if it is an evidence-based practice, where's the practice-based evidence?
    Fam Pract 2008;25(suppl_1):i20-4

66. Parkin-Smith G, Amorin-Woods L, Davies S, Losco B, Adams J.
    Spinal pain: current understanding, trends, and the future of care.
    J Pain Res 2015;8:741-52

67. Cooper C.
    Transforming health care through the use of evidence-based practice.
    Nursing Excellence 1998;1(1)

68. Bishop PB, Quon JA, Fisher CG, Dvorak MF.
    The Chiropractic Hospital-based Interventions Research Outcomes (CHIRO) Study:
    A Randomized Controlled Trial on the Effectiveness of Clinical Practice Guidelines
    in the Medical and Chiropractic Management of Patients with Acute Mechanical Low Back Pain
    Spine J. 2010 (Dec); 10 (12): 1055-1064

69. COST13.
    European guidelines for the management of low back pain.
    Eur Spine J 2006;15((Suppl. 2):S125-S7)

70. Walker BF, French SD.
    Pain in the Neck: Many (Marginally Different) Treatment Choices.
    Ann Int Med 2012;156(1_Part_1):52-3

71. Dunning JR., Cleland JA., Waldrop M, Arnot C, Young I, Turner M, et al.
    Upper cervical and upper thoracic thrust manipulation versus non-thrust mobilization in patients with mechanical neck pain: a multi-center randomized clinical trial.
    J Orthop Sports Phys Ther 2012;42(1):5-18

72. Dunning JR, Butts R, Mourad F, Young I, Fernandez-de-las Peñas C, Hagins M, et al.
    Upper Cervical and Upper Thoracic Manipulation Versus Mobilization and Exercise
    in Patients with Cervicogenic Headache: A Multi-center Randomized Clinical Trial
    BMC Musculoskelet Disord. 2016 (Feb 6); 17 (1): 64

73. Guyatt G., Sackett D., Cook D.
    User's guide to the medical literature: II. How to use an article about therapy.
    JAMA 1993:270

74. Sackett DL., Rosenberg WMC., Muir Gray JA., Haynes RB., Richardson WS.
    Evidence-Based Medicine:
    What It Is and What It Isn't
    British Medical Journal 1996 (Jan 13); 312 (7023): 71–72

75. Green LW, Glasgow RE.
    Evaluating the relevance, generalization, and applicability of research.
    Eval Health Prof 2006;29(1):126-53

76. Eddy DM.
    Evidence-based medicine: a unified approach.
    Health Affairs 2005;24(1):9-17

77. Rubinstein SM, Terwee CB, de Boer MR, van Tulder MW.
    Is the methodological quality of trials on spinal manipulative therapy for low-back pain improving?
    Int J Osteopath Med 2012;15(2):37-52

78. Bonetti D, Eccles M, Johnston M, Steen M, Grimshaw J, Baker R, et al.
    Guiding the design and selection of interventions to influence the implementation of evidence-based practice: an experimental simulation of a complex intervention trial.
    Soc Sci Med 2005;60(9 Epub):2135-47

79. Shiell A., Hawe P., Gold L.
    Complex interventions or complex systems? Implications for health economic evaluation.
    BM. 2008;336:1281-3

80. Amorin-Woods L., Parkin-Smith G., Cascioli V., Kennedy D., editors.
    An exploratory (pilot) non-randomised controlled trial examining chiropractic care within a therapeutic community in terms of student and patient oriented outcomes.”
    The Palmerston-Murdoch Pilot Project; P-MPP”.
    “Making it happen: improving services through collaboration”
    AOD conference proceedings; 2012 (August 6-8); Perth, Australia

81. Amorin-Woods L, Parkin-Smith G, Cascioli V, Kennedy D.
    Manual care of residents with spinal pain within a therapeutic community.
    Ther Communities 2016;In Press.

82. Adams J, Steel A, Chang S, Sibbritt D.
    Helping address the national research and research capacity needs of Australian chiropractic: introducing the Australian Chiropractic Research Network (ACORN) project.
    Chiropr Man Therap 2015;23(12)

83. The Spine Journal 25 Hottest Articles January to December 2011 2012 (June).
    Available from:
    http://top25.sciencedirect.com/subject/medicine-and-dentistry/17/journal/
    the-spine-journal/15299430/archive/36/

84. Baruss I.
    Authentic Knowing: The convergence of science and spiritual aspiration.
    Lafayette, Ind: Purdue University Press; 1996

85. Campbell NC, Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H.,Wilson, P., Kinmouth, A.M.,
    Designing and evaluating complex interventions to improve health care.
    BMJ 2007;334:455-9

86. Craig P, Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M.,
    Developing and evaluating complex interventions: the new Medical Research Council guidance.
    BMJ 2008;337:a1655

Return to SUBLUXATION

Return to RESEARCH ARTICLES

Since 7-16-2016