RESEARCHING THE APPROPRIATENESS OF CARE IN THE COMPLEMENTARY AND INTEGRATIVE HEALTH PROFESSIONS PART 3: DESIGNING INSTRUMENTS WITH PATIENT INPUT
 
   

Researching the Appropriateness of Care in the Complementary
and Integrative Health Professions Part 3:
Designing Instruments With Patient Input

This section is compiled by Frank M. Painter, D.C.
Send all comments or additions to:
   Frankp@chiro.org
 
   

FROM:   J Manipulative Physiol Ther. 2019 (Jun);   42 (5):   307–318 ~ FULL TEXT

Margaret D. Whitley, MPH, Ian D. Coulter, PhDb, Ryan W. Gery, PhD, Ron D. Hays, PhD,
Cathy Sherbourne, PhD, Patricia M. Herman, PhD, Lara G. Hilton, PhD

RAND Corporation,
University of California Los Angeles,
Southern California Health Sciences,
Santa Monica, California.
coulter@rand.org.


OBJECTIVES:   The purpose of this article is to describe how we designed patient survey instruments to ensure that patient data about preferences and experience could be included in appropriateness decisions. These actions were part of a project that examined the appropriateness of spinal manipulation and mobilization for chronic low back pain and chronic neck pain.

METHODS:   We conducted focus groups, cognitive interviews, a literature review of measures in prior chiropractic and complementary and integrative health research, and a pilot study to develop questionnaires of patient preferences, experiences, values, and beliefs.

RESULTS:   Questionnaires were administered online to 2,024 individuals from 125 chiropractic clinics. The survey included 3 long questionnaires and 5 shorter ones. All were administered online. The baseline items had 2 questionnaires that respondents could complete in different sittings. Respondents completed shorter biweekly follow-ups every 2 weeks and a final questionnaire at 3 months. The 2 initial questionnaires had 81 and 140 items, the 5 biweekly follow-up questionnaires had 37 items each, and the endline questionnaire contained 121 items. Participants generally responded positively to the survey items, and 91% of the patients who completed a baseline questionnaire completed the endpoint survey 3 months later. We used "legacy" measures, and we also adapted measures and developed new measures for this study. Preliminary assessment of reliability and validity for a newly developed scale about coping behaviors indicates that the items work well together in a scale.

CONCLUSIONS:   This article documents the challenges and the efforts involved in designing data collection tools to facilitate the inclusion of patient data into appropriateness decisions.

KEYWORDS:   Chiropractic; Chronic Pain; Complementary Therapies; Low Back Pain; Neck Pain; Pain; Surveys and Questionnaires



From the FULL TEXT Article:

Introduction

Appropriateness of care decisions have been based on the published literature on safety and efficacy and the judgments of experts, both clinical and scientific experts. What is missing is the voice of patients in this process. However, in an era of patient-centered care reflected in organizations such as the Patient-Centered Outcome Research Institute, [1] inclusion of patient input should be considered essential. The Center of Excellence for Research in Complementary and Alternative Medicine (CERC) was established at RAND specifically to develop a method for studies on appropriateness that included patient input and costs. [2] Although it is now self-evident that patient input should play a role in decisions that affect them, it is important to do that while at the same time ensuring the decisions are clinically appropriate and safe. In developing a method at RAND/University of California Los Angeles to measure appropriateness (the RAND/University of California Los Angeles Appropriateness Method), [3, 4] considerable effort was made to make sure the decisions were evidence based or based on clinical experience that could be agreed upon by a panel of experts. Hence, the patient component should be equally evidence based, that is, based on actual data collected from patients.

The CERC national study collected data to assess patient beliefs and preferences, patient- reported outcomes, costs, and resource allocation. These data were provided to the study’s expert panels so that expert panelists could take these findings into account when determining their ratings about the appropriateness of manipulation and mobilization for chronic low back and neck pain. [5, 6]

Although we describe experiences from a research study, our lessons learned may be applicable to complementary and integrative health (CIH) providers as well. Complementary and Integrative Health providers and researchers both need rigorous patient measures to help them collect reliable and valid data that are relevant and not burdensome to patients.Our research team prioritized parsimony and survey items that were relevant to our respondents to improve participation rates and engagement with the study.

In this paper, we share lessons learned from our literature review, cognitive interviews, pilot study, and national study about (1) how to identify appropriate existing instruments to measure beliefs, preferences, and experiences with chronic pain and coping among CIH patients; (2) how to decide whether to modify a tool to better fit one’s study or clinical circumstance; (3) how to develop new measures and evaluate their reliability and validity; and (4) how to assemble multiple measures together into a single questionnaire.

      The Problem

Researchers need rigorous methodologies and reliable and valid self-report measures to evaluate the efficacy and effectiveness of therapies and to understand patients’ experiences and beliefs in chiropractic and other areas of CIH. Rigorous patient measures are also essential for clinicians. Clinicians may want information about patients’ perceptions of care, adherence to recommendations, and health-related quality of life. [7–10] Measures can be useful as a research tool and for patient care. The past half century has seen a gradual shift away from exclusive reliance on clinical and laboratory measures of illness or disease-specific outcome measures toward the development and utilization of comprehensive indices of patient health status, including patient self-report (what might be considered a more holistic approach to measurement). [11, 12]

There are various options when creating a patient survey:

  • Using all or part of existing measures

  • Revising items from an existing measure

  • Creating new items

Many researchers and clinicians prefer to use existing measures that have already been evaluated and published because this is often more efficient than designing a tool from scratch. [13, 14] Using an existing tool enables them to compare results to other studies and other practitioners that used the same tool, and this provides a helpful point of comparison across research or patient subgroups. A challenge is that there are few existing measures of patient beliefs and coping that have been evaluated in chiropractic. It is important for CIH researchers and practitioners to know how to find measures, decide whether a measure will suit their needs, and understand other options if no existing measures are appropriate.

Complementary and integrative health researchers may opt to design their own data collection tools. This has the major advantage of enabling them to ask exactly the question they want, perhaps addressing a topic that no one else has attempted to rigorously study or measure. It gives them the opportunity to cover all their domains of interest, and to word items in a way that will make sense and be relevant to their target population.

Creating an instrument is a complicated and lengthy process. How can a researcher or clinician be sure he or she has identified all the relevant domains that a tool should capture? Which are the best response options to use and how will they affect analytic options later? How can one feel confident that their respondents will understand and respond to the items in the way that the study team intends? If researchers want to measure multiple constructs using multiple tools within the same questionnaire, how can they make sure that fatigue or confusion among respondents are not adversely affecting their responses? Lastly, how can researchers test a novel set of items to be confident that the items are reliable and valid? There are ways to address all these questions, but they require careful planning.

To summarize, our primary questions in this study were as follows:

1.   How do we choose instruments that are patient-centric and relevant to their experiences and that will capture their preferences and values?

2.   How do we choose between, on one hand, utilizing legacy measures [15–17] that have been widely used in previous studies so we can compare our study to previous work, and on the other hand, designing new instruments specific to this study?

3.   How do we choose data collection instruments that are comprehensive but concise, reliable, valid, relevant, and nonburdensome?

      The Solution

In the following sections are the steps we took and the solutions we arrived at in answering the 3 questions above. In the results section, we discuss what the outcomes were.



Discussion

We noted earlier that this study set out to answer 3 main questions regarding collecting data from chiropractic practices:

1.   How do we choose instruments that are patient-centric and relevant to their experiences and that will capture their preferences and values?

2.   How do we choose between, on one hand, using existing (legacy) measures that have been widely used in previous studies so we can compare our study to previous work and, on the other hand, designing new instruments specific to this study?

3.   How do we choose data collection instruments that are comprehensive but concise, reliable, valid, relevant, and nonburdensome?

As noted here, answering those 3 questions requires considerable effort and multi-method solutions: a literature review, exploratory interviews, focus groups, cognitive interviews, a pilot study, and a national survey. At first the questions seem quite simple, but they are woven into the broader research question of this Center. In this instance, the broader question is the appropriateness of chiropractic manipulation and mobilization in the treatment of chronic low back and neck pain. This is only the second study ever conducted in chiropractic to try to calculate a rate of appropriate care. Traditionally, appropriate care was thought to be that which was efficacious and safe, and this was decided by research, researchers, and clinicians. [4, 43] However, the question is now being raised in an era of patient-centered care and patient-centered outcomes or outcomes that are significant to the patient. [44] That raises a further question of what patient data can be collected within chiropractic treatment clinics that will allow researchers to answer that. In that broader context, the question changes: How do we ensure that evidence-based practice is truly practice-based evidence? The significance of the study goes well beyond the 3 simple questions posed in this paper, but without solutions to those 3, the broader questions cannot be answered.

This study has demonstrated that it is possible to collect a wealth of patient-centered data from chiropractic clinics, but the process is multifaceted and quite demanding in effort and resources. It can be difficult to find appropriate measurement tools to use with chiropractic or CIH patients. There may not be existing questionnaires that address a particular study’s constructs of interest, or existing tools may not apply to or have been validated with this population. Here we have shared our experiences with developing a questionnaire to assess multiple domains of patient experiences, beliefs, and preferences about chiropractic care for chronic pain to demonstrate a set of approaches that researchers can use to identify and create tools.

We have also presented a detailed example showing how we used these methods to create a scale for measuring coping behaviors. We showed how the exploratory data collection and literature review findings, combined with logical assumptions, led us to identify key domains and key patient perspectives that we needed to capture if we wanted to understand what patients do from day to day to cope with their pain. From there, we created and tested 21 items for the general pain population and 5 additional items for people who are employed. Although the validity analysis is ongoing, we have presented preliminary findings using data from our national study to show that 19 of 21 general population items worked well together in a scale and had acceptable reliability. We believe these novel items are a useful contribution to the existing array of legacy measures related to coping with chronic pain, such as the Chronic Pain Self-Efficacy Scale. [27]

The purpose of describing these methods is to encourage researchers and clinicians to consider the many possible approaches at their disposal for collecting information from patients. We are not suggesting that everyone should combine all these approaches the way we did, but rather that they should think carefully about which approach fits their needs best.

Limitations

The limitations included that this study was conducted only within the United States. Although this should not affect the methods used in that many of the instruments selected were not developed in the United States and have been used on other populations (and the literature review was not restricted to US articles), it does mean this study was only focused on the United States.

Although not a limitation for this study and the results reported here, the generous funding of this study by the National Center for Complementary and Integrative Health would make it difficult for others to replicate the approach we have describedat least in its totality. We were able to use a very comprehensive approach to develop the instruments (literature reviews, cognitive interviews, a pilot study, a national online survey, etc.). Although this type of approach will ensure that the instruments developed have been rigorously tested, clearly this level of work would be beyond most research projects in CIH. At its peak, some 16 researchers were employed on this project. This level of funding for chiropractic research to date has not been replicated outside of the United States.



Conclusion

It is important to collect valid data about patients’ experiences and beliefs for research and clinical care. In many instances, as with our study, the best approach may be to use existing measures for some constructs, to modify existing measures for other constructs, and to create entirely new measures for constructs where the existing measures are insufficient. In this article, we have described how we used multiple qualitative methods and a review of the literature to identify constructs and then design questionnaires that were successfully administered as part of a national survey of chiropractic patients with chronic low back and neck pain. We have presented preliminary reliability and validity data for one of our novel measures, which addresses coping behaviors. We have also outlined suggestions for CIH researchers and providers who want to collect this sort of information from patients.


Practical Applications

  • This study describes approaches that can be used to find or develop patient survey instruments for complementary and integrative health.

  • These approaches were successfully applied in a national study of chiropractic patients with chronic pain.

  • The findings will be of interest to researchers and clinicians in the complementary and integrative health professions who want to collect data about patient preferences, experiences, and beliefs.


Funding Sources and Conflicts of Interest

This study was funded by the National Institutes of Health’s National Center for Complementary and Integrative Health Grant No: 1U19AT007912-01. All authors report that they were funded by a grant from the National Center for Complementary and Integrative Health during the study. No conflicts of interest were reported for this study.


Contributorship Information

Concept development   (provided idea for the research): M.D.W., I.D.C., P.M.H. Design   (planned the methods to generate the results): M.D.W., R.D.H., C.S. Supervision   (provided oversight, responsible for organization and implementation, writing of the manuscript): M.D.W., I.D.C. Data collection/processing   (responsible for experiments, patient management, organization, or reporting data): M.D.W., I.D.C., R.W.G., L.G.H. Analysis/interpretation   (responsible for statistical analysis, evaluation, and presentation of the results): M.D.W., R.D.H., C.S. Literature search   (performed the literature search): R.W.G., C.S., L.G.H. Writing   (responsible for writing a substantive part of the manuscript): M.D.W., I.D.C., R.D.H. Critical review   (revised manuscript for intellectual content, this does not relate to spelling and grammar checking): R.W.G., R.D.H., C.S., P.M.H., L.G.H.



References:

  1. Patient-Centered Outcomes Research Institute (PCORI). (Available at:)
    https://www.pcori.org/
    Date accessed: November 19, 2018

  2. Coulter, I, Herman, PM, Ryan, GG, Hays, RD, Hilton, LG, and Whitley, MD.
    Researching the appropriateness of care in the complementary and integrative health professions: part I.
    J Manipulative Physiol Ther. 2018; 41: 800–806

  3. Coulter, I, Elfenbaum, P, Jain, S, and Jonas, W.
    SEaRCH™ expert panel process: streamlining the link between evidence and practice.
    BMC Res Notes. 2016; 9: 16

  4. Fitch, K, Berstein, SJ, Aguilar, MD et al.
    The RAND/UCLA Appropriateness Method User’s Manual.
    RAND Corporation, Santa Monica, CA; 2001

  5. Coulter, I, Whitley, M, Hurwitz, E, Vernon, H, Shekelle, P, and Herman, PM.
    Determining the Appropriateness of Spinal Manipulation and Mobilization for Chronic Low Back Pain:
    Indications and Ratings by a Multidisciplinary Expert Panel.
    RAND Corp, Santa Monica, CA; 2018

  6. Coulter, I, Whitley, M, Vernon, H, Hurwitz, E, Shekelle, P, and Herman, PM.
    Determining the Appropriateness of Spinal Manipulation and Mobilization for Chronic Neck Pain:
    Indications and Ratings by a Multidisciplinary Expert Panel.
    RAND Corp, Santa Monica, CA; 2018

  7. Fung, CH and Hays, RD.
    Prospects and challenges in using patient-reported outcomes in clinical practice.
    Qual Life Res. 2008; 17: 1297–1302

  8. Deshpande, PR, Rajan, S, Sudeepthi, BL, and Abdul Nazir, CP.
    Patient-reported outcomes: a new era in clinical research.
    Pers Clin Res. 2011; 2: 137–144

  9. Bitton, A, Onega, T, Tosteson, ANA, and Haas, JS.
    Toward a better understanding of patient-reported outcomes in clinical practice.
    Am J Manag Care. 2014; 20: 281–283

  10. Snyder, CF, Aaronson, NK, Choucair, AK et al.
    Implementing patient-reported outcomes assessment in clinical practice:
    a review of the options and considerations.
    Qual Life Res. 2012; 21: 1305–1314

  11. Khorsan, R, Coulter, ID, Hawk, C, and Choate, CG.
    Measures in chiropractic research: choosing patient-based outcome assessments.
    J Manipulative Physiol Ther. 2008; 31: 355–375

  12. Wood-Dauphinee, S.
    Assessing quality of life in clinical research: from where have we come and where are we going?.
    J Clin Epidemiol. 1999; 52: 355–363

  13. Cherepanov, D and Hays, R.
    Health and quality-of-life-outcomes: the role of patient-reported measures.
    in: Outcomes Measurement in the Human Services:
    Cross-Cultural Issues and Methods in the Era of Health Reform. 2nd ed.
    NASM Press, Washington, DC; 2011

  14. Ganz, P, Hays, RD, Kaplan, RM, and Litwin, MS.
    Measuring health-related quality of life and other outcomes.
    in: GG Kominski (Ed.) Changing the U.S. Health Care System. 4th ed.
    Jossey-Bass, San Francisco, CA; 2014: 307–341

  15. Shahgholi, L, Yost, KJ, Carter, RE et al.
    Correlation of the Patient Reported Outcomes Measurement Information System with legacy outcomes
    measures in assessment of response to lumbar transforaminal epidural steroid injections.
    Am J Neuroradiol. 2015; 36: 594

  16. Sikorskii, A, Victorson, D, O’Connor, P et al.
    PROMIS and legacy measures compared in a supportive care intervention for breast cancer patients
    and caregivers: experience from a randomized trial.
    Psychooncology. 2018; 27: 2265–2273

  17. Clover, K, Lambert, SD, Oldmeadow, C et al.
    PROMIS depression measures perform similarly to legacy measures relative to a structured diagnostic
    interview for depression in cancer patients.
    Qual Life Res. 2018; 27: 1357–1367

  18. Ryan, GW and Bernard, HR.
    Techniques to identify themes.
    Field Methods. 2003; 15: 85–109

  19. Spiegel, BMR, Hays, RD, Bolus, R et al.
    Development of the NIH Patient-Reported Outcomes Measurement Information System (PROMIS)
    Gastrointestinal Symptom Scales.
    Am J Gastroenterol. 2014; 109: 1804–1814

  20. Maida, CA, Marcus, M, Hays, RD et al.
    Child and adolescent perceptions of oral health over the life course. vQual Life Res. 2015; 24: 2739–2751

  21. Fairbank, J, Couper, J, Davies, J, and O’Brien,
    J. The Oswestry Low Back Pain Disability Questionnaire.
    Physiotherapy. 1980; 66: 271–273

  22. Fairbank, JC and Pynsent, PB.
    The Oswestry Disability Index.
    Spine. 2000; 25: 2940–2953

  23. Vernon, H and Mior, S.
    The Neck Disability Index: a study of reliability and validity.
    J Manipulative Physiol Ther. 1991; 14: 409–415

  24. National Institutes of Health (NIH).
    Patient-Reported Outcomes Measurement Information Systems (PROMIS®). (Available at:)
    https://commonfund.nih.gov/promis/index
    Date accessed: November 19, 2018

  25. Jensen, MP, Karoly, P, and Braver, S.
    The measurement of clinical pain intensity: a comparison of six methods.
    Pain. 1986; 27: 117–126

  26. Jensen, MP, Turner, JA, Romano, JM, and Lawler, BK.
    Relationship of pain-specific beliefs to chronic pain adjustment.
    Pain. 1994; 57: 301–309

  27. Anderson, KO, Dowds, BN, Pelletz, RE, Edwards, WT, and Peeters-Asdourian, C.
    Development and initial validation of a scale to measure self-efficacy beliefs in patients with chronic pain.
    Pain. 1995; 63: 77–83

  28. Morone, NE, Greco, CM, Moore, CG et al.
    A mind-body program for older adults with chronic low back pain: a randomized clinical trial.
    JAMA Intern Med. 2016; 176: 329–337

  29. Shpaner, M, Kelly, C, Lieberman, G et al.
    Unlearning chronic pain: a randomized controlled trial to investigate changes in intrinsic brain
    connectivity following cognitive behavioral therapy.
    Neuroimage Clin. 2014; 5: 365–376

  30. Williams, KA, Petronis, J, Smith, D et al.
    Effect of Iyengar yoga therapy for chronic low back pain.
    Pain. 2005; 115: 107–117

  31. Devilly, GJ and Borkovec, TD.
    Psychometric properties of the credibility/expectancy questionnaire.
    J Behav Ther Exp Psychiatry. 2000; 31: 73–86

  32. The Center for Epidemiologic Studies Depression Scale Revised. (Available at:)
    (Accessed October 23, 2018)
    http://cesd-r.com/

  33. Hays, RD, Weech-Maldonado, R, Teresi, JA, Wallace, SP, and Stewart, AL.
    Commentary: copyright restrictions versus open access to survey instruments.
    Med Care. 2018; 56: 107–110

  34. Fowler, FJ.
    Improving Survey Questions: Design and Evaluation.
    SAGE Publications, Thousand Oaks, CA; 1995

  35. Johnson RL, Morgan GB.
    Survey Scales: A Guide to Development, Analysis, and Reporting.
    New York, NY: Guilford Publications; 2016.

  36. Willis GB, Artino AR Jr.
    What do our respondents think we’re asking? Using cognitive interviewing to improve medical education surveys.
    J Grad Med Educ 2013;5(3):353-356.

  37. Barry, LC, Kerns, RD, Guo, Z, Duong, BD, Iannone, LP, and Reid, MC.
    Identification of strategies used to cope with chronic pain in older persons receiving primary care
    from a Veterans Affairs Medical Center.
    J Am Geriatr Soc. 2004; 52: 950–956

  38. Fleishman, JA, Sherbourne, CD, Cleary, PD, Wu, AW, Crystal, S, and Hays, RD.
    Patterns of coping among persons with HIV infection: configurations, correlates, and change.
    Am J Community Psychol. 2003; 32: 187–204

  39. R.A. Deyo, S.F. Dworkin, D. Amtmann, G. Andersson, et al.,
    Report of the NIH Task Force on Research Standards for Chronic Low Back Pain
    Journal of Pain 2014 (Jun);   15 (6):   569–585

  40. Singleton, R Jr. and Straits, BC.
    Approaches to Social Research. 5th ed.
    Oxford University Press, Oxford, England; 2010

  41. Cronbach, LJ.
    Coefficient alpha and the internal structure of tests.
    Psychometrika. 1951; 16: 297–334

  42. Krosnick, JA and Alwin, DF.
    An evaluation of a cognitive theory of response-order effects in survey measurement.
    Public Opin Q. 1987; 51: 201–219

  43. Brook, RH, Chassin, MR, Fink, A, Solomon, DH, Kosecoff, J, and Park, RE.
    A method for the detailed assessment of the appropriateness of medical technologies.
    Int J Technol Assess Health Care. 2009; 2: 53–63

  44. World Health Organization Regional Office for Europe.
    Appropriateness in health care services: report on a WHO workshop, 23–25 March 2000.
    Koblenz, Germany. WHO Regional Office for Europe, Copenhagen, Denmark; 2000

  45. Von Korff, M, Ormel, J, Keefe, FJ, and Dworkin, SF.
    Grading the severity of chronic pain.
    Pain. 1992; 50: 133–149

  46. Linton, SJ and Halldén, K.
    Can we screen for problematic back pain? A screening questionnaire for predicting outcome
    in acute and subacute back pain.
    Clin J Pain. 1998; 14: 209–215

  47. Lorig, K, Chastain, RL, Ung, E, Shoor, S, and Holman, HR.
    Development and evaluation of a scale to measure perceived self-efficacy in people with arthritis.
    Arthritis Rheum. 1989; 32: 37–44

  48. Symonds, T, Burton, AK, Tillotson, KM, and Main, CJ.
    Do attitudes and beliefs influence work loss due to low back trouble?.
    Occup Med. 1996; 46: 25–32

  49. Calin, A, Garrett, S, Whitelock, H et al.
    A new approach to defining functional ability in ankylosing spondylitis:
    the development of the Bath Ankylosing Spondylitis Functional Index.
    J Rheumatol. 1994; 21: 2281–2285

  50. Beck, AT, Epstein, N, Brown, G, and Steer, RA.
    An inventory for measuring clinical anxiety: psychometric properties.
    J Consult Clin Psychol. 1988; 56: 893

  51. Beck, AT, Ward, CH, Mendelson, M, Mock, J, and Erbaugh, J.
    An inventory for measuring depression.
    Arch Gen Psychiatry. 1961; 4: 561–571

  52. Borg, G.
    Perceived exertion as an indicator of somatic stress.
    Scand J Rehabil Med. 1970; 2: 92–98

  53. Bolton, JE and Breen, AC.
    The Bournemouth Questionnaire: a short-form comprehensive outcome measure. I.
    Psychometric properties in back pain patients.
    J Manipulative Physiol Ther. 1999; 22: 503–510

  54. Radloff, LS.
    The CES-D scale: a self-report depression scale for research in the general population.
    Appl Psychol Meas. 1977; 1: 385–401

  55. McCracken, LM, Vowles, KE, and Eccleston, C.
    Acceptance of chronic pain: component analysis and a revised assessment method.
    Pain. 2004; 107: 159–166

  56. Jensen, MP, Turner, JA, Romano, JM, and Strom, SE.
    The chronic pain coping inventory: development and preliminary validation.
    Pain. 1995; 60: 203–216

  57. Rosenstiel, AK and Keefe, FJ.
    The use of coping strategies in chronic low back pain patients:
    relationship to patient characteristics and current adjustment.
    Pain. 1983; 17: 33–44

  58. Hudak, PL, Amadio, PC, Bombardier, C et al.
    Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder, and head).
    Am J Ind Med. 1996; 29: 602–608

  59. Goldberg, DP and Hillier, VF.
    A scaled version of the General Health Questionnaire.
    Psychol Med. 1979; 9: 139–145

  60. Schwarzer, R and Jerusalem, M.
    Generalized self-efficacy scale. in: J Weinman, S Wright, M Johnston (Eds.)
    Measures in Health Psychology: A User’s Portfolio. Causal and Control Beliefs.
    Windsor, UK: NFER-NELSON. ; 1995: 35–37

  61. Fischer, D, Stewart, AL, Bloch, DA, Lorig, K, Laurent, D, and Holman, H.
    Capturing the patient’s view of change as a clinical outcome measure.
    JAMA. 1999; 282: 1157–1162

  62. Jaeschke, R, Singer, J, and Guyatt, GH.
    Measurement of health status: ascertaining the minimal clinically important difference.
    Control Clin Trials. 1989; 10: 407–415

  63. Karsdorp, PA and Vlaeyen, JWS.
    Goals matter: both achievement and pain-avoidance goals are associated with pain severity
    and disability in patients with low back and upper extremity pain.
    Pain. 2011; 152: 1382–1390

  64. Centers for Disease Control and Prevention.
    Health-related quality-of-life measures--United States, 1993.
    MMWR Morb Mortal Wkly Rep. 1995; 44: 195–200

  65. Sullivan, MJ, Adams, H, Horan, S, Maher, D, Boland, D, and Gross, R.
    The role of perceived injustice in the experience of chronic pain and disability:
    scale development and validation.
    J Occup Rehabil. 2008; 18: 249–261

  66. Beattie, PF, Nelson, R, and Murphy, DR.
    Development and preliminary validation of the MedRisk instrument to measure patient satisfaction
    with chiropractic care.
    J Manipulative Physiol Ther. 2011; 34: 23–29

  67. Wallston, BS, Wallston, KA, Kaplan, GD, and Maides, SA.
    Development and validation of the health locus of control (HLC) scale.
    J Consult Clin Psychol. 1976; 44: 580

  68. Leak, A, Cooper, J, Dyer, S, Williams, K, Turner-Stokes, L, and Frank, A.
    The Northwick Park Neck Pain Questionnaire, devised to measure neck pain and disability.
    Rheumatology. 1994; 33: 469–474

  69. Sullivan, MJ, Bishop, SR, and Pivik, J.
    The Pain Catastrophizing Scale: development and validation.
    Psychol Assess. 1995; 7: 524

  70. Chibnall, JT and Tait, RC.
    The Pain Disability Index: factor structure and normative data.
    Arch Phys Med Rehabil. 1994; 75: 1082–1086

  71. Nicholas, MK.
    The pain self-efficacy questionnaire: taking pain into account.
    Eur J Pain. 2007; 11: 153–163

  72. Shaw, WS, Woiszwillo, MJ, and Krupat, E.
    Further validation of the Patient-Practitioner Orientation Scale (PPOS) from recorded visits for back pain.
    Patient Educ Couns. 2012; 89: 288–291

  73. Kim, HS.
    Collaborative decision making in nursing practice: a theoretical framework.
    in: P Chinn (Ed.) Advances in Nursing Theory Development.
    Aspen Systems, Rockville, MD; 1983: 271–283

  74. Stratford, P, Gill, C, Westaway, M, and Binkley, J.
    Assessing disability and change on individual patients:
    a report of a patient specific measure.
    Physiother Canada. 1995; 47: 258–263

  75. Leeuw, M, Goossens, ME, van Breukelen, GJ, Boersma, K, and Vlaeyen, JW.
    Measuring perceived harmfulness of physical activities in patients with chronic low back pain:
    the Photograph Series of Daily Activities—short electronic version.
    J Pain. 2007; 8: 840–849

  76. Wicksell, RK, Lekander, M, Sorjonen, K, and Olsson, GL.
    The Psychological Inflexibility in Pain Scale (PIPS)–Statistical properties and model fit of an
    instrument to assess change processes in pain related disability.
    Eur J Pain. 2010; 14: 771.e1–771.e14

  77. Kopec, JA, Esdaile, JM, Abrahamowicz, M et al.
    The Quebec Back Pain Disability Scale. Measurement properties.
    Spine. 1995; 20: 341–352

  78. Roter, D and Larson, S.
    The Roter interaction analysis system (RIAS): utility and flexibility for analysis of medical interactions.
    Patient Educ Couns. 2002; 46: 243–251

  79. Altmaier, EM, Russell, DW, Kao, CF, Lehmann, TR, and Weinstein, JN.
    Role of self-efficacy in rehabilitation outcome among chronic low back pain patients.
    J Couns Psychol. 1993; 40: 335

  80. Antonovsky, A.
    The structure and properties of the sense of coherence scale.
    Soc Sci Med. 1993; 36: 725–733

  81. Brazier, J, Usherwood, T, Harper, R, and Thomas, K.
    Deriving a preference-based single index from the UK SF-36 Health Survey.
    J Clin Epidemiol. 1998; 51: 1115–1128

  82. Roach, KE, Budiman-Mak, E, Songsiridej, N, and Lertratanakul, Y.
    Development of a shoulder pain and disability index.
    Arthritis Rheum. 1991; 4: 143–149

  83. Brewer B, Van Raalte J, Petitpas A, Sklar J, Ditmar T.
    A brief measure of adherence during sport injury rehabilitation sessions.
    J Appl Sport Psychol 1995;7(suppl):S44.

  84. Vlaeyen, JWS, Kole-Snijders, AMJ, Boeren, RGB, and van Eek, H.
    Fear of movement/(re)injury in chronic low back pain and its relation to behavioral performance.
    Pain. 1995; 62: 363–372

  85. Brown, GK and Nicassio, PM.
    Development of a questionnaire for the assessment of active and passive coping strategies in chronic pain patients.
    Pain. 1987; 31: 53–64

  86. Bellamy, N, Buchanan, WW, Goldsmith, CH, Campbell, J, and Stitt, LW.
    Validation study of WOMAC: a health status instrument for measuring clinically important patient
    relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee.
    J Rheumatol. 1988; 15: 1833–1840

  87. Horvath, AO and Greenberg, LS.
    Development and validation of the Working Alliance Inventory.
    J Couns Psychol. 1989; 36: 223

  88. Zung, WW.
    A self-rating depression scale.
    Arch Gen Psychiatry. 1965; 12: 63–70



Return to ALT-MED/CAM

Since 8-31-2019

                       © 1995–2020 ~ The Chiropractic Resource Organization ~ All Rights Reserved