The Use of Complementary and Integrative Health Approaches
for Chronic Musculoskeletal Pain in Younger US Veterans:
An Economic Evaluation

This section is compiled by Frank M. Painter, D.C.
Send all comments or additions to:

FROM:   PLoS One. 2019 (Jun 5); 14 (6): e0217831 ~ FULL TEXT


Valerie F. Williams, MA, MS; Leslie L. Clark, PhD, MS; Mark G. McNellis, PhD

RAND Corporation,
Santa Monica, California,
United States of America.

FROM:   Schneider ~ Spine 2015 (Feb 15)

OBJECTIVES:   To estimate the cost-effectiveness to the US Veterans Health Administration (VA) of the use of complementary and integrative health (CIH) approaches by younger Veterans with chronic musculoskeletal disorder (MSD) pain.

PERSPECTIVE:   VA healthcare system.

METHODS:   We used a propensity score-adjusted hierarchical linear modeling (HLM), and 2010-2013 VA administrative data to estimate differences in VA healthcare costs, pain intensity (0-10 numerical rating scale), and opioid use between CIH users and nonusers. We identified CIH use in Veterans' medical records through Current Procedural Terminology, VA workload tracking, and provider-type codes.

RESULTS:   We identified 30,634 younger Veterans with chronic MSD pain as using CIH and 195,424 with no CIH use. CIH users differed from nonusers across all baseline covariates except the Charlson comorbidity index. They also differed on annual pre-CIH-start healthcare costs ($10,729 versus $5,818), pain (4.33 versus 3.76), and opioid use (66.6% versus 54.0%). The HLM results indicated lower annual healthcare costs (-$637; 95% CI: -$1,023, -$247), lower pain (-0.34; -0.40, -0.27), and slightly higher (less than a percentage point) opioid use (0.8; 0.6, 0.9) for CIH users in the year after CIH start. Sensitivity analyses indicated similar results for three most-used CIH approaches (acupuncture, chiropractic care, and massage), but higher costs for those with eight or more CIH visits.

CONCLUSIONS:   According to our propensity score-weighted HLM analysis, in a cohort of younger Veterans with chronic MSD pain during 2010 through 2013, any use of complementary and integrative health (CIH) was not only cost-effective, it was cost saving.

Editorial Comment:   CIH was initially referred to as “Unconventional Medicine”, then became “Alternative Medicine”, and then evolved into “Complementary and Alternative Medicine”. As it has become more accepted by conventionsl (“usual care”) medicine, it is typically referred to as “integrative” and/or “complementary”.

Any use of CIH was associated with an average reduction in healthcare costs of $637, a 0.34-point reduction in pain intensity on a 0–10 pain scale, and a less than one percentage point increase in opioid use during the year after CIH start. Given the VA's growing interest in the use of CIH, further, more detailed analyses of its impacts are warranted.

From the FULL TEXT Article:


Chronic pain is prevalent among Veterans in general, [1, 2] but especially among younger Veterans, including participants in Operation Enduring Freedom, Operation Iraqi Freedom and Operation New Dawn (OEF/OIF/OND). [3, 4] Musculoskeletal pain (e.g., back pain, neck pain, osteoarthritis) is the most common type of chronic pain. [5] This pain also often co-occurs with conditions such as anxiety, depression, post-traumatic stress disorder, and substance abuse. [3, 5]

Complementary and integrative health (CIH) approaches (therapies) such as yoga, meditation and acupuncture are being used for pain management in the Veterans Affairs (VA) healthcare system. [6, 7] CIH approaches are commonly provided to treat pain and its comorbidities, with acupuncture and chiropractic the most prevalent CIH approaches for pain. [8–10] A recent national survey found Veterans have substantial interest in and use of CIH approaches, with half reporting a use of a CIH approach in the prior year and 84% reporting interest in using CIH. [11]

Several guidelines recommend CIH approaches as effective for chronic pain, [12–15] and a few recent studies have shown these approaches to be associated with lower healthcare costs in the general population. [16–21] However, little is known about their impact on healthcare costs and utilization including opioid use in the VA where CIH is being increasingly emphasized. The VA has adopted the goal of implementing a Whole Health approach, in which CIH approaches are a key component. [22, 23]

We used VA administrative data to identify younger Veterans with chronic painful musculoskeletal disorders (MSD) who did and did not use CIH. We then estimated the impact of CIH use on VA healthcare costs, pain and opioid use to generate a preliminary estimate of the cost-effectiveness of CIH to the VA.


      Chronic MSD pain cohort

The cohort for this study was drawn from a large previously-constructed cohort [24] of Veterans who used the VA’s healthcare system during the 2010–2013 period with at least one visit with a MSD International Classification of Disease, Version 9, Clinical Modification (ICD-9-CM) diagnosis code. To identify MSD, we used 1,685 ICD-9-CM MSD codes assembled from previous reviews and team input, as well as text downloaded from members’ medical records.

From this larger cohort, we identified younger Veterans who would have been 18 to 40 years of age at the time of the OEF/OIF/OND conflicts (i.e., those born 1960–1995). We then identified those with chronic MSD pain using either: 1) 2 or more occurrences ≥30 days apart of MSD ICD-9-CM codes from the “likely to represent chronic pain” list from a study on identifying chronic pain from electronic records (S1 Appendix); [25] or 2) 2 or more occurrences of common MSD ICD-9-CM codes plus 2 or more self-reported pain scores ≥4 on a 0 (no pain) to 10 (maximum pain) scale, both within 90 days.

As we aimed to examine the impact of CIH use on VA healthcare users, and on healthcare utilization outcomes that could be influenced by short-term CIH use (versus utilization due to a high-cost catastrophic event), we eliminated those with $0 or >$50,000 annual costs, [26] and those who started CIH during a hospital stay.

      CIH user group, control group, and start dates

From this younger-Veteran chronic MSD pain cohort we defined CIH users as users of one or more of nine CIH approaches which could be identified from codes available in data from the VA’s Corporate Data Warehouse (CDW). [27] The codes used to identify CIH use were: Current Procedural Terminology (CPT) codes for the five types of CIH with these codes (i.e., acupuncture, biofeedback, chiropractic care, hypnosis/hypnotherapy, and massage); CHAR4 codes (used to track workload in the VA system) for the eight types of CIH that have these codes (i.e., acupuncture, biofeedback, guided imagery, hypnosis/hypnotherapy, therapeutic massage, meditation, Tai Chi, and yoga); and provider-type codes for chiropractors. We only considered CIH use that started after entry into the chronic MSD pain cohort, and to ensure we identified initial CIH use, we dropped those who used CIH during the first six months of 2010.

The control group included all Veterans in the cohort with no CIH codes, and no indications of CIH-related key words in their medical records. We defined the start dates for the CIH group as the first CIH use documented in each Veteran’s medical record as part of an outpatient visit using CPT, CHAR4 or chiropractic provider-type codes. Controls lack comparable “start dates” for CIH initiation, but to assure we compared their outcomes during a similar period in their healthcare history, we first randomly assigned the control group pseudo-start dates following the same general distribution as seen in the CIH user group in terms of days between qualifying for the MSD cohort and CIH start. We then identified the nearest (+/-90 days) outpatient visit and set that as the control group member’s “start date” to ensure both groups had start dates that were roughly the same length of time after entry into the MSD cohort and associated with an outpatient visit. If a member of the control group did not have an outpatient visit near their randomly assigned pseudo-start date, we dropped them from the cohort. The control group’s start dates were only used to define the time periods for their year-before and year-after outcomes.


For each outcome, we captured data one year before and one year after each cohort-members’ start date.

      Healthcare costs

We derived healthcare utilization data from the VA’s Medical SAS Inpatient and Outpatient Datasets, which contain VA inpatient, outpatient and pharmacy services use, including the use of CIH approaches. We obtained estimates of the cost of each type of care from the average cost database of the VA’s Health Economics Resource Center (HERC). [28–30] The average cost database allows VA healthcare utilization to be valued at costs that are similar to Medicare payment rates. Payments for inpatient, outpatient and pharmacy care provided by non-VA providers, but authorized and paid for by VA, were captured from the VA Fee Basis files. [31] Costs incurred in each year were converted 2013 costs using the appropriate Consumer Price Index (CPI). [32]

      Pain intensity

A 2000 VA initiative established pain as the “fifth vital sign” and began requiring regular documentation of Veterans’ reported pain on a 0–10 numerical rating scale (NRS) in clinical settings where vital signs are recorded. [33] This policy and its adherence allow these pain scores to provide a reasonable estimate of the health effects of interventions targeting painful conditions. We captured reported pain intensity for our chronic MSD cohort from the VA CDW Vital Signs Tables. This measure of pain is simple and has been extensively studied. [34–38] When multiple pain scores were recorded during a day, we used the highest score. [24]

      Opioid use

We used VA pharmacy data to categorize opioid use according to the scheme outlined by a panel of experts in opioid therapy and VA pain management practice and policy. [39] We defined opioid users as those for whom at least one outpatient opioid prescription for an oral or transdermal formulation was dispensed from a VA pharmacy in the period.


We utilized 32 covariates extracted from the VA CDW and Vital Status files in our analyses. For demographics we captured age in years at start date (18–34, 35–44 and 45–54 years), gender, race/ethnicity (non-Hispanic white, non-Hispanic black, Hispanic, non-Hispanic other), and marital status (married, single/never married, divorced/separated/widowed). We captured the following data on Veterans’ interaction with the healthcare system: financial means test (copay required or copay exempt based on an annual financial assessment), service connectedness (the extent to which any disability was caused by, incurred during, or aggravated by military service; <50 percent, ≥50 percent), and eligibility priority group (priority group 1, priority group 2–6, priority group 7–8, see website for details [40]). We captured the type of MSD pain in our chronic MSD pain cohort using ICD-9-CM codes for back pain, neck pain, joint pain, osteoarthritis, temporomandibular disorder, and fibromyalgia. We also used ICD-9-CM codes to capture data on 6 conditions commonly associated with chronic pain: anxiety, depression, post-traumatic stress disorder (PTSD), traumatic brain injury (TBI), sleep disturbance, and substance abuse. Finally, we captured Veterans general health on a number of dimensions commonly used in propensity scores using the categories and codes defined by the Agency for Healthcare Research and Quality’s Clinical Classification Software (CCS). [41] The CCS codes are groups of ICD-9-CM codes and are defined for asthma, cancer, cerebrovascular disease, chronic obstructive pulmonary disease, coronary artery disease, diabetes, headache, HIV infection, hypertension, mental illness, and obesity. We also calculated the Charlson comorbidity index. [42]


Our outcome data are available for multiple points in time over two years for each Veteran, and these outcomes occurred within the VA facility where the Veteran received care. Thus, the outcome data are likely correlated both within Veteran and within VA facility, and as such, do not meet the usual independence assumptions needed for traditional regression models. [26] Hierarchical linear modeling [43–45] (HLM) corrects for the error structure violations that can be caused by methods that require each observation’s errors to be independent. [43, 46] HLM also allows for unbalanced designs (e.g., different numbers of observations in the subgroups studied) which are troublesome to other methods, [43, 47] and because it uses all information available in the dataset, optimizes estimation in the presence of randomly missing data. [44, 47] Finally, the specification of the HLM allows for a different intercept and linear time effect for each of the following three groups: CIH users before CIH start, CIH user after CIH start, and a control group. The flexibility of this model specification allowed us to examine differences between the control groups and CIH users before the start of CIH and the impact of CIH use on CIH users after CIH start. Below we discuss how we used propensity score weighting to create a control group, and a good propensity score weight in this context would be indicated by near zero differences in the intercept and time (slope) coefficients between the control group and CIH users before CIH start. However, even in the face of an insufficient control group, the HLM allows for estimation of the impact of CIH use on CIH users by comparing the intercept and time coefficients of CIH users before and after CIH start. That is, the HLM leverages the within-individual change in outcomes over time to assess the impact of CIH use.

Because Veterans’ use of CIH was self-selected systematic differences in outcomes may exist between those who do and do not use CIH that are not due to the CIH use itself. [48–57] Instead of directly adjusting for these differences in the HLM by including covariates, we opted to use a propensity score adjustment. Propensity scores were originally introduced as a way to generate an unbiased estimate of the (causal) effect of an exposure (e.g., CIH use) in the presence of confounding. [58] In brief and ideally, propensity scores have the ability to replicate the benefits of a randomized trial by allowing comparison across those with the same probability of exposure (same propensity score) but who did and did not experience the exposure. [59] Propensity scores were estimated using logistic regression predicting CIH use from the covariates and baseline values of each outcome (healthcare costs, pain, and opioid use) in the year before the CIH start date. Because the availability of CIH varied by facility we did not include facility in the propensity scores. The propensity scores were subsequently used to weight the HLM so that those who do not use CIH (the control group) are as similar as possible (except for the CIH use itself) to those that used CIH. The propensity score weights were derived as ‘1’ for CIH users and the inverse of the propensity score for the controls. This allowed us to estimate the average treatment effect on the treated (ATT).

      Data preparation

For the HLM analysis we used monthly costs, daily pain scores, and monthly opioid use for the 12 months before and after each member’s start date. We used enrollment files to identify Veterans who had less than one year of data before their start date and dropped those with less than 180 days. We also dropped anyone with no pain scores within a year. Opioid use was measured as the release to the Veteran of any opioid prescription during a month. Months without costs and without opioid use were coded as zero costs and zero opioid use, respectively, while days without pain measurements were treated as missing data. All monthly or daily data points before a Veteran’s qualifying date (first MSD diagnosis) were dropped from the analysis.

      Sensitivity analyses

The CIH user group in our primary analyses included all cohort members with at least one visit for a CIH-coded approach. The three most common types of CIH used were acupuncture, chiropractic care and massage, and some consider a therapeutic dose to be 8 or more treatments. [60–63] Our sensitivity analyses examined healthcare cost, pain and opioid use for four subgroups: those who used any number of visits for acupuncture, massage, and chiropractic care, and those who had eight or more CIH visits of any kind.

      Cost-effectiveness analysis

Our cost-effectiveness analyses used the perspective of the VA. For the main analysis, we compared the difference in total annual VA healthcare costs to the difference in pain over the year after CIH start for those using any type and amount of CIH. Opioid use was considered a secondary outcome. Our one-year outcomes made discounting unnecessary. All analyses were run using STATA 15, StataCorp, College Station, Texas.

This study was approved by the Department of Veterans Affairs, Greater Los Angeles Healthcare System, Institutional Review Board, and by the VA GLA Associate Chief of Staff, Research and Development, R&D Committee, and all required subcommittees. This manuscript conforms to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS). See S1 Table.


Figure 1

Table 1

Table 2

Table 3

The steps involved in building our analytic cohort are shown in Figure 1. As can be seen, applying either of our two chronic MSD diagnosis criteria alone produced similar results: 99.2% (535,851/540,042) of our cohort met the first criterion and 91.0% (491,438/540,042) met the second. Further cleaning resulted in a final analytic cohort of 30,634 CIH users and 195,424 nonusers (control).

Table 1 shows the values for each covariate and for healthcare costs, pain intensity and opioid use in the year before start date for each group. As can be seen, the groups differed significantly on all patient characteristics other than the Charlson comorbidity index. The last three columns provide balance diagnostics on the propensity score weighting. The propensity score weighting improves the similarity of the groups. The average absolute standardized difference decreased after weighting to less than 0.01 in absolute value for all but joint pain and year-before healthcare costs and opioid use, and ratios of the variances between groups after weighting were within 0.02 of 1.0 for all but fibromyalgia, and year-before pain and healthcare costs.

Table 2 reports the results of the HLM models. The negative significant coefficients for time indicated that healthcare costs, pain and opioid use all generally decrease over the two-year period for the control group. The significant CIH user coefficients indicated that the CIH group had significantly higher healthcare costs and pain and lower opioid use to start. The positive and significant CIH x time interaction coefficients indicated that pre-CIH start trajectories are different between groups, and the size of these coefficients is such that the CIH group experienced less of a reduction in healthcare costs and a net increase in pain and opioid use in the year before CIH start. Therefore, despite the propensity score weighting the two groups have different levels of patterns of healthcare costs, pain or opioid use in the year before CIH start.

Among CIH users there was a significant downward shift in the trajectories for healthcare costs and pain and significant upward shift in opioid use at CIH start (main effect of CIH start) and minor changes in slope thereafter (interaction of CIH start and time). Average monthly healthcare costs by general cost category for the CIH group in the years before and after the start date are shown in S1 Figure. The significant random effects for facility indicated that outcomes are clustered by facility. Initial model runs also indicated a small random effect on facility slope. However, this term was excluded from later models because it added little to explanatory power and significantly hampered model convergence. As shown at the bottom of the table, and according to the estimated coefficients, CIH use was associated with a significant decrease in healthcare costs and pain, and a less than a percentage point increase in opioid use compared to control.

Table 3 gives the HLM results for four CIH subgroups included in our sensitivity analyses. The models for acupuncture, chiropractic care and massage all show a reduction in healthcare costs and pain and a small increase in opioid use. The subgroup with 8+ visits had a similar reduction in pain and increase in opioid use, but with an increase in healthcare costs.


This study used hierarchical linear modeling to examine the effect of the use of CIH approaches on VA healthcare costs, pain intensity and opioid use in a cohort of younger Veterans with chronic MSD who use VA healthcare services. The results showed lower healthcare costs and reduced pain for Veterans who used CIH: an average of $637 lower healthcare costs and 0.34 lower pain intensity on the 0–10 NRS. The estimates also revealed a 0.76 percentage point increase in the rate of opioid use for the year after onset of CIH use.

As expected, because each make up a substantial proportion of CIH use, the results seen for acupuncture, chiropractic care and therapeutic massage users were similar to those seen for users of any of the nine CIH approaches examined. Those who had 8+ visits for CIH during the year had similar reductions in pain and increases in opioid use, but an increase in healthcare costs of almost $1,500 per year. This increase in costs could partially be explained by the cost of the additional CIH visits. However, half of those with 8+ visits had 8–10 visits and all but 1 percent had fewer than 30 visits. The cost increase could also be related to how Veterans use VA healthcare. A recent analysis of Medical Expenditures Panel Survey (MEPS) data found that in 2014–2015 Veterans who used the VA healthcare system had higher overall costs than non-VA users and that only one-third of the total healthcare received by VA users was paid by the VA. [64] Our healthcare cost and CIH codes only captured utilization that occurred at or was reimbursed by the VA; we could be missing costs and CIH use that happened completely or partially outside the VA.

A number of systematic reviews have shown that CIH use is associated with reductions in MSD pain. [12, 13, 65–76] The reduction in pain intensity of 0.34 points on a 0–10 scale may seem clinically insignificant. However, these pain scores were captured not as outcomes in a clinical trial, but at admission during hospital stays and primary care visits when the Veteran could be experiencing other non-MSD pain or have made the visit because of a pain flare-up. The fact that these scores decreased significantly after CIH start is one indication of potential CIH benefit. Given that pain scores are only captured when Veterans interact with the health system, the reduction in healthcare utilization implied by the reduction in healthcare costs might also be considered evidence of reduced pain in CIH users. Little is known about the impact of CIH use on opioid use. One theory for the small increase in opioid use among CIH users is that their CIH use onset might have occurred after trying many other pain management options, with CIH use as a last resort. Further examination of the relationship between and timing of CIH and opioid use may be worthwhile.

Two other studies found cost savings across individuals using a variety of CIH approaches. [17, 21] The first used 2000–2003 data from two large insurers in Washington State (where coverage was mandated for all licensed providers) to examine the impact of CIH use for back pain. [17] CIH use was defined as at least one visit to a chiropractor, licensed massage therapist, acupuncturist, or naturopathic physician during the year after diagnosis. This study used frequency matching between CIH users and nonusers based on gender, 10-year age group, total expenditures prior year, and disease burden, and found an average reduction in healthcare costs of $329 per plan member with back pain in the year after diagnosis.

The second study used 2002–2008 MEPS data to examine the impact of CIH use on spinal pain. [21] CIH users were defined as having at least one visit to a chiropractor, massage therapist, homeopathic provider, acupuncturist, or “other CAM provider.” In propensity score-matched samples average expenditures for the CIH users were $526 lower for spine-specific costs and $298 lower for total health costs.

Other studies have examined healthcare costs for those who use acupuncture and chiropractic care. A study of the use of acupuncture in Alberta for low back pain age- and gender-matched those who used acupuncture to those who did not found that physician visits decreased by almost half in the year after acupuncture start, the cost of these visits dropped by 37%, and that the cost reductions were larger for those with 10+ acupuncture visits during the year. [19] Another study examining the effect of having a chiropractic coverage benefit on healthcare utilization and costs found that compared to members with healthcare use, but without chiropractic coverage, those with use and this coverage had an average of $208 lower overall healthcare costs per year during 2000.[20] Finally, a study of back-related healthcare costs for those using chiropractic or medical providers to manage their low back pain during a 2-year period 2004–2005 found that the average back-related cost of chiropractic treatment was $1,933 less over the 2-year period than medical care. [18]

This study benefits from a large real-world cohort and the use of administrative data. However, it also has several limitations, including all the limitations inherent in a cohort study attempting to create comparison groups that only differ by CIH use. The only variables available for adjusting between groups were those in administrative data, and although we included 32 covariates, they were insufficient to create an adequate balance between the groups at baseline. Therefore, it is likely that the differences seen between groups in this study were not all attributable to CIH use. We used a commonly used, well-accepted and understood approach (logistic regression) to estimate our propensity score weights. However, other approaches (e.g., covariate balancing propensity scores [77] or machine learning [78]) are available and may have produced different results. Nevertheless, because of our modeling approach we were able to estimate of the impact of CIH use on CIH users. To best identify those who used CIH for their chronic MSD we eliminated those who used CIH before they qualified as having chronic MSD. Thus, our estimates should not be assumed to apply to long-term CIH users.

Our HLM models assumed linear trends when healthcare costs, pain and/or opioid use over time might not be linear. We did allow a one-time change at CIH start followed by a change in trajectory for those in the CIH group. However, other than this pattern of rapid change in the first weeks of treatment followed by more gradual change seen in many trials of CIH, we had no hypotheses as to actual trajectory shape, so left the models as linear. Our healthcare cost data only captured healthcare utilization that occurred at or was reimbursed by the VA, which according to a recent study of MEPS data only accounts for about one-third of the total healthcare received by VA users. [64] Therefore, we could be missing healthcare costs and CIH use that happened completely or partially outside the VA system. The pain scores were assumed to be due to Veterans’ chronic MSD.

However, since these pain scores were captured in healthcare encounters and not as outcomes in a clinical trial they could be due to other painful conditions. They also likely understate pain reductions over time since high-pain events would be more likely than lower pain states to lead to healthcare encounters where pain would be recorded. We were limited to data from 2010–2013 because this previously assembled dataset contained a full set of medical records. It is also worth noting that our data span the height of opioid prescribing at the VA. [79] Analysis of more recent data may show different patterns in CIH, opioid, and other healthcare use.


According to our propensity score-weighted HLM analysis, in a cohort of younger Veterans with chronic MSD pain during 2010 through 2013, any use of CIH was not only cost-effective, it was cost saving. Any use of CIH was associated with an average reduction in healthcare costs of $637, a 0.34-point reduction in pain intensity on a 0–10 pain scale, and a less than one percentage point increase in opioid use during the year after CIH start.

Before CIH start, CIH users’ healthcare costs and pain are different than those of nonusers in ways that are not explained by data available in administrative records on demographics, socioeconomic factors, comorbidities, and MSD pain type. Care should be taken in future studies of CIH use using propensity scores in this population to ensure their adequacy in balancing groups. Also, given the prevalence of and increasing interest in CIH use in the VA, additional analyses should be undertaken to confirm these results on more recent data, and to understand the healthcare cost impacts of high-CIH utilizers.

S1 Appendix   (DOC)

Musculoskeletal ICD-9 codes likely to represent chronic pain and used as one criterion
to identify Veterans with chronic musculoskeletal pain

S1 Table   (DOC)

CHEERS Checklist: Items to include when reporting economic evaluations of health interventions.

S1 Figure   (PPTX)

Graphs of average monthly healthcare costs in the CIH group in the months before and after CIH start.


The study team would also like to acknowledge the contributions of Dr. Steven Zeliadt and Nui Brown to this project.

Funding Statement

All authors were funded under a grant provided by the Department of Veterans Affairs, Health Services Research and Development Service, 1 R01 HX001704-1, Principal Investigator: Dr. Stephanie L Taylor. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.


  1. Haskell SG, Heapy A, Reid MC, Papas RK, Kerns RD.
    The prevalence and age-related characteristics of pain in a sample of women veterans receiving primary care.
    J Womens Health. 2006;15(7):862–9.

  2. Kerns RD, Otis J, Rosenberg R, Reid MC.
    Veterans' reports of pain and associations with ratings of health, health-risk behaviors, affective distress,
    and use of the healthcare system.
    J Rehabil Res Dev. 2003;40(5):371–80.

  3. Clark ME, Bair MJ, Buckenmaier CC III, Gironda RJ, Walker RL.
    Pain and combat injuries in soldiers returning from Operations Enduring Freedom and Iraqi Freedom:
    implications for research and practice.
    J Rehabil Res Dev. 2007;44(2).

  4. Kerns RD, Dobscha SK.
    Pain among veterans returning from deployment in Iraq and Afghanistan:
    update on the Veterans Health Administration Pain Research Program.
    Pain Med. 2009;10(7):1161–4. 10.1111/j.1526-4637.2009.00722.x

  5. Macey TA, Morasco BJ, Duckart JP, Dobscha SK.
    Patterns and correlates of prescription opioid use in OEF/OIF veterans with chronic noncancer pain.
    Pain Med. 2011;12(10):1502–9. 10.1111/j.1526-4637.2011.01226.x

  6. McEachrane-Gross FP, Liebschutz JM, Berlowitz D.
    Use of Selected Complementary and Alternative Medicine (CAM) Treatments in Veterans with
    Cancer or Chronic Pain: A Cross-sectional Survey

    BMC Complement Altern Med 2006 (Oct 6); 6: 34

  7. Denneson LM, Corson K, Dobscha SK.
    Complementary and Alternative Medicine Use Among Veterans With Chronic Noncancer Pain
    Journal of Rehabilitation Research and Development 2011; 48 (9): 1119–1128

  8. Healthcare Analysis & Information Group. 2011.
    Complementary and Alternative Medicine Survey.
    Washington, DC: Veterans Health Administration,
    Department of Veterans Affairs; 2011 September.

  9. Healthcare Analysis & Information Group.
    FY 2015 VHA Complementary & Integrative Health (CIH) Services (formerly CAM).
    Washington, DC: Veterans Health Administration,
    Department of Veterans Affairs; 2015. May.

  10. Healthcare Analysis & Information Group.
    Survey of Complementary and Alternative Medicine.
    Washington, DC: Veterans Health Administration,
    Department of Veterans Affairs; 2002. May.

  11. Taylor SL, Hoggatt K, Kligler B.
    Complementary and Integrated Health Approaches: What Do Veterans Use and Want
    J Gen Intern Med. 2019 (Jul); 34 (7): 1192–1199

  12. Skelly AC, Chou R, Dettori JR, et al.
    Noninvasive Nonpharmacological Treatment for Chronic Pain:
    A Systematic Review

    Comparative Effectiveness Review no. 209
    Agency for Healthcare Research and Quality (US), Rockville (MD) (2018)

  13. Chou R, Deyo R, Friedly J, Skelly A, Hashimoto R, Weimer M, et al.
    Nonpharmacologic Therapies for Low Back Pain: A Systematic Review
    for an American College of Physicians Clinical Practice Guideline

    Annals of Internal Medicine 2017 (Apr 4); 166 (7): 493–505

  14. Qaseem A, Wilt TJ, McLean RM, Forciea MA;
    Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain:
    A Clinical Practice Guideline From the American College of Physicians

    Annals of Internal Medicine 2017 (Apr 4); 166 (7): 514–530

  15. The Diagnosis and Treatment of Low Back Pain Work Group.
    VA/DoD Clinical Practice Guideline for Diagnosis and Treatment of Low Back Pain   PDF
    Washington, DC: The Office of Quality, Safety and Value, VA, &
    Office of Evidence Based Practice, U.S. Army MedicalCommand, 2017, Version 2.0.

  16. Herman PM, Poindexter BL, Witt CM, Eisenberg DM.
    Are complementary therapies and integrative care cost-effective?
    A systematic review of economic evaluations.
    BMJ Open. 2012;2(5):e001046 10.1136/bmjopen-2012-001046

  17. Lind BK, Lafferty WE, Tyree PT, Diehr PK.
    Comparison of health care expenditures among insured users and nonusers of complementary and alternative
    medicine in Washington state: A cost minimization analysis.
    J Altern Complement Med. 2010;16(4):411–7. 10.1089/acm.2009.0261

  18. Grieves B, Menke JM, Pursel KJ.
    Cost minimization analysis of low back pain claims data for chiropractic vs medicine in a managed
    care organization.
    J Manipulative Physiol Ther. 2009;32(9):734–9. 10.1016/j.jmpt.2009.10.001

  19. Moritz S, Liu MF, Rickhi B, Xu TJ, Paccagnan P, Quan H.
    Reduced health resource use after acupuncture for low-back pain.
    J Altern Complement Med. 2011;17(11):1015–9. 10.1089/acm.2010.0619

  20. Legorreta, AP, Metz, RD, Nelson, CF, Ray, S, Chernicoff, HO, and Dinubile, NA.
    Comparative Analysis of Individuals With and Without Chiropractic Coverage:
    Patient Characteristics, Utilization, and Costs

    Archives of Internal Medicine 2004 (Oct 11); 164 (18): 1985–1892

  21. Martin, BI, Gerkovich, MM, Deyo, RA et al.
    The Association of Complementary and Alternative Medicine Use and Health Care Expenditures
    for Back and Neck Problems

    Medical Care 2012 (Dec); 50 (12): 1029–1036

  22. VHA Office of Patient Centered Care and Cultural Transformation.
    Whole Health—It's All About You: Partnering with You for Health and Well-Being.
    Washington, DC: US Department of Veterans Affairs,
    Veterans Health Administration; 2017.

  23. VA Health Care.
    Whole Health for Life: Expanding the VA Whole Health System Washington, DC:
    US Department of Veterans Affairs; 2017.

  24. Goulet JL, Kerns RD, Bair M, Becker WC, Brennan P, Burgess DJ, et al.
    The Musculoskeletal Diagnosis Cohort:
    Examining Pain and Pain Care Among Veterans

    Pain 2016 (Aug); 157 (8): 1696–1703

  25. Tian TY, Zlateva I, Anderson DR.
    Using electronic health records data to identify patients with chronic pain in a primary care setting.
    J Am Med Inform Assoc. 2013;20(e2):e275–e80. 10.1136/amiajnl-2013-001856

  26. Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U.
    Good research practices for comparative effectiveness research: analytic methods to improve causal
    inference from nonrandomized studies of treatment effects using secondary data sources:
    the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part III.
    Value Health. 2009;12(8):1062–73. 10.1111/j.1524-4733.2009.00602.x

  27. Veterans Health Administration (VHA).
    Corporate Data Warehouse (CDW) Washington, DC: US Department of Veterans Affairs; 2017.

  28. Wagner TH, Chen HY, Barnett PG.
    Using average cost methods to estimate encounter-level costs for medical-surgical stays in the VA.
    Med Care Res Rev. 2003;60(3):15S–36S.

  29. Yu W, Ravelo A, Wagner TH, Phibbs CS, Bhandari A, Chen S, et al.
    Prevalence and costs of chronic conditions in the VA health care system.
    Med Care Res Rev. 2003;60(3):146–67.

  30. Phibbs CS, Bhandari A, Yu W, Barnett PG.
    Estimating the costs of VA ambulatory care.
    Med Care Res Rev. 2003;60(3 Suppl):54S–73S. 10.1177/1077558703256725

  31. Smith MW, Chow A.
    Fee Basis Data: A Guide for Researchers.
    Menlo Park, CA: Health Economics Resource Center,
    US Department of Veterans Affairs; 2010.

  32. Bureau of Labor Statistics.
    Consumer Price Index. 2013.

  33. National Pain Management Coordinating Committee.
    Pain as the 5th vital sign toolkit, revised edition
    Washington, DC: Veterans Health Administration; 2000. October.

  34. Bolton JE, Wilkinson RC.
    Responsiveness of pain scales: a comparison of three pain intensity measures in chiropractic patients.
    J Manipulative Physiol Ther. 1998;21(1):1–7.

  35. Downie W, Leatham P, Rhind V, Wright V, Branco J, Anderson J.
    Studies with pain rating scales.
    Ann Rheum Dis. 1978;37(4):378–81. 10.1136/ard.37.4.378

  36. Farrar JT, Young JP, Jr, LaMoreaux L, Werth JL, Poole MR.
    Clinical Importance of Changes in Chronic Pain Intensity
    Measured on an 11-point Numerical Pain Rating Scale

    Pain 2001 (Nov); 94 (2): 149-158

  37. Jensen MP, Karoly P, Braver S.
    The measurement of clinical pain intensity: a comparison of six methods.
    Pain. 1986;27(1):117–26.

  38. Lorenz KA, Sherbourne CD, Shugarman L, Rubenstein L, Wen L, Cohen A, et al.
    How reliable is pain as the fifth vital sign?
    J Am Board Fam Med. 2009;22(3):291–8. 10.3122/jabfm.2009.03.080162

  39. Midboe AM, Lewis ET, Paik MC, Gallagher RM, Rosenberg JM, Goodman F, et al.
    Measurement of adherence to clinical practice guidelines for opioid therapy for chronic pain.
    Transl Behav Med. 2012;2(1):57–64. 10.1007/s13142-011-0104-5

  40. VA Health Care.
    Enrollment Priority Groups—What are they and how do they work?
    Washington, DC: US Department of Veterans Affairs; 2018. [updated April 2, 2018.

  41. Agency for Healthcare Research and Quality.
    Appendix C—Clinical Classification Software—Expanded Diagnosis Categories (January 1980 through
    September 2015): Multi-level CCS Category and Diagnoses
    Rockville, MD: Agency for Healthcare Research and Quality; 2016. [updated 4/18/16.

  42. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, et al.
    Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
    Medical Care. 2005:1130–9.

  43. Heo M, Faith MS, Mott JW, Gorman BS, Redden DT, Allison DB.
    Hierarchical linear models for the development of growth curves: an example with body mass index
    in overweight/obese adults.
    Stat Med. 2003;22:1911–42. 10.1002/sim.1218

  44. Raudenbush SW, Bryk AS.
    Hierarchical Linear Models: Applications and Data Analysis Methods. Second Edition ed.
    Thousand Oaks, CA: Sage Publications; 2002. 483 p.

  45. Singer JD.
    Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth curve models.
    Journal of Educational and behavioral statistics. 1998;24(4):323–55.

  46. Feldman HA.
    Families of lines: random effects in linear regression analysis.
    J Appl Physiol. 1988;64(4):1721–32. 10.1152/jappl.1988.64.4.1721

  47. O'Connell AA, McCoach DB.
    Applications of hierarchical linear models for evaluations of health interventions:
    demystifying the methods and interpretations of multilevel models.
    Eval Health Prof. 2004;27(2):119–51. 10.1177/0163278704264049

  48. Astin JA.
    Why patients use alternative medicine: results of a national survey.
    JAMA. 1998;279(19):1548–53.

  49. Avogo W, Frimpong JA, Rivers PA, Kim SS.
    The effects of health status on the utilization of complementary and alternative medicine.
    Health Educ J. 2008;67(4):258–75.

  50. Caspi O, Koithan M, Criddle MW.
    Alternative medicine or "alternative" patients: a qualitative study of patient-oriented
    decision-making processes with respect to complementary and alternative medicine.
    Med Decis Making. 2004;24:64–79. 10.1177/0272989X03261567

  51. Owens JE, Taylor AG, Degood D.
    Complementary and alternative medicine and psychologic factors: toward an individual differences model
    of complementary and alternative medicine use and outcomes.
    J Altern Complement Med. 1999;5(6):529–41. 10.1089/acm.1999.5.529

  52. Rickhi B, Quan H, Moritz S, Stuart HL, Arboleda-Flórez J.
    Mental disorders and reasons for using complementary therapy.
    Canadian Journal of psychiatry. 2003;48(7):475–9. 10.1177/070674370304800708

  53. Sirois FM.
    Treatment seaking and experience with complementary/alternative medicine: a continuum of choice.
    J Altern Complement Med. 2002;8(2):127–34.

  54. Sparber A, Wootton JC.
    Surveys of complementary and alternative medicine: part v. use of alternative and complementary therapies
    for psychiatric and neurologic diseases.
    J Altern Complement Med. 2001;8(1):93–6.

  55. Swartzman LC, Harshman RA, Burkell J, Lundy ME.
    What accounts for the appeal of complementary/alternative medicine, and what makes complementary/
    alternative medicine "alternative"?
    Med Decis Making. 2002;22:431–50. 10.1177/027298902236943

  56. Thorne S, Paterson B, Russell C, Schultz A.
    Complementary/alternative medicine in chronic illness as informed self-care decision making.
    Int J Nurs Stud. 2002;39:671–83.

  57. Wootton JC, Sparber A.
    Surveys of complementary and alternative medicine: part iv. use of alternative and complementary
    therapies for rheumatologic and other diseases.
    J Altern Complement Med. 2001;7(6):715–21. 10.1089/10755530152755270

  58. Rosenbaum PR, Rubin DB.
    The central role of the propensity score in observational studies for causal effects.
    Biometrika. 1983;70(1):41–55.

  59. Williamson E, Morley R, Lucas A, Carpenter J.
    Propensity scores: from naive enthusiasm to intuitive understanding.
    Stat Methods Med Res. 2012;21(3):273–93. 10.1177/0962280210394483

  60. Schwehr NA, Shippee ND, Johnson PJ.
    Acupuncture ‘dose’(number of treatments) and insurance benefits in the USA.
    Acupunct Med. 2018:acupmed-2016-011341.

  61. Haas M, Vavrek D, Peterson D, Polissar N, Neradilek MB.
    Dose-response and Efficacy of Spinal Manipulation for Care of Chronic Low Back Pain:
    A Randomized Controlled Trial

    Spine J. 2014 (Jul 1); 14 (7): 1106–1116

  62. Perlman AI, Ali A, Njike VY, Hom D, Davidi A, Gould-Fogerite S, et al.
    Massage therapy for osteoarthritis of the knee: a randomized dose-finding trial.
    PLoS One. 2012;7(2):e30248 10.1371/journal.pone.0030248

  63. Sherman KJ, Cook AJ, Wellman RD, Hawkes RJ, Kahn JR, Deyo RA, et al.
    Five-week outcomes from a dosing trial of therapeutic massage for chronic neck pain.
    Ann Fam Med. 2014;12(2):112–20. 10.1370/afm.1602

  64. Machlin S, Muhuri P.
    Characteristics and Health Care Expenditures of VA Health System Users versus Other Veterans, 2014–2015 (Combined).
    Rockville, MD: Agency for Healthcare Research and Quality; 2018. Contract No.: #508.

  65. Boyd C, Crawford C, Paat CF, Price A, Xenakis L, Zhang W, et al.
    The impact of massage therapy on function in pain populations—A systematic review and meta-analysis
    of randomized controlled trials: Part II, cancer pain populations.
    Pain Med. 2016;17(8):1553–68. 10.1093/pm/pnw100

  66. Cheng YH, Huang GC.
    Efficacy of massage therapy on pain and dysfunction in patients with neck pain:
    a systematic review and meta-analysis.
    Evid Based Complement Alternat Med. 2014;2014.

  67. Clare HA, Adams R, Maher CG.
    A systematic review of efficacy of McKenzie therapy for spinal pain.
    Aust J Physiother. 2004;50(4):209–16.

  68. Coulter ID, Crawford C, Hurwitz EL, Vernon H, Khorsan R, Suttorp Booth M, Herman PM.
    Manipulation and Mobilization for Treating Chronic Low Back Pain:
    A Systematic Review and Meta-analysis

    Spine J. 2018 (May); 18 (5): 866–879

  69. Coulter ID, Crawford C, Hurwitz EL, Vernon H, Khorsan R, Booth MS, et al.
    Manipulation and mobilization for treating chronic neck pain: a systematic review and meta-analysis.
    Pain Physician. 2018;(under review).

  70. Cramer H, Lauche R, Haller H, Dobos G.
    A systematic review and meta-analysis of yoga for low back pain.
    The Clinical journal of pain. 2013;29(5):450–60. 10.1097/AJP.0b013e31825e1492

  71. Crawford C, Boyd C, Paat CF, Price A, Xenakis L, Yang E, et al.
    The impact of massage therapy on function in pain populations—A systematic review and meta-analysis
    of randomized controlled trials: Part I, patients experiencing pain in the general population.
    Pain Med. 2016;17(7):1353–75. 10.1093/pm/pnw099

  72. Furlan AD, Imamura M, Dryden T, Irvin E.
    Massage for low back pain: an updated systematic review within the framework of the Cochrane
    Back Review Group.
    Spine. 2009;34(16):1669–84. 10.1097/BRS.0b013e3181ad7bd6

  73. Furlan, A.D., Yazdi, F., Tsertsvadze, A. et al.
    A Systematic Review and Meta-analysis of Efficacy, Cost-effectiveness, and Safety of Selected
    Complementary and Alternative Medicine for Neck and Low-back Pain

    Evid Based Complemen Alt Med. 2012 (Nov 24); 2012: 953139

  74. Hilton L, Hempel S, Ewing BA, Apaydin E, Xenakis L, Newberry S, et al.
    Mindfulness meditation for chronic pain: systematic review and meta-analysis.
    Ann Behav Med. 2016;51(2):199–213.

  75. Lam M, Galvin R, Curry P.
    Effectiveness of acupuncture for nonspecific chronic low back pain: a systematic review and meta-analysis.
    Spine. 2013;38(24):2124–38. 10.1097/01.brs.0000435025.65564.b7

  76. Posadzki P, Ernst E.
    Yoga for low back pain: a systematic review of randomized clinical trials.
    Clin Rheumatol. 2011;30(9):1257–62. 10.1007/s10067-011-1764-8

  77. Imai K, Ratkovic M.
    Covariate balancing propensity score.
    J Roy Stat Soc Ser B (Stat Method). 2014;76(1):243–63.

  78. McCaffrey DF, Ridgeway G, Morral AR.
    Propensity score estimation with boosted regression for evaluating causal effects in observational studies.
    Psychol Methods. 2004;9(4):403 10.1037/1082-989X.9.4.403

  79. Hadlandsmyth K, Mosher H, Vander Weg MW, Lund BC.
    Decline in prescription opioids attributable to decreases in long-term use:
    A retrospective study in the Veterans Health Administration 2010–2016.
    J Gen Intern Med. 2018;33(6):818–24. 10.1007/s11606-017-4283-8





                  © 1995–2023 ~ The Chiropractic Resource Organization ~ All Rights Reserved