Select Committee on Science and Technology Sixth Report


CHAPTER 7: RESEARCH AND DEVELOPMENT

7.1 There are many uncertainties surrounding CAM and its practice. In order to respond to these concerns, high-quality research into CAM is vital. Nearly all the evidence we have received has confirmed that research is important in order to ensure that the treatments which the public are accessing are safe, that the efficacy of different therapies can be proved and that their cost-effectiveness can also be established. It is also important to determine why the public are accessing CAM.

7.2 The priority areas for research into each therapy will vary, depending on what, if any, evidence already exists. However, the range of high-quality research activity that has occurred and is occurring in the CAM sphere is minimal.

7.3 Several reasons have been put forward to explain why so little high-quality CAM research is being done. The five most common reasons suggested to us are: a lack of research training across the CAM professions; a lack of research funding available for CAM projects; a poor, almost non-existent, research infrastructure within the CAM sector; a lack of interest in this field of research by conventional scientists who are trained in research methodology; and finally methodological issues, with many CAM practitioners believing that conventional research methods are not suitable tools with which to investigate CAM.

7.4 The three main questions that this chapter will cover are:

  • What are the priority areas for research?

  • How can the amount of high-quality research be increased?

  • How could the outcomes of such research be translated into clinical practice?

Priority Areas For Research

7.5 Important areas for research in the CAM field may be considered to fall into six main categories:

    (i)  Research into the effects of each individual therapy; its efficacy, its safety and its cost-effectiveness.

    (i)  Research into the mechanisms of action of each individual therapy, including patterns of response to treatment and research into the placebo effect.

    (ii)  Research into the CAM genre itself, including social research into the motivation of those patients seeking CAM and the usage patterns of CAM.

    (iii)  Research into new research strategies which are sensitive to the CAM paradigm.

    (iv)  Research into the efficacy of the diagnostic methods used.

    (v)  Research into the implementation and effects of CAM in specific healthcare settings.

7.6 In the book by Vincent & Furnham[45] referred to in para 3.14 it is suggested that in well-established therapies questions regarding mechanisms of action or the optimum way to deliver treatment may be allowed to dominate research but, in the case of relatively new and untested treatments, questions about safety and efficacy should first be determined.

7.7 To conduct research into the CAM disciplines will require much work and resources, and will therefore be time-consuming. Hence, we recommend that three important questions should be addressed in the following order:

    (i)  To provide a starting point for possible improvements in CAM treatment, to show whether further inquiry would be useful, and to highlight any areas where its application could inform conventional medicine — does the treatment offer therapeutic benefits greater than placebo?

    (ii)  To protect patients from hazardous practices — is the treatment safe?

    (iii)  To help patients, doctors and healthcare administrators choose whether or not to adopt the treatment — how does it compare, in medical outcome and cost-effectiveness, with other forms of treatment?

Should Specific Therapies be Prioritised?

7.8 In all medical research a structure of testing exists where treatments with no evidence base are tested first in small-scale studies, and only once some evidence of efficacy has been confirmed are large-scale studies begun. In our opinion those therapies in Group 1 are at a stage where large-scale studies are justifiable because the ground work has already been completed. This is in contrast to the majority of the other CAM therapies where there is little or no objective evidence to support a beneficial effect and as a result small-scale studies are more appropriate. This is the experience of the NCCAM in the USA (see paras 7.84-7.92). Dr Stephen Straus, Director of NCCAM, told us that applications for research funding from those therapies in our Groups 2 and 3 "…are more developmental research applications, what we at the NIH [National Institutes for Health][46] call R-21s, with a neighbourhood of about $100,000 or $125,000. The evidence base is not sufficient to justify what we call an RO-1, which is a full project, which is four times that size. Most of our applications in number are the smaller developmental projects. Most of our funding goes to larger projects within those very few areas where the evidence base is strongest" (Q 1731). We would expect to see a similar pattern in the United Kingdom.

7.9 We believe that those therapies in Group 1 are likely to command the highest proportion of research resources. Those therapies in Groups 2 and 3 will need to build up their respective evidence bases with small preliminary studies before large-scale studies are justifiable.

Research Methods

7.10 In order to address the research questions identified above, it is imperative that research methods are defined that are both sympathetic to the CAM paradigm and rigorous in their design, execution and analysis. We have received a considerable amount of evidence indicating that CAM may have some specific features that makes it less amenable to testing by conventional methods of investigation, most especially the Randomised Controlled Trial (RCT).

7.11 A number of research methods and their applicability to CAM are discussed below.

Randomised Controlled Trials (RCTs)

7.12 An RCT is a trial with two defining features. First, control: outcomes for patients in the trial are compared with outcomes for patients in a "control" group who do not receive the treatment. Second, randomness: patients are assigned to the two groups at random, so that any difference in outcome can be attributed solely to the treatment.

7.13 The control population in an RCT can either consist of patients with the same pathology who receive a "dummy" (or placebo) treatment (to test overall efficacy) or a group who receive a comparison treatment to assess whether the therapy under investigation has any advantages in terms of efficacy, safety, cost or other benefits. Where the most important question is whether the effect is psychological, it is crucial to have a placebo control group who believe they are receiving the treatment when in fact they are not. (This is a "blind" trial).

7.14 Much of our evidence states that the RCT is regarded as the most powerful scientific tool for evaluating medical therapies and is accepted as the gold standard in research against which to assess a therapy's efficacy.

7.15 Although many scientists see RCTs as the ultimate analytical tool, RCTs may not fully embrace the CAM paradigm. Some CAM practitioners suggest that RCTs cannot do justice to the individualised, person-centred approach of many CAM therapies.

7.16 FIM explained some of the reasoning behind such claims: "In particular two things are really important in many CAM therapies. Firstly, the idea of standardisation versus individualisation, that actually individualised treatment has a higher value than a standard treatment, and standardising CAM treatments can lead to research outcomes which are not true of the therapy. The second point is that the traditional diagnoses which are used or the hands-on skill…are incredibly important in ensuring a good outcome, and that too has to be taken into account when you look at research" (Q 91).

7.17 The Aromatherapy Organisations Council also explained why they do not believe RCTs are always the appropriate method for CAM research: "It is not necessarily appropriate for all medical and healthcare interventions to be backed by the evidence of controlled clinical trials and by conventional scientific thinking. As Kleijnen[47] points out: 'the healing process is traditionally in three parts: the self-healing properties of the body; the changes induced by non-specific effects of the therapist and the setting in which the therapy takes place; and specific effects of physical and pharmacological interventions.' In mainstream medicine, the latter usually predominates (but not always) while in complementary medicine it is the second mechanism, and the first process is invariably a factor in both cases. The second may be considered a placebo response and this has made the conducting of randomised controlled trials in aromatherapy difficult" (p 14).

7.18 In March 2000 we visited the Marylebone Health Centre (see Appendix 4) which conducts practice-based research. During our visit, Dr David Peters, a GP and osteopath, discussed the applicability of research to real-life practice. He suggested that although RCTs and meta-analysis of RCTs are valuable, in that they provide certainty about the efficacy of a medication for a particular condition, real-life primary care does not mirror the way illness and treatment are defined in such research. He further explained that many patients do not come to their practitioners with specific, well-defined conditions, but the conditions of entry into most trials eliminates all but the most clear-cut examples of a condition. He suggested that this is especially a problem for CAM as GPs often refer the more complicated patients with chronic, complex conditions to CAM practitioners. Often these patients were not suffering from a single problem; although a particular condition may have been the primary reason for referral, further discussion often unveiled an array of inter-related problems. Thus, the simple definitions of clinical problem and treatment that good RCTs require do not always mirror the complexity of CAM practice.

7.19 Many conventional researchers and medical practitioners have criticised the arguments CAM practitioners use against RCTs, and have suggested that they are just excuses to avoid having to submit a type of therapy to scientific scrutiny. Professor Tom Meade of the Royal Society told us: "I do not see any reason why any of those CAMs cannot be subjected to randomised control trials. There is absolutely nothing different, in principle, between a CAM and a conventional treatment. A lot of …CAMs have actually already been subjected to randomised control trials" (Q 163). Dr Stephen Straus concurred with Professor Meade's opinion: "The tools of science are such that they can be brought to bear on this field. There is often the prejudice that the tools of science have nothing to add to the field of complementary and alternative medicine. I respectfully disagree with that opinion" (Q 1733).

7.20 Despite accounts that only a few years ago no middle ground could be found between those who rejected RCTs as a research method for CAM and those who advocated them as the only way to prove a therapy's efficacy, we found a growing acceptance from all sides that RCTs had their place but that that place was alongside other research methodologies. Therefore evidence from RCTs must be seen in the context of other evidence from different sources. For example the British Acupuncture Council wrote that: "When researching the evidence for acupuncture, different research questions require different methodologies. To answer questions of efficacy, effectiveness and cost effectiveness, the randomised control trial (RCT) will provide the most vigorous evidence. However where RCTs are too difficult, non-randomised controlled studies, outcome studies and observational studies may provide the best evidence. The variety of research methods range from quantitative to qualitative, and often these two approaches, when appropriately combined, provide triangulation which strengthens the evidence base" (p 28).

7.21 The Academy of Medical Sciences agreed: "…all medical and healthcare interventions should ideally be backed by the evidence of controlled clinical trials and by scientific thinking…evaluation cannot, however, always be achieved by double-blind trial but requires a synthesis of evidence from every reliable source" (p 3).

7.22 It has been suggested that one of the reasons many CAM practitioners are so suspicious of the RCT is because of a widespread perception of the rigidity of such trials and a lack of understanding that their design can be somewhat flexible. For example, not everyone realises that RCTs do not invariably have to be blinded or fully standardised[48] in the way many imagine. Professor Tom Meade described how RCTs can be adapted for different treatments: "Many of them, of course, cannot be double- or single-blind (see para 3.30); the patient and the therapist are both going to know what treatment is being given. There are ways of getting round that. For example, the patient can be asked to go to an assessor who does not know what treatment he or she was having, and in my experience patients enjoy that; they rather like the air of mystery and slight deception that is involved in that. It works very well…I think the point that you make about the amount of treatment varying in some circumstances is not, again, in principle, any different from conventional medicine, and what you may be saying there is that you are testing a package of treatment against some other treatment and you are not necessarily concerned, at that stage, with the dosage effect; you are just wanting to see if patients who have had this treatment, devised according to whatever the therapist thinks is best, will do better than patients who are given another treatment" (Q 163).

7.23 The RCCM confirmed that there is often misunderstanding among proponents of CAM concerning the flexibility of the RCT: "I think from the point of view of CAMs there is, to some extent, a misunderstanding about the nature of the RCT and the fact that perhaps it needs to be blinded, and it does not; it is possible to carry out randomised control trials that are unblinded. There is a question of what is a comparable or useful control because, for example, it is very difficult to find a believable placebo for acupuncture. Various researchers have tried it but it is very difficult to find one. However, what is possible to do is to carry out pragmatic randomised control trials; for example, to randomise patients with a particular problem — chronic back pain for example — into a group where they receive a holistic assessment from a traditional Chinese acupuncturist and then receive individualised care. This group of patients is compared with the best available treatment we have" (Q 129).

7.24 It is clear that, in the treatment of many conditions which are not life-threatening; sequential, longitudinal or cross-over trials (with an intervening 'wash out' period) in which a conventional medicine is given for a period, followed by a CAM therapy, or vice-versa, are powerful and scientifically acceptable methods for assessing the efficacy of a particular treatment. Alternatively, the results of the best-known conventional treatment can be compared with those of a CAM therapy in matched groups of patients.

7.25 Concerns over RCTs distorting a therapy or disguising its efficacy are not the unique concerns of CAM practitioners, there are many potential problems for all therapies when designing an RCT, and these are reviewed in Appendix 1. However, Vincent & Furnham acknowledge that some of these methodological problems are particularly pertinent to CAM research: these are elaborated upon in Box 9.


Box 9
  
Issues to Consider When Designing RCTs for CAM Therapies
  
Alternative theoretical frameworks: This is only an issue if two different disciplines have incompatible theories supporting them, which is the case when some of the traditional theories used to explain CAM systems are compared with the scientific basis of conventional medicine. However, Vincent & Furnham point out that this need not matter unduly in clinical trials as long as the CAM practitioners are allowed to practise in the way they see fit and the researchers understand and are tolerant towards the different approach of CAM. They suggest that both sides have to take a 'black box' view of the therapy, by which they mean evaluate it as a package and study the specific ingredients of the package later.
  
Different diagnostic systems: Patients in controlled trials need to be homogeneous with respect to their disease. In conventional medicine a specific diagnosis is specified but in CAM people are classed in different ways; for example by symptom (meridian imbalances etc). Although both systems may take into account a wide range of information, much of which may overlap, they are unlikely to correspond exactly and the diagnosis will differ. Therefore in clinical trials of CAM an important question to consider is: 'Are the groups going to be homogeneous in terms of a conventional medicine diagnosis or a CAM diagnosis?'
  
Single-blind or double-blind trials: Some treatments cannot be tested in a double blind format. This is especially true with some physical interventions which require a skilled practitioner who will, of course, know whether they are giving a sham treatment or not. Independent assessment can get over some of the problems of designing trials for such treatments, but independent assessment does not make a trial double-blind. This is also a problem with many conventional medicine treatments including surgery and thus is not an issue unique to CAM.
  
Individual differences in treatment and responses to treatment: It is integral to many CAM therapies that the patients are treated as individuals. (However, again, this is not a characteristic unique to CAM.) The problem here is that RCTs usually involve the testing of a standard therapy for a standard disease. For some CAM therapies which insist that a treatment strategy should take account of the individual as well as the disease, such an approach would mean that the treatment fell so far short of the optimal therapy that results from such a trial would be irrelevant.
  
Individual responses to treatment: Some CAM therapists believe the outcome of any treatment results from a unique constellation of factors that are particular to each individual patient. Therefore the controlled trial which obscures individual differences in outcome in favour of group averages and generalisations can seem meaningless to them. Vincent & Furnham observe that trials are a simplification of the clinical situation and that the fundamental questions posed in a controlled trial concern groups of patients and not individuals. Therefore it is important to know what questions RCTs are addressing before criticising their approach or design.
  
The contributions of the patient and practitioner: The central purpose of a placebo-controlled trial is to separate the influence of the therapy from other factors such as psychological effects of the treatment or the doctor-patient relationship. The very fact that such care is taken to separate out these factors acknowledges the potential influence they have, and yet they are ignored. Vincent & Furnham suggest the lack of research on these factors is regrettable and that CAM practitioners with their emphasis on holism may especially find it so. Ideally, holistic beliefs should be incorporated into research on CAM, although they suggest it is not necessary always to include them for meaningful research to be carried out in this area. Research into patient and practitioner variables is extremely difficult and costly. Psychotherapy researchers have been painfully struggling with these issues for many years. Vincent & Furnham suggest that this means that, however important such issues are, they should not be the first priority for research into CAM.
  
Source: Vincent, C. & Furnham, A. (1997) Complementary Medicine: A Research Perspective. Chichester: Wiley & Sons.



7.26 Although the design of RCTs for CAM therapies may require very careful attention this is rarely impossible. We recommend that CAM practitioners and researchers should attempt to build up an evidence base with the same rigour as is required of conventional medicine, using both RCTs and when appropriate other research designs. How CAM may attract or train researchers with the skill to design such trials, and how funding may be gained to make such trials possible, will be discussed later in this chapter.

Qualitative Research

7.27 RCTs, sequential studies and comparative studies are all types of quantitative research. Qualitative research employs smaller samples to address more open-ended questions, for example: "what dimensions of x pose what risks, and to what?" Samples of this sort of research are typically too small to be representative, but one important role of qualitative research is to complement quantitative research by identifying and exploring variations of meaning in people's understanding and responses.

7.28 It is also possible to design good quality qualitative research; focus groups have successfully identified end-points which can later be used in an RCT. And rigorously designed questionnaires can provide answers which can be converted into validated quantitative scores. This can deal with issues such as quality of life or can be disease-specific, as in asthma or arthritis.

7.29 Quantitative and qualitative research designs are both important and can reveal different features of a treatment. Qualitative research is important in the early stages of research and has the benefit of being flexible and examining different aspects of the therapeutic intervention than the RCT examines; over the last few years the results from such studies have grown in acceptance and are being used to form part of a therapy's evidence base[49].

7.30 Some other qualitative research designs and their applicability to CAM are reviewed in Box 10.


Box 10
  
Some Forms of Qualitative Research: Their Advantages and Disadvantages for CAM Research
  
Case Studies - involve taking a detailed account of the effect of a treatment on an individual or a group. This method is good for studying rare clinical situations, for reporting new information on side effects etc. and for introducing new views and challenging existing theories. Reports from this type of study often provide hypotheses to be followed up in more formal studies. However, case studies cannot prove a hypothesis or generalisation as they are not controlled. Case studies are probably of limited use in CAM where there is already a mass of clinical description and many hypotheses that need testing.
  
Single Case Designs - an attempt to formalise case studies. For example, different treatments, (perhaps including placebo) may be given in sequence and the effects of each observed and compared with a baseline no-treatment stage. Single case designs have the advantages of being cheap and flexible in approach. They offer different levels of formality and rigour and can incorporate an emphasis on providing the best individually tailored care. Their methodological rigour can equal that of controlled trials. However, single case designs present various problems if treatment has long-term or irreversible effects. They are not suitable for broad questions about applicability of treatments across a range of individuals.
  
Clinical Audits - progress of patients under treatment is monitored and any adverse effects are noted. There is not usually any comparison group. Clinical audits have been developing strongly in conventional medicine for the last few years. They can be carried out on any aspect of a treatment, and are a good stepping-stone to producing formal studies. Clinical audit does not include a control group, so cannot prove or disprove a hypothesis.




45   Vincent, C. & Furnham, A. (1997) Complementary Medicine: A Research Perspective. Chichester: Wiley & Sons. Back

46   The National Institutes for Health is one of 8 health agencies in the public health service in the USA, which in turn is part of the US Department of Health and Human Services. The NIH is a research organisation, conducting research in its own laboratories and in universities, medical schools, hospitals and research institutes. NCCAM is one office within the NIH. Back

47   Kleijnen J et al, "Placebo effect in double-blind clinical trials: a review of interactions with medications". Lancet 1994. 344,1347. Back

48   A common feature of RCTs is that the treatment being administered is given in a standard form to all patients, not allowing for individual difference. Back

49   See our report Science and Society, 3rd Report 1999-2000, HL Paper 38.  Back