Alternative Medicine:
Expanding Medical Horizons

A Report to the National Institutes of Health on Alternative
Medical Systems and Practices in the United States

Part II:   Conducting and Disseminating Research

Introduction

This part of the report discusses issues that confront all the alternative medical systems and practices, particularly those whose efficacy is under investigation. Although alternative medicine does not differ from mainstream medicine in the need for reasonable, responsible research and validation of safety and effectiveness, there are some issues unique to this branch of medicine:

• Lack of dedicated alternative medical research facilities.
  
  

• Lack of adequate funding for alternative medical research.
  
  

• Lack of training for alternative medical researchers.
  
  

• Lack of an adequate, centrally located research database.
  
  

• Difficulties in matching appropriate research methods to subjects being researched.
  
  

• Difficulties in obtaining appropriate National Institutes of Health (NIH) peer review of alternative medical grant applications.
  
  

• Difficulties in data collection related to various legal and regulatory constraints.

Consumers and clinicians have a definite and immediate need for access to the best and latest information about alternative medical practices, and NIH needs to be able to hear from health care consumers about their experiences with alternative practices.

Research Infrastructure: Institutions and Investigators

PANEL MEMBERS AND CONTRIBUTING AUTHORS

David Eisenberg, M.D.--Chair
Barbara A. Brennen, M.S.
Seymour Brenner, M.D.
Deepak Chopra, M.D.
Serafina Corsello, M.D.
Michael L. Culbert, D.Sc.
Jonathan Davidson, M.D.
Patrick M. Donovan, N.D.
Robert Duggan
Judy Epstein
Thomas E. Harries, Ph.D.
Tori Hudson, N.D.
Norma Jennings
Abraham R. Liboff, Ph.D.
Nancy Lonsdorf, M.D.
Laura Nader, Ph.D.
Richard Pavek
Kenneth Pittaway, N.D., Ph.D.
Nelda Samarel, Ed.D.
Paul Scharff, M.D.
Oscar Carl Simonton, M.D.
James P. Swyers, M.A.
Marvin C. Ziskin

Introduction

This chapter reviews and discusses separately the following issues: the status of alternative medical research at conventional and nonconventional research institutions, the availability of properly trained investigators who can adequately and professionally investigate and validate potentially promising alternative medical treatments and systems, and the degree of exposure that conventionally trained medical students and researchers have to alternative medical principles. Recommendations are offered for revising the present-day research infrastructure to create opportunities for alternative medical research.

Status of Alternative Medical Research in the United States

Conventional Research Institutions

Although there are pockets of alternative medical research going on at many conventional scientific research institutions across the United States, including the National Institutes of Health (NIH), by far most alternative medical research is being conducted outside such institutions. A major factor that promotes conventional research over novel research at most institutions in this country is the peer review process, which is intended to prevent poor research from being funded or disseminated.

Peer reviewers have a major role in shaping the general direction of all research. If peer reviewers favor conventional research, they may be inclined to fund research proposals that stay within the bounds of the conventional. Thus, in this country, which prides itself on innovation and discovery, there is an increasing tendency to do only "safe" research. Researchers who cannot get their results published and/or funded because of peer review bias can have only limited careers in the research sciences. To be an unpublished investigator is to be isolated; to be published is to obtain status. An investigator in academia who does not publish enough original research within a certain time likely will not be tenured and may be terminated altogether. Furthermore, once published, the investigator must continue to publish regularly to remain employed and employable. The pressure to publish breeds a tendency to perform research that builds logically and in small steps upon generally known or suspected phenomena, rather than research on novel ideas (see the "Peer Review" chapter for more details on the shortcomings of the peer review process).

Therefore, investigators wanting to pursue novel research projects often hesitate to step far outside the conventional path. If they do try to pursue something revolutionary, they may find it hard to obtain funding or may be advised by their peers that pursuing "offbeat" ideas can lead to lost status, unpublished work, lack of funding for even conventional projects, or being shunned by other researchers who fear disapproval by association (Sherrill and Larson, 1993).

Favoring the conventional is also a factor in funding researchers because many funding sources use peer review committees of researchers as grant reviewers. Although funding for research projects and investigators comes from many sources, Government funds are the largest source in this country (U.S. Department of Commerce, 1989). Foundations, universities, colleges, and private sources are lesser but highly important sources, especially for alternative medicine investigators. How NIH, a large dispenser of biomedical research funds, approaches the evaluation of alternative medical research proposals is likely to have a major influence on attitudes toward alternative medical research in the United States.

Nonconventional Research Institutions

Several alternative medical colleges have research departments and are actively engaged in research. Their approach to research usually differs from that of conventional medical institutions because of different emphasis and less exposure to methodological training. First, what is alternative to the conventional institution is normal to the alternative institution. Second, research that would be considered basic at a conventional institution--such as that which asks whether something works at all--is frequently not an issue for the alternative institution. Researchers at unconventional institutions are much more likely to be interested in determining dosages or conducting outcomes studies than in investigating whether or how something works. In contrast, the interest of conventional researchers in dosages and outcome studies is more likely later in the research process than near the beginning.

Most of the 16 chiropractic colleges and all of the osteopathic medical colleges have research facilities. Other institutions with research capacities in the United States include Bastyr University (Seattle), National College of Naturopathic Medicine (Portland, OR), Southwest College of Naturopathic Medicine and Health Sciences (Scottsdale, AZ), and the Traditional Acupuncture Institute (Columbia, MD). Many current research efforts at these institutions are in long-term health issues, the very issues that are of current concern for conventional medicine and the public health system._

However, funding is precarious for these institutions. Almost without exception, NIH funding has not been available. Limited funding is available from private sources but is inadequate for current needs. Because of the limited funding, research departments at such institutions have had only minimal development of infrastructure and faculty. Further, at present there is little communication between these research facilities and their conventional counterparts, even though increased communication may benefit this country's health care.

Alternative Medical Investigator Training

Today, alternative medical researchers represent a spectrum of disciplines and training. Many of these researchers are conventionally trained investigators who see in alternative medical practices and approaches to health a means of addressing some shortcomings of conventional biomedicine. Others are trained by and conduct research at nonconventional colleges and institutions devoted to systems of health that derive from nonconventional perspectives. In addition, there are disciplined investigators in the social sciences who see strong connections between their daily work and healing disease. Still others are less formally trained but believe they have developed the ability to heal others through various direct, personal means; these individuals may have little or no formal academic training but may spend time in clinical investigation.

The disparate groups in alternative medicine need training to become accomplished alternative medical investigators. Some require training in proper and acceptable research methods, and others need exposure to alternative medical practices so as to be better prepared to evaluate those practices properly. Indeed, the basic contention here is that individuals conducting research in alternative medicine are more likely to be successful if they have some level of dual training in conventional medical research methodology and a field of alternative medicine. Further, alternative medicine practitioners have suggested that research in alternative medicine should be performed by individuals and teams trained in as wide as possible an array of research methodologies.

Exposure to Alternative Medical Principles in Conventional Medical Schools

Training of medical and health investigators in colleges and universities begins at the undergraduate level. Currently, most undergraduate institutions and conventional medical schools and teaching hospitals do not offer exposure to alternative medical practices or views. This omission from the standard medical curriculum adversely affects the use of alternative medicine in the clinic and the nature and extent of biomedical research.

Nevertheless, several mainstream medical institutions have recently begun or are developing basic academic medical courses to introduce medical students and physicians in training to the history, theory, and practice of alternative medical therapies. Currently, there are courses or programs at the following universities: Arizona, Columbia, Georgetown, Harvard, Louisville, Maryland, Michigan State University, Stanford, Tufts, the University of California at San Francisco, and Virginia. The courses are potentially the foundation for future systematic exploration of alternative medicine practices at these schools.

A few other conventional institutions integrate alternative medicine in at least a limited way into their curriculums. For example, several nursing colleges, universities, and teaching hospitals are currently providing practical courses in one or more of the biofield therapies and in biofeedback, yoga, or meditation. It is likely that additional opportunities exist to train in alternative medical practices in some departments of psychology, anthropology, and social sciences.

If they are properly designed, courses like these will not only provide information on the utility of specific therapeutic approaches but also develop a larger framework for understanding the strengths and limitations of Western medicine. They also will promote recognition of the contributions that theoretical and research models in alternative medicine may make to enlarging conventional research methodology.

Recommendations

The following are specific recommendations relating to improving the research infrastructure so that there are fewer inherent barriers to those interested in pursuing research into topics that do not necessarily fall into predefined categories.

Improving Research Infrastructure

The Office of Alternative Medicine (OAM) should make it a priority to survey the basic needs of alternative medical research institutions and help arrange funding and other support. In addition, it should look for ways to encourage the ongoing work in alternative medicine being conducted at conventional institutions. Bringing these alternative and conventional facilities into the same arena could create new dialog to enhance all medical research efforts.

However, it is not enough to increase the funding for alternative medical research at existing institutions; a genuine atmosphere of collaboration must be fostered. In order for alternative medicine research to proceed with reasonable speed, there must be dedicated alternative medicine research centers with support facilities. The following are all viable approaches that should be considered:

• First, it is recommended that OAM be upgraded to become the Center for Alternative Medicine (CAM) similar to NIH's current National Center for Research Resources (see sidebar). With CAM as a freestanding unit, NIH would be able to fund, as well as investigate, systems and processes that fall outside the normal purview of other freestanding NIH units.
  
  

  Unlike current institutes at NIH where alternative medicine research is required to compete for priority with many other subjects, the proposed CAM could concentrate on alternative medicine, serving as both a grant-funding agency and an "in-house" evaluator of alternative medical practices.
  
  

  If NIH were to lead the way in this fashion, universities and medical colleges would be encouraged to begin their own alternative medical research programs. Researchers funded by CAM would gain experience in alternative medical research and could use their new expertise to become the core faculty of research facilities in independent medical institutions.
  
  

• It is recommended that existing research centers be enhanced and new ones installed at alternative medical institutions throughout the United States. Here, also, expert faculty from various disciplines would join to evaluate efficacy, safety, cost-effectiveness, and mechanisms of action of alternative medicine through basic and clinical research.
  
  

• It is recommended that new research centers devoted primarily to the assessment of alternative medicine be founded at leading universities and conventional medical institutions throughout the United States. Expert faculty members from various disciplines would evaluate efficacy, safety, cost-effectiveness, and mechanisms of action of alternative medicine through basic and clinical research. Where appropriate, they would also compare different methodologies for conducting these evaluations. (This concept has received considerable support from proponents of alternative medicine.)

Ideally, these centers would create an academic "critical mass" that would begin to bring the conventional and alternative medical communities together. This approach may be an effective way of generating authoritative, dispassionate investigations of alternative medicine. The establishment of research centers of this type would complement, not replace, the proposed CAM or intramural or extramural investigations of existing NIH institutes.

At the new and enhanced centers, highly trained research investigators and alternative medicine practitioners would collaborate to conduct interdisciplinary research, developing protocols and implementing clinical investigations. Core faculties at the centers would develop protocols for initial review by an institutional research advisory board, which would be responsible for critical review of each protocol for clinical importance, methodological soundness, and administrative feasibility. Advisory boards could include the OAM director, ex officio, thus ensuring ongoing collaboration with NIH.

Although each center would receive sufficient funding for protocol development and the implementation of small feasibility or pilot studies, larger trials (such as those requiring hundreds of patients randomized to a variety of experimental or control conditions) would likely require additional funding from NIH, the Agency for Health Care Policy and Research (AHCPR), or other public or private sources. Larger outcome studies and surveys could use multiple research centers as individual sites for ongoing studies.

These centers would need sufficient funding for a core faculty, support of a research board, a modest fellowship program, and a library database. Funding of specific projects would be determined competitively.

The research centers could take different forms; for example, they might be modeled after the comprehensive cancer centers established and funded by the National Cancer Institute (NCI). The programs of these NCI-funded centers must include several key elements: basic laboratory research, clinical research, and linkages between basic and clinical research; high-priority clinical trials research; research on prevention and control; education and training of researchers and health care professionals; public information services; and community service and outreach.

Development and Funding of Research Projects

Should funding become more widely available for alternative medical researchers, a natural offshoot would be more institutions taking an interest in conducting alternative medical research, which would necessarily attract more trained investigators. The following are recommended as ways to support research projects:

• OAM's current competitive request for applications (RFA) program (see the "Peer Review" chapter) should be continued and expanded. In its first round of 30 research grants, OAM encouraged collaborations between medical researchers and practitioners of alternative medicine; proposals were reviewed on a competitive basis and funded accordingly. (NIH reported that the first RFA elicited more than 800 inquiries and 463 grant applications.)
  
  

• In addition to its current RFA program, OAM should initiate RFAs to perform selected clinical trials and certain critical experiments, specifically with the aim of validating previously reported results, resolving apparent conflicts, and testing new approaches.
  
  

• OAM should also assemble one or more patient outcomes research teams (PORTs) based on the AHCPR model. The PORTs could conduct multiple field investigations. In addition, or alternatively, OAM should hire several full-time field investigators, as has been recommended by two U.S. senators and the office's first advisory committee, the 1993 Ad Hoc Advisory Panel to the Office of Alternative Medicine.
  
  

• OAM should investigate ways to provide joint public-private funding of its research and educational programs. Expansion of the funding base would increase opportunities for alternative medical research. Individual foundations and philanthropists may be interested in providing cofunding for a variety of initiatives concerning alternative medicine research and training.

Incentive Activities

• RFAs should be generated by individual institutes at NIH seeking clinical and basic research projects from investigators who are receptive to researching alternative medical practices relevant to the missions of those institutes.
  
  

• Some incentive process should be developed to encourage experienced, previously funded investigators to add experts in alternative medical practices to their investigation teams and to add study arms involving aspects of alternative medical practices to their existing research foci. The incentive process could provide for earmarked funds or special scoring during the rating of proposals.
  
  

• RFAs should be generated by AHCPR, a Public Health Service agency within the Department of Health and Human Services, for conducting alternative medical research relevant to the outcomes research this agency sponsors for the determination of appropriate clinical practices.
  
  

• A similar funding or rating incentive scheme could be used to encourage broadening AHCPR-sponsored research designs and clinical care studies to include alternative medical practices.

A vigorous, broadly based peer review system with participation by experts in alternative medical practices will be necessary to ensure that any such affirmative research incentives are awarded to studies where the research really does test appropriately designed alternative interventions.

Establishment of an OAM Research Project Database

To address its research project needs, it is recommended that OAM do the following:

• Establish a database of ongoing research projects and project proposals on alternative medicine that is readily available to researchers. Such a listing of both existing and proposed research projects could enhance alternative medical research by preventing redundancy and fostering joint efforts by several researchers. The database could include OAM-funded research, other NIH-funded research (many institutes currently have ongoing alternative medical research projects),_ and all other identifiable alternative medical research that has been approved by an institutional review board.
  
  

• OAM should also develop a clearinghouse function within OAM for planned projects so that unnecessary duplicate studies can be avoided and appropriate collaborations encouraged.

In addition, OAM should do the following:

• Facilitate access to information and guidance concerning all aspects of methodology from study planning through conduct, analysis, and development of reports or manuscripts for publication. (See app. F.)
  
  

• Develop a list or network of experienced investigators willing to help with such designated areas as project development, conduct, and analysis; link alternative and conventional researchers; provide methodology seminars and workshops; and offer other services on a volunteer or consultant basis. OAM should organize the network and maintain responsibility for planning and hosting seminars.

Fostering Collaboration on Existing Studies Within the Federal Government

For collaboration on such existing studies, OAM should perform the following:

• Identify ongoing randomized controlled trials supported by NIH that could allow for the simultaneous testing of alternative medical therapies as adjuncts or additional experimental conditions. For example, a controlled trial assessing the efficacy of chemotherapy for a particular cancer could allow for patients to be randomized to chemotherapy alone or to a group receiving chemotherapy and an alternative medical practice such as herbal treatment or visualization.
  
  

• Include alternative medical practices among the procedures under study when AHCPR conducts outcomes research. This could be accomplished by including in the planning process a spectrum of alternative medical practice researchers to help create study protocols of alternative methods to compare with the conventional methods, focused on specific outcomes (e.g., optimal recovery from acute back pain, optimal recovery from surgical intervention, or optimal management of conditions for which both medical and surgical options are under consideration, including heart disease and prostate disease).

Interactions Within the Public Health Service

Enacting the following recommendations could strengthen alternative medical research and training in the United States:

• OAM should develop and maintain a close working relationship with the Food and Drug Administration (FDA) and with NIH's Office of Protection for Research Risks to ensure that the highest standards of protection of human subjects are applied to all aspects of alternative medical research.
  
  

• OAM should establish and maintain a working relationship with FDA to better assist alternative practitioners with the drug development and regulatory processes. OAM and FDA should also ensure that protocols for testing alternative treatments and regulating relevant devices are commensurate (that is, neither unduly restrictive nor unduly lenient) with the risks involved in their use and are based on appropriate scientific principles.
  
  

• Since many questions relating to alternative medicine involve outcomes studies, OAM should investigate areas of mutual interest with AHCPR, which presently does such studies.

Interactions With Other Countries

In pursuit of international collaboration, OAM should explore the following:

• Learn about alternative medical practices in other countries and establish collaborations with their alternative medical practitioners. For example, it is well known that Great Britain has considerable clinical experience and government interest in "complementary medicine." Extensive professional and government interest in alternative medicine have also been shown in other European nations (e.g., Germany), as well as in China, Japan, India, and the former Soviet Union.
  
  

• Explore areas of mutual interest with appropriate government agencies from other countries with interests in alternative medicine and encourage collaborations with NIH's Fogarty International Center.

Upgrading Medical Education

It is recommended that comprehensive programs be developed and disseminated as soon as possible to bring an understanding of alternative medicine and its practices into conventional medical education. These programs should include both theoretical presentations and practical approaches in alternative medicine. OAM could speed the process greatly by hosting a conference of interested institutional administrators and helping to develop an implementation plan.

Theoretical courses in such a plan should include the following:

• The history of medicine, including alternative medicine and medical education.
  
  

• Perspectives on how alternative medicine may enrich and enlarge contemporary medical education.
  
  

• Western philosophical and medical perspectives on the "mind-body relationship."
  
  

• Philosophical bases, research literature, and clinical effectiveness of systems of healing from one's own and other cultures and with other parameters (e.g., Chinese, African, Indian, Native American, homeopathy, biofield therapeutics).
  
  

• Effects of social context--including family, socioeconomic status, culture, race, and gender--on health and illness. (Studies of this nature have been funded by NIH for many years; inclusion in a curriculum is needed now.)

Practical courses in an implementation plan could include the following:

• Experiential training in alternative medicine practices taught by skilled alternative medicine practitioners (e.g., biofeedback, meditation, guided imagery, manual therapies, hypnosis, yoga, tai chi, biofield therapeutics).
  
  

• Critical evaluation of the most current and significant data on alternative medical research.
  
  

• Use of case studies to illustrate the effects of clinical practices in alternative medicine.
  
  

• Examination of the ways alternative medical practices may be integrated into various training and teaching experiences and into comprehensive programs in different medical specialties.
  
  

• Implications of alternative medicine in our understanding of individual psychology and psychobiology, the physician's role and self-concept, and the doctor-patient (or patient-doctor) relationship; awareness of different cultural concepts concerning the relationship between mind and body; and interest in undertaking research studies of various alternative medical practices and the ways they might be fruitfully integrated with conventional practices.

Further steps can be taken in upgrading medical education:

• University-sponsored continuing medical education courses in alternative medicine can and should be made more broadly available to health care providers, including physicians, nurses, dentists, pharmacists, other allied health professionals, and medical school faculties. This education may take the form of consensus conferences, workshops and symposia, continuing medical education programs, tutorials, or lectures to be offered at annual meetings of selected medical societies and associations. For example, annual meetings of psychologists, neurologists, or endocrinologists could include guest lectures on the current state of science in the field of prayer healing or cognitive behavioral therapies. Conferences and symposia of this kind could facilitate communication and collaboration between the conventional medical community and practitioners of alternative medical therapies.
  
  

• The various centers proposed later in this chapter could also develop continuing medical education programs and assist OAM with the development of conferences and symposia to be held either at NIH or at individual research center sites.
  
  

• Implementation of this interdisciplinary training should be through research training fellowships.

Several funding options exist:

• Established faculty members at conventional medical institutions could spend several months or longer in settings where alternative medicine practices are taught and practiced. Doing so would familiarize the academic faculty members with the theory and practice of a given alternative medical technique and would enable them to participate in designing protocols with increased clinical insight.
  
  

• Individuals presently trained in alternative medical practice or conducting alternative medical research could receive grants to support several months or years of training in research methodology. These individuals could be supported to undertake master's, doctoral, or postdoctoral research fellowships and then participate in protocol development regarding alternative medical practices.
  
  

• OAM could identify existing fellowship training programs that may be receptive to pursuing research in alternative medicine. Fellows trained in this fashion would have the advantage of working in partnership with highly trained research scientists. For example, the Robert Wood Johnson Clinical Scholars Program or fellowship training programs administered by individual institutes within NIH may be willing to add fellowship slots earmarked for clinical or basic alternative medical science, relevant social sciences, or the biophysical sciences needed to pursue research in a field such as bioelectromagnetism. This approach would take advantage of expert faculty currently in existing training programs.
  
  

  Funds could be made available to existing fellowship programs to ensure the additional positions in both intramural and extramural NIH research programs.
  
  

• OAM could support an annual or biennial competition for the best original research proposal by nonacademics on a presently unresearched idea.

References

Sherrill, K.A., and D.B. Larson. 1993. The anti-tenure factor in clinical research in clinical epidemiology and aging: diagnostic assessment and treatment recommendations. In J.S. Levin, ed. Religion in Aging and Health: Theoretical Formulations and Methodological Frontiers. Sage Publications, Thousand Oaks, Calif.

U.S. Department of Commerce. Bureau of the Census. 1989. Statistical Abstract of the United States 1989 (109th ed.). U.S. Department of Commerce, Washington, D.C.

Sidebar

National Center for Research Resources

The National Center for Research Resources (NCRR) develops and provides the shared resources essential for biomedical research funded by NIH: research project and resource grant support are used to develop cutting-edge biomedical research technologies and sophisticated instrumentation; to locate and characterize the most appropriate models for the study of human disease; to establish and maintain clinical environments in which technology can be transferred from the laboratory to the bedside; and to develop research capability in minorities and minority institutions. NCRR funds the following programs and research centers:

• Biological Models and Materials Research Program
  
  

• Biomedical Engineering and Instrumentation Program
  
  

• Biomedical Research Support Program
  
  

• Biomedical Research Technology Program
  
  

• Comparative Medicine Program
  
  

• General Clinical Research Centers Program
  
  

• Research Centers in Minority Institutions Program

---

Research Databases

PANEL MEMBERS AND CONTRIBUTING AUTHORS

Carola Burroughs--Chair
Jonathan Collin--Cochair
Gerald Bodeker, Ed.D.
Carlo Calabrese, N.D., M.P.H.
Aimee Carruth
Peter Chowka
Jonathan Collin, M.D.
Wayne B. Jonas, M.D.
David Larson, M.D., M.P.H.
Kevin McNamee, D.C., L.Ac.
Richard Pavek
Anne Phillips, J.D., M.S.L.
James P. Swyers, M.A.

Introduction

The first step in developing a research strategy is to study previously published research literature on the subject and related subjects. A centralized source of information on a medical system or particular therapy allows investigators to go directly to the most current and best research on a topic rather than wasting valuable time attempting to collect data from disparate sources. Thus, if investigators have access to a comprehensive research database, they can avoid repeating existing research and can obtain vital information for designing their own research.

Unfortunately, research into alternative medicine has been hampered because there is currently no easily accessible comprehensive database. Although a great deal of information can be found in the major medical databases on various aspects of alternative medicine, expert searching skills are needed to locate these materials. In addition, much of what has been collected on alternative medicine in the major medical databases has not been sufficiently indexed and cataloged. The problem is compounded if there are no journals available for a particular alternative discipline, if the relevant journals are not indexed and cataloged for inclusion in the databases, or if the data were not collected or reported properly. Further, other potentially valuable information is available only in foreign-language sources, such as the substantial bodies of literature on traditional Chinese and traditional Ayurvedic medicines. Consequently, a common complaint shared by the researchers of this report was the difficulty of tracking down material on alternative medicine that was known to exist but that nevertheless could not be located.

National Library of Medicine and MEDLINE

The premier source of medical science research information in the world is the U.S. National Library of Medicine (NLM). To make research information as accessible as possible, NLM has indexed much of its 16 million printed references into a computer-based bibliographic retrieval and publication system called MEDLARS (Medical Literature Analysis and Retrieval System). MEDLARS is accessible through more than 40 online electronic databases and databanks. The database of greatest interest to alternative medical researchers is MEDLINE (MEDLARS on Line). Most medical and health investigators in the United States turn first to MEDLINE for research materials.

MEDLINE contains more than 7.2 million records, with some 31,000 new citations added each month. Although the full text of each article is not in the database, approximately 60 percent of the citations contain author-generated abstracts or summaries of the articles. (Researchers may order copies of the full text of the articles that are indexed in MEDLINE from NLM.)

MEDLINE is readily accessible either through Grateful Med search software (available from the National Technical Information Service) or by directly dialing NLM via standard online communications software, such as ProComm or Awremote. (For information on ordering Grateful Med and on using MEDLINE to conduct literature reviews, see "Information on the National Library of Medicine" in app. F.)

Current MEDLINE Indexing Terms for Alternative Medicine

MEDLINE uses a "controlled vocabulary," or "key words," indexing system called MeSH (Medical Subject Headings) to access information. There are now more than 18,000 MeSH headings and subheadings. MeSH also includes more than 20,000 chemical term records.

Inadequacies of MeSH Listings for Alternative Medicine

The 20 headings currently listed in MEDLINE that are relevant to alternative medicine are the following:

Acupuncture

Anthroposophy

Biofeedback

Chiropractic

Color therapy

Diet fads

Eclecticism

Electrical stimulation therapy

Homeopathy

Massage

Medicine--traditional

Mental healing

Moxibustion

Music therapy

Naturopathy

Radiesthesis

Reflexotherapy

Rejuvenation

Relaxation techniques

Tissue therapy

More specific titles and subjects are listed under those headings; for example, Ayurvedic and herbal medicines are both subheadings under traditional medicine. An April 1994 search of the available headings for alternative medicine brought up 29,080 citations (entries) dating back to 1966. Although NLM recently has made great strides in making more alternative medicine research literature more readily accessible, the MeSH headings used by NLM do not yet include many of the key words used in alternative medical therapies. For example, craniotherapy (a common term in chiropractic) and therapeutic touch (a practice in biofields therapeutics used by many nurses and others) are not indexed by NLM. A search for either of these terms will find nothing, even though there are articles on these subjects in NLM's database. Thus, although there are many articles relating to alternative medicine from conventionally focused peer-reviewed journals on MEDLINE, the researcher often has difficulty finding them. Such incompleteness in MeSH terms for alternative medicine is a major obstacle in implementing research on this subject.

NLM is aware of the increasing interest in alternative medicine and the need for adequate MeSH terms. NLM has contacted the Office of Alternative Medicine (OAM) and asked the office to review the current array of terms, make suggestions for new terms, and work with NLM to improve the indexing for alternative medicine.

Alternative Medical Journals Currently in MEDLINE

The list of alternative medical journals now being indexed by NLM is inadequate for current research needs. For example, MEDLINE at this time abstracts only 3 of the 16 journals available on chiropractic and carries no journals relating to homeopathy, naturopathy, or orthomolecular medicine.

The following are the journals relating to alternative medicine currently indexed in MEDLINE:

Acupuncture and Electro-Therapeutics Research

American Journal of Chinese Medicine

Biofeedback and Self Regulation

Chen Tzu Yen Chui (Acupuncture Research)

Chinese Medical Journal

Chung-Hua I Hsueh Tsa Chih (Chinese Medical Journal)

Chung-Kuo Chung Hsi I Chieh Ho Tsa Chih

Chung-Kuo Chung Yao Tsa Chih (China Journal of Chinese Materia Medica)

Journal of Manipulative and Physiological Therapeutics

Journal of Natural Products

Journal of Traditional Chinese Medicine

Planta Medica

In the NLM stacks are other related journals not indexed on MEDLINE (MEDLINE does not index all the journals related to any field).

The fact that MEDLINE does not include articles published before 1966 especially affects alternative medical research. Medical research literature before the 1960s contains a wealth of information on such practices as botanical medicine, homeopathy, hydrotherapy, nutrition, and manipulation. Research on these alternative processes slowed to a near standstill when medical focus shifted to manufactured drugs (before MEDLINE existed). Therefore, even though such information may still be available, possibly in the NLM stacks, it is largely out of print and unavailable to present-day researchers unless they know the information exists.

Other Alternative Medical Journals Proposed for Inclusion in MEDLINE

The following is a partial list of serials that alternative practitioners have proposed to OAM for inclusion in MEDLINE:

Acta Pharmacologica Sinica

Acta Pharmacutia Sinica

Advances, the Mind-Body Journal

Aktuelle Ern@hrungsmedizen

Alternatives

American Academy of Medical Acupuncture Review

American Chiropractor

Antha

Archives of Physical Medicine and Rehabilitation

Arzneimittel-Forschung

Australian Journal of Medical Herbalism

Biological Therapy

Birth Gazette

Brain/Mind and Common Sense

British Homeopathic Journal

British Journal of Clinical Pharmacology

British Journal of Midwifery

British Journal of Phytotherapy

Complementary Medicine Index

CP Currents & CP News

Current Medical Research Opinion

Economic Botany

Explore

Fitoterpia

Fortschritte der Medizin

Foster's Botanical and Herb Review

Frontier Perspectives

Gan To Kagaku Ryoho

Health Facts

Herbal Update and Natural Healthcare Quarterly

HerbalGram

Human Ecologist

Indian Journal of Homeopathic Medicine

Indian Journal of Medical Research

International Clinical Nutrition Review

International DAMS Newsletter

International Journal of Biosocial and Medical Research

IRCS Medical Science Library Compendium

Korean Biochemistry Journal

Journal of the Acupuncture Society of New York

Journal of Alternative and Complementary Medicine

Journal of the American Academy of Osteopathy

Journal of the American Institute of Homeopathy

Journal of Anthroposophic Medicine

Journal of Ethnopharmacology

Journal of Manual and Manipulative Therapy

Journal of Musculoskeletal Medicine

Journal of the National Academy of Acupuncture and Oriental Medicine

Journal of Naturopathic Medicine

Journal of Nurse Midwifery

Journal of Nutritional Medicine

Journal of Orthomolecular Medicine

Journal of Spinal Disorders

Journal of Traditional Acupuncture

Klinische Monatsbl@tter fhr Augenheilkunde

Massage Therapy Journal

Medical Anthropology

Medical Anthropology Quarterly

Medical Herbalism

Midwifery Today

New England Journal of Homeopathy

Onkologie

Orvosi Hetilap

Pharmacologic Biochemic Behavior

Phyto-Pharmica Review

Professional Journal of Botanical Medicine

Progressive Clinical Biological Research

Quintessence

Resonance

Simillimum

Social Science and Medicine

STEP Perspectives

Townsend Letter for Doctors

Veterinary and Human Toxicology

Vital Communications

Western Journal of Medicine

Zeitschrift fhr Phytotherapie

Other Indexes

Like MEDLINE, Science Citation Index (available at NLM but not available on line) and Index Medicus (a bound listing of references without abstracts) suffer from lack of early alternative medical articles and inadequate indexing.

If searches for alternative medical research focus on medical journals, some other reference sources that address relevant issues are likely to be overlooked. Among these are Social Science Citation Index, Cumulative List of Nursing and Allied Health Literature, Agricola, National League of Nursing International Nursing Index, and Folklife Center Database.

National Library of Medicine Selection of Journals

The NLM procedure for reviewing and accepting journals of current interest is appropriately rigorous; unfortunately, there are no alternative medicine investigators or practitioners on the Literature Selection Technical Review Committee at this time.

OAM should propose expert candidates for the NLM selection committee and submit their names to the Associate Director of Library Operations, NLM.

The NLM staff has expressed awareness of an increasing need for additional alternative medical journals and materials to be included in MEDLARS and MEDLINE and have indicated that when OAM submits materials, NLM will present them to its Literature Selection Technical Review Committee (Pavek, 1994).

Other Important Databases and Databanks

CATS, the British Library Medical Information Service's Current Awareness TopicS in allied and alternative medicine, is the next largest public database on alternative medicine after MEDLINE. (The British Library is the equivalent of the U.S. Library of Congress.) CATS currently lists more than 50,000 entries for such disciplines as Ayurvedic medicine, chiropractic, homeopathy, naturopathy, occupational therapy, oriental medicine, osteopathy, and physiotherapy. Arrangements are under way to make CATS commercially available on line through a U.S. company, probably in 1994.

It may be difficult, however, for clinical and basic sciences researchers to extract much useful material from CATS, because the database is a mixture of everything from peer-reviewed journal articles to newspaper clippings. Nevertheless, much material in CATS can benefit alternative medical researchers in this country when it becomes available.

NAPRALERT (Natural Products Alert), a database of the College of Pharmacy of the University of Illinois at Chicago, contains bibliographic and medicinal, pharmacologic, taxonomic, and chemical information on a great number of natural product extracts. This database contains more than 100,000 records, some dating as far back as 1650.

A growing number of alternative medical journals and publications carry research and clinical findings of varying levels of scientific rigor. The following databases are supported by or are specific to individual professions:

• CHIROLARS, a computer database (available on CD-ROM), contains 16,000 journal abstracts and conference proceedings in the field of chiropractic. Data are also available in book form in the Index to Chiropractic Literature and the Chiropractic Research Abstracts Collection, a compilation of more than 6,000 journals, journal articles, and books.
  
  

• A traditional Chinese medicine database at the American College of Traditional Chinese Medicine in San Francisco was begun with a small grant from the rock music group the Grateful Dead; lack of further funding has stalled the project.
  
  

• An electronic database of traditional Chinese medicine is available in Beijing.
  
  

• Ayurvedic databases are available in India.
  
  

• Alternative medical databases in various fields are available in Europe.

Research Database Enhancement

The role of OAM is pivotal in facilitating the needs of alternative medical researchers and meeting the information requirements of alternative medical health care practitioners. Enhancing the NLM national database, MEDLARS, should be one of the first steps.

Logic

Combining data from alternative medical journals and other research materials into the NLM's MEDLINE database and databanks--as opposed to developing a separate OAM-sponsored, comprehensive research database--is advisable for several reasons:

• A unified research database is consistent with the intent (as stated in the preface to this report) of incorporating the best of the alternative medical systems into the present U.S. medical health system rather than developing a separate medical health system.
  
  

• A separate database would continue the current subordination of alternative practices by making them appear unworthy of inclusion in MEDLINE.
  
  

• Having separate comprehensive databases would require that future researchers access at least two databases to complete their literature reviews.
  
  

• A separate research database would require considerable unnecessary duplication of effort, as well as additional OAM expense.
  
  

• Improving the NLM database is essential for communicating alternative medical treatment issues to conventional health care practitioners. These practitioners, who are responsible for the safety and efficacy of their health care practices, require guidance when a controversy exists about therapeutic or diagnostic options. Since they already use MEDLINE and other NLM resources, forcing practitioners to access a separate database would cause confusion and not serve alternative medicine's best interests.
  
  

• Often, materials on alternative medical systems from conventional medical journals that are referenced in MEDLINE are antagonistic to alternative medical research and practices. When offsetting information from alternative medical journals becomes available in MEDLINE, naive readers will not be left, as they are now, with the misleading impression that little research has been done and that the worth of the particular alternative medical practice is a negatively settled issue. When both sides of an issue are available, as is the case with most conventional issues, the responsible researcher is able to more fairly evaluate the situation.
  
  

• The insufficiency of alternative medical information on MEDLINE promotes the fiction that there is little research on alternative medicine; consequently, conventional researchers are unlikely to initiate investigatory efforts (Easterbrook et al., 1991; Kleijnen and Knipschild, 1992; Knipschild, 1993).

Though useful to researchers, MEDLINE is not suitable for use by the general public. A separate resource database is needed for consumer information (see the "Public Information Activities" chapter).

Directional Oversight

OAM should establish a standing research database committee to prepare materials for and to communicate with the NLM staff. This committee should be composed of knowledgeable alternative medical investigators and practitioners, historians, and other experts.

This undertaking will require scientific and technical competence so that information can be acquired, analyzed, and prepared in a timely, efficient, and cost-effective manner. The participation by experts in the various subjects will allow for thorough analysis of the range of alternative medical disciplines and literature. This will also help maintain the currency and vitality of the selected items and indexing terms.

Communications will be enhanced by having one member of the research database committee serve on the public information clearinghouse committee (this recommendation is proposed in "Public Information Activities").

Specific Objectives

The following specific objectives should be addressed:

• Vocabulary. The MeSH headings used by NLM in MEDLINE provide a meaningful hierarchy of indexing terms. An OAM research database committee should compile a list of additional alternative medical terms to enrich the MeSH headings now available. In doing so, the committee should follow accepted principles of NLM lexicography (in accordance with the MEDLARS Indexing Manual, parts I and II) and confer with the NLM staff.
  
  

• Periodicals. Many journals and other materials need critical and prompt evaluation to determine whether they should be recommended to NLM's selection committee for inclusion in MEDLARS and MEDLINE. Prescreening by an OAM research database committee using NLM's selection standards will speed up the process of inclusion.
  
  

• Regular updating. After the research database committee develops and initially applies a method of scanning literature for quality and inclusion, it should develop a process for regularly updating the list of included materials. Library professionals in health science from the academic and research communities should be invited to participate by submitting materials as they come to notice. The process should also include future listings that emerge from CATS and other major foreign electronic libraries.
  
  

• Funding. Funding possibilities for acquisition of presently uncataloged materials should be explored by the OAM research database committee in conjunction with NLM.

Data Collection and Dissemination

Data Collection

At this time, no centralized data collection process exists for gathering information on alternative medical practices, "anomalous healing events," and seemingly odd or extraneous sudden improvements in health and cure rates in individuals. To support both the research database and the consumer information clearinghouse detailed in this report, and to document evidence of improvements attributable to alternative medical practices, continual collection of a wide range of data will be needed.

Dissemination

Some research findings in alternative medicine will have immediate applicability to health care practices. OAM should implement a process to disseminate such research findings to the public and the biomedical community as they become available. The aim is to improve alternative medical health care treatment and prevention practices in a coordinated way as the knowledge base expands. This dissemination task falls under the aegis of the public information clearinghouse (see "Public Information Activities").

Specific Recommendations

The following can help significantly in building research databases:

• Enhancing databases should begin with a MEDLINE search for already indexed materials. Consideration should also be given to adding material currently carried in NLM stacks but not yet on-line.
  
  

• Methods should be selected or developed to scan the quality of information available to ensure the appropriateness of the material included. A mechanism to rate the quality of supporting scientific evidence could assist in identifying promising practices. Furthermore, such a search may well turn up areas of practice especially worthy of further investigation.
  
  

• Databases available from professional associations, schools, foundations, individuals, and corporations should be surveyed. The National Institutes of Health should solicit information on other information resources, which should include all alternative medical materials of note identified in publications in medical social science and medical humanities.
  
  

• Alternative medical literature currently available in foreign languages should be surveyed and then translated and included where appropriate. This is particularly important for those alternative medical therapies in use in the United States, such as the medicines of Asia, that have roots in other cultures. Early emphasis should be placed on Japanese, German, Chinese, and French literature.
  
  

• OAM should investigate the indexes of CATS for relevant materials.
  
  

• OAM should consult with State and national associations of various alternative health care professions and libraries for information and advice on key indexing words to be included from the various alternative disciplines. Continuing interaction with these groups will be necessary in order to ensure that the database is updated regularly.

---

References

Easterbrook, P.J., J.A. Berlin, R. Gopolan, and D.R. Matthews. 1991. Publication bias in clinical research. Lancet 337:867-872.

Kleijnen, J., and P. Knipschild. 1992. Review articles and publication bias. Arzneimittelforschung 42:587-591.

Knipschild, P. 1993. Searching for alternatives: loser pays. Lancet 341:1135-1136.

Pavek, R. 1994. Personal communication.

---

Research Methodologies

PANEL MEMBERS AND CONTRIBUTING AUTHORS

Barrie Cassileth, Ph.D.--Cochair

Wayne Jonas, M.D.--Cochair

Claire M. Cassidy, Ph.D.--Cochair

Robert Becker, M.D.

Berkley Beddell

Stephen Birch

Carlo Calabrese, N.D., M.P.H.

Harris L. Coulter, Ph.D.

Patricia Culliton, M.A., Dipl.Ac.

Etel E. DeLoach

Allen H. Frey, Ph.D.

James S. Gordon, M.D.

Elliott Greene, M.A.

Sandra Harner, Ph.D.

D. Warren Harrison, M.D.

George Kindness, Ph.D.

Kenneth A. Kivington, Ph.D.

Fredi Kronenberg, Ph.D.

Peter Lechner, M.D., F.A.C.A.

Kyriacos C. Markides, Ph.D.

Michael E. McGuinnis, Ph.D.

Patricia Muehsam, M.D.

Judith A. O'Connell, D.O.

Michael M. Patterson, Ph.D.

Richard Pavek

John C. Reed, M.D.

Kenneth M. Sancier, Ph.D.

Linda Silversmith, Ph.D.

Leanna Standish, N.D., Ph.D.

John Stegmaier

James P. Swyers, M.A.

Vernon M. Sylvest, M.D.

Jon D. Vredevoogd, Ph.D.

Jan Walleczek, M.D.

William S. Yamanashi, Ph.D.

Michael F. Ziff, D.D.S.

Introduction

In 1977 G. L. Engel, professor of psychiatry and medicine at the University of Rochester School of Medicine, wrote:

The biomedical model assumes disease to be fully accounted for by deviations from the norm of measurable biological (somatic) variables. It leaves no room within its framework for the social, psychological, and behavioral dimensions of illness. . . . The biomedical model has thus become a cultural imperative, its limitations easily overlooked. In brief, it has now acquired the status of dogma. In science, a model is revised or abandoned when it fails to account adequately for all the data. A dogma, on the other hand, requires that discrepant data be forced to fit the model or be excluded._

This chapter deals with methods of testing, strategies of validation, proofs of efficacy, and the application of these to alternative medical systems. The evaluation of alternative medical systems is no different from the study of conventional methods in that appropriate methods must be chosen to evaluate the system._ No medical system or method, alternative or otherwise, should be recommended for inclusion in the medical health system until it has been adequately tested. Data produced by incorrect or inadequate research methods do not have validity and cannot contribute to knowledge.

Research Methodologies

The need to expand biomedical assumptions to include psychosocial and behavioral factors has become increasingly understood since Engel's 1977 comment. Indeed, the vast literature of studies that now includes psychosocial dimensions of disease outcomes, health, illness behavior, and correlates of well-being attests to wide acceptance of Engel's challenge. Many of these studies either encompass or cover aspects of some of the alternative therapies discussed in this report.

But other alternative systems and methods have not been adequately studied. One reason for this, according to various alternative medicine practitioners, is that conventional medicine researchers typically and inappropriately demand application of the "gold standard"--that is, prospective randomized clinical trials--when they are not appropriate. This demand occurs despite the availability of a range of suitable research methods from which to choose and the possibility that new methods will have to be identified to fit the situation. Sometimes the demand is for unusually large and complex designs intended to solicit multiple data rather than more appropriate, smaller designs that focus on first-step issues (Pavek, 1994).

Indeed, a review of published conventional research over the years indicates that prospective randomized clinical trials are not always possible or preferred. A 1990 report of the Institute of Medicine's Committee on Technological Innovation in Medicine supports this tacit reality and discusses methodological options:

It has also become clear that randomized controlled clinical trials are not necessarily practical or feasible for answering all clinical questions. Therefore, a variety of other methods, such as nonrandomized trials or observational methods, have been adopted to provide complementary information. Traditionally these methods were regarded as weaker than randomized clinical controlled clinical trials for clinical evaluation. Recent methodological advances, such as the use of non-classical statistics and the ability to link large-scale automated databases for analysis . . . are strengthening these approaches.

Issues in Evaluating Alternative Medical Systems

Research design, even for conventional medicine, is a difficult and challenging process, even more so for alternative systems (Patel, 1987a). As should be abundantly clear from the introduction to the report, there is not one alternative medicine, but several. These consist of new approaches to patient care, new and unusual biomedical disease fighters, discrete methods of treatment, and systems of diagnostics and therapeutics that rely on and are governed by new paradigms.

Fortunately, helpful guidelines exist. The foremost guideline to keep in mind is that the basic goal of any investigation concerning a treatment for human beings is to determine whether the treatment makes a difference. Campbell and Stanley's classic monograph on experimental and quasi-experimental designs discusses lack of "internal validity"_ as the most serious threat to answering that fundamental question, commenting that "internal validity is the basic minimum without which any experiment is uninterpretable: did in fact the experimental treatments make a difference?" (Campbell and Stanley, 1963). The monograph lists several types of threats to internal validity, including

• the possibility that events during the evaluation may unintentionally influence outcomes;
  
  

• the possibility that changes occurring naturally over time may be mistaken for treatment results; and
  
  

• the risk that subject self-selection, rather than the treatment under study, caused the result or lack of result.

Sophisticated experimental designs eliminate or control threats to internal validity. Many threats--though not all--are controlled when studies compare two or more treatments by randomly assigning subjects to each of those treatments. In designs lacking random assignment, additional efforts are needed to bolster as much as possible the validity of the evaluation.

The second major concern for designing evaluations is external validity--that is, the ability to generalize the results of the evaluation to other populations and settings.

Other methodological concerns that have been raised address certain problems that are typically advanced in discussions of alternative medical research. Indeed, research in alternative medicine often appears fraught with conditions that seem uncontrollable or impossible to study. In many instances, however, the technique or a comparable one has already been studied, and published results can provide both encouragement and specific guidance.

As examples, there are useful approaches for studying music therapy provided in papers on how to assess the effects of music therapy in Alzheimer's disease (Aldridge, 1993). Eisenberg and colleagues (1993) reviewed numerous controlled studies assessing the effectiveness of cognitive behavioral techniques in managing hypertension. Methods for controlling or dealing with unwanted influences of subject, practitioner, and environment in learning situations can be found in such general texts as Complementary Methods for Research in Education (Jaeger, 1988), Research in Education (Best and Kahn, 1986), and Introduction to Educational Research (Charles, 1988).

Additional examples of research concerns in alternative medicine, along with proposed solutions, are presented below.

Measuring the Perspectives of Patients

Systems of health, as well as individual health care practitioners, vary in their approaches to health care, patients' decisionmaking, and intended outcomes (end points). Some systems and practitioners focus on quality-of-life issues as being paramount to surgical operations and chemical treatments. This emphasis on quality of life--and on patients making their own decisions--often is considered typical of alternative medicine. There is also a large conventional medicine literature dealing with quality-of-life issues.

Sometimes, difficult decisions must be made that are influenced by the views of practitioner, patient, and health care system. The choice might involve enduring long-term minor discomfort by not electing surgery or choosing a surgical procedure that will eliminate the discomfort but that carries a 2-percent chance of death. Or the choice might concern electing to have or to forego artificial life-sustaining procedures and equipment, such as resuscitation and heart-lung machines in terminal stages of disease.

Breast cancer research is an example of research that led to choices for patients. Under many circumstances, women may elect treatment for their breast cancer: today's choices are lumpectomy plus radiation therapy versus mastectomy. Men diagnosed with advanced prostate cancer also have medically equivalent treatment choices--surgery (orchiectomy) versus subcutaneous injections--with both approaches achieving the same goal of halting the male hormone that promotes prostate tumor growth. In these two examples, patient choice became possible when careful clinical research produced treatment options and then documented that the old and new treatments were equivalent in their effects.

Patients have even more choice in the absence of major or potentially fatal disease. Faced with symptoms of an enlarged prostate, for example, many men elect to endure their symptoms rather than undergo surgery. Recent Agency for Health Care Policy and Research (AHCPR) guidelines made this choice explicit for patients with enlarged prostates and increased national awareness of the need for patient involvement and sensitivity to quality-of-life issues. The guidelines were developed by a panel that was required to include health care consumer representation. AHCPR advised doctors and men with enlarged prostates (not prostate cancer) to curtail the use of two widely used diagnostic tests that frequently led to surgery and to rely instead on a questionnaire that quantifies how severely the condition affects the patient's quality of life.

Because some patients are comfortable with symptoms that other patients consider unacceptable, and because an enlarged prostate is not life threatening, the recommended focus on the patient's view of symptoms--rather than clinical measurement of prostate enlargement--is expected to lead to more appropriate treatment and decreased costs for surgery. (In 1992, before the guidelines, some 220,000 medicare patients received corrective prostate surgery performed at a cost of more than $1 billion [see sidebar].) Alternative practitioners note that increased use of patient choice and quality-of-life decisionmaking--as exemplified by these guidelines--both encourage financial savings and address psychosocial concerns that should not be neglected.

Disbelief

Disbelief is a factor not frequently addressed. Here, disbelief refers to the opinion of a physician, investigator, or research organization that a particular procedure or approach is ineffective. If this opinion is held by someone in a position to influence research funding or conduct, its impact can be widespread. In the view of alternative medicine researchers, the two typical ways in which they are affected include (1) outright, knee-jerk rejection of study proposals and (2) insistence on inappropriate and/or unnecessarily cumbersome study design.

A recent study provides corroboration concerning the alternative medicine researchers' perception of bias (Wilson et al., 1994). Wilson and his colleagues found that both medical and psychological researchers were more willing to overlook or disregard methodological flaws in studies that addressed "important" topics rather than "less serious" ones and to be more demanding in their standards for the latter._ In an analogous way, scientists with little knowledge of or interest in alternative medical subjects could be expected not to take them seriously or to demand additional proofs.

Indeed, a frequent complaint of alternative medicine practitioners is that they are often obliged to conduct later stage studies even before preliminary information is gathered (Pavek, 1994). They also report the strong tendency of research institutions and methodologists to insist that very stringent controls be included in beginning studies--controls that would never be considered for early investigation of more conventional subjects. In other words, alternative medicine researchers protest that the standards of proof are raised for research on alternative practices.

In The Cancer Industry, Ralph Moss provides another good example of such misdirection and misperception. Moss examined the American Cancer Society's (ACS's) list of unproven methods for the 1970s and 1980s, which mentioned 70 practitioners and 63 methods. Although ACS had described its "unproven" list as containing mostly unqualified practitioners and only a few researchers with appropriate degrees, Moss found that more than 70 percent held an M.D., Ph.D., or D.O. Further, more than 50 percent of the methods had never been investigated to prove whether or not they worked. Only 29 percent had received some investigation leading ACS to term them ineffective or "unproven" (Moss, 1989).

Perceiving these biases discourages alternative medicine researchers from attempting even preliminary studies. The discouraged investigator sees little point in proceeding if small, preliminary, information-gathering studies are so readily rejected. Likewise, large-scale, controlled trials demanded by those with such biases will likely not be funded.

Specific Testing Difficulties

The examples below serve to illustrate some of the controversy surrounding methodological decisionmaking for alternative medical systems.

Systematic therapeutic learning. Some alternative methods--such as biofeedback, meditation, imagery, and dance therapy--involve a learning process. With repetition, the person using them becomes more adept. Evaluating the benefit of such methods requires ensuring that a basic minimum of training is achieved by study subjects (i.e., the study must control for the amount of learning) and carefully selecting the appropriate research technique.

In studies of therapies involving learning, the research methodology usually includes appropriate control groups. Typically, the controls receive another intervention or none at all and would be students, patients, or clients of similar ages, talents, problems, interests, and whatever else is relevant to the process or technique under study. Many practitioners of alternative medicine consider the situation unethical if the control group receives no intervention. In addition, they note that subjects in the control group might be angry and frustrated (Goeble et al., 1993).

Furthermore, some alternative medicine practitioners are concerned about any research design involving a control group comparison, because they believe this test structure does not adequately evaluate certain alternative systems (Shellenberger and Green, 1986, 1987).

In contrast, methodologists are likely to insist that because research involving learning situations is influenced by the thoughts, feelings, intentions, and attitudes of both experimenter and subject, by practice effects, and often by the learning environment, this research requires control of these unwanted influences on results. For example, since the teacher cannot be blinded to the method or the results of the treatment in learning situations, methodologists propose as a solution that the contemplated research be conducted by someone other than the teacher or practitioner and that objective data (for example, physiological and laboratory measurements or subject self-report tests) be obtained._

Before determining that new research methodologies might be needed, alternative medical investigators can look for guidance at the methods that have been used to research issues relevant to their own work and evaluate the resulting study. For example, those interested in biofeedback and similar self-training techniques can turn to studies published in periodicals such as the Journal of Biofeedback and Self Regulation and the Journal of Behavior Therapy and Experimental Psychiatry and can consult with members of the Association for Applied Psychophysiology and Biofeedback.

Similarly, problem-oriented journals provide illustrations for how to assess biofeedback and related techniques when applied to particular problems. Studies of the effectiveness of biofeedback, relaxation, or meditation in controlling epileptic seizures, for example, appear in journals such as Neurological Clinics and Perceptual and Motor Skills. Randomized clinical trials and other methods used to evaluate imagery training and migraine headaches are published in such journals as Headache.

Manual healing methods. Much like surgery on babies born with cleft palates, each manual healing procedure, no matter how well structured, is highly individualized. Even for one individual, the procedures in a treatment series may vary from session to session. Research methods must take this point into account. One possible approach is to conduct comparative studies of the effectiveness of one manual method versus another. Another approach is to compare outcomes with the results of standard medical therapy or no treatment.

Research and documentation are needed to objectively present the baseline status of the patients and measure actual changes in physiological function, work capacity, or functional activities induced by the manual healing method. Monitoring techniques like those developed for biofeedback studies (Goeble et al., 1993; Shellenberger and Green, 1986, 1987) and for orthopedic rehabilitation (gait analysis) (Harris and Wersch, 1994; Perry, 1992; Sutherland et al., 1988) may be adaptable to studies of manual healing methods.

An example of an approach applying monitoring methods and measuring an outcome is the following research strategy: Narula (1993) wanted to determine whether subjects with rheumatoid arthritis showed improvements after training in a manual healing method called the Feldenkrais method (see the "Manual Healing Methods" chapter). She applied video filming and "peak performance" software to examine whether subjects showed changes in their sit-to-stand movement pattern. Narula also measured grip strength with a dynamometer, walking speed by timing the number of seconds required to walk 50 feet, and pain and disease status with a quantifying questionnaire. (The treatment produced some improvements by all measures except grip strength.)

Another way that methodologists have proposed to deal with the problem of practitioner bias (and produce more accurate data on the merits of the procedure per se) would be to have subjects treated with their eyes closed or covered--in effect "blinded" to the individual therapist. Frequently, however, the subject is still able to recognize the therapist by other means. In addition, the subject's discomfort at being "blindfolded" is likely to interfere with the treatment process.

Alternatively, and depending on the research question under study, a more useful approach would be to have more than one practitioner treat an individual, or the same practitioner could provide both the experimental and control treatments. However, when more than one practitioner provides treatment to the same subject, the subject is likely to have preferences and may therefore be uncomfortable with this process. In addition, difficulties may arise in transferring information from one practitioner to the next when much of that information involves physical sensing.

The use of independent observers who are not administering the therapy to gather and analyze results is another approach that can reduce practitioner bias. For example, in osteopathic research, this approach sometimes takes the form of having several practitioners independently assess the condition of patients before and after treatment, using a negotiated system of evaluation, then collating their results to determine whether significant changes have occurred (Beal et al., 1982). (This subject--inter-rater assessment--is discussed in another context in the osteopathic medicine section of the "Manual Healing Methods" chapter.)

Thus, although prospective, randomized, controlled clinical trials are not always feasible for studying manual healing methods, other methodologically sound studies can be constructed to evaluate these techniques.

New or unusual biochemical substances. Tech-niques for both laboratory and human testing of novel drugs or substances have been applied in conventional research for decades and also are applicable in alternative medical research. The research question dictates which methodology can and should be applied. For instance, to learn whether a new treatment increases length of remission (period without active disease) in lung cancer, patients receiving the new or alternative therapy and those on standard conventional treatment would be followed over time. The percentage of patients surviving after a given time, as well as the number of months to relapse or death, would then be compared for the two groups.

As another illustration, here are two possible ways to manage the apparent difficulty of studying iscador, a mistletoe extract used in alternative medicine as an anticancer treatment (see the "Pharmacological and Biological Treatments" chapter). Some alternative medicine practitioners have speculated that iscador cannot be studied with conventional methodologies. The reasoning is that since employing iscador produces a definite rash, blinding the investigator and the patient to the treatment used is impossible. One research approach avoiding this problem is to ask a question that does not require blinding: for example, "How much iscador is needed to produce a certain desired effect?" Another approach is to compare iscador treatment with a control procedure that has no effect against cancer but includes a (harmless) substance that also produces a rash. Patients would be randomly assigned to receive iscador or the control treatment.

Approaches that mix physics and biology. Standard methodological approaches can be used in researching interventions that invoke mechanisms that depend on physical properties. The essential step in such research--for example, in studying bioelectromagnetic applications--is to characterize the physical variable in greater detail than is common in clinical research. For example, artifacts caused by ambient electromagnetic fields in the laboratory environment (such as from power lines and laboratory equipment) must be avoided in bioelectromagnetics research. Furthermore, the bioelectromagnetic fields under study, which can involve very small quantities of energy, must be measured accurately in order to detect whether observed effects correlate with level of treatment. (A sample protocol for magnetic-field therapy is presented in the "Bioelectromagnetics Applications in Medicine" chapter.)

Systems with unconventional paradigms. Conventional research concepts and methods may not be capable of determining the mechanisms of action for systems that do not operate under conventional paradigms (Anthony, 1987; Bensoussan, 1991; Diamond and Denton, 1993; Patel, 1987b). Relevant examples include traditional Chinese medicine, with its concepts of an invisible qi moving through the body to organize it into balance and harmony; homeopathy, with extreme dilution rates that leave less than one molecule of a substance (too little for conventional science to study); and biofield therapeutics, which operate through application of an energy field that has not yet been fully characterized. Indeed, these situations are likely to lead to disbelief and claims of placebo effects.

For systems with unconventional paradigms, the question of clinical effectiveness--as opposed to underlying paradigm, belief system, or mechanism of action--can be readily studied. Indeed, many medical treatments have been implemented on the basis of evidence of safety and efficacy long before their mechanisms of action were detailed.

When unconventional paradigms are involved, it is particularly important that researchers trained in the biomedical model develop considerable knowledge of the alternative system under study and work jointly with an expert in that alternative field to design effective and valid research protocols.

Issues That Affect Both Conventional and Alternative Medical Systems

Effects on therapeutic outcome of patients' choices of treatment; participation by patients in their own care; and the relationship between the expectations of patients, cultural context, and lifestyle activities are examples of issues that are usually minor considerations in conventional medicine but are especially important in the alternative spectrum. Some alternative medical systems rely heavily on one or more of these issues.

Quality of Life

How one studies factors associated with quality of life depends on the research question being asked: this point makes generalization difficult._ The following are examples of research approaches when the attitudes of patients are under study:

• Patients' expectations. To answer the question of whether there is a correlation between patients' expectations for therapy, or hopelessness, and their clinical outcome, one research approach was to ask patients (who were part of a broader study) with newly diagnosed, prognostically poor cancers to complete a questionnaire concerning what they expected their treatment to accomplish. Patients checked one out of a choice of such response options as "Cure my disease"; "Hold my disease in check"; "Not much"; and "Don't know." Patients also completed a standardized "hopelessness" scale. (Patients were previously informed in detail through discussions with physicians, written consent forms, and other printed material about their diagnosis and prognosis.) The patients were followed over time. Data were analyzed to determine whether there was an association between length of survival and treatment expectations or hopelessness; none was found (Cassileth et al., 1985, 1988).
  
  

• Patient choice. Research on patient choice has ranged from questionnaire surveys of the preferences of patients for information about their illness and treatment and desire to participate in their own care (Cassileth et al., 1980) to intervention studies that address the later satisfaction of patients with the treatment option that they themselves had selected (Cassileth et al., 1989).

Additional suggested approaches can be gleaned from medical literature and other resources. Research on quality of life is common in conventional medicine; journals, annual meetings, international workshops, textbooks, and a project of the World Health Organization are devoted to quality-of-life research and measurement techniques.

Avoiding Patient Bias

The results of some studies may consist of patient or client reports of how they feel before versus after the treatment. Because people are likely to try to please or at least not insult their care givers, this is another situation in which to use independent observers (Kassirer, 1994). For example, the results could be collected by an investigator not involved in providing the therapy.

General Procedural Issues

A sequence of steps defines the development and application of research, starting with the initial idea or hypothesis under consideration for testing (see app. F). Whether or not a research project should be carried out depends first on what is already known about the subject and then on the research questions posed and the proposed experimental design to answer those questions.

Before implementing new clinical investigations, the researcher conducts critical reviews of the literature. With assistance from certain methodology specialists, these reviews can include sophisticated systematic reviews and meta-analyses (see app. F) of existing studies of alternative medical practices. These reviews are useful to learn what others have done, obtain information about methods employed, and determine the shortcomings or missing information that the proposed study can redress. Such reviews are likely to summarize existing clinical information;

• identify methodologic inadequacies found in existing controlled experiments; and
  
  

• document evidence, or lack of evidence, of clinical effectiveness.

Whether researchers are using the newer analytical techniques or older ones, caution is always needed in determining whether the selected tools have been applied appropriately. When the tools are applied appropriately, the results can strengthen the believability of the alternative intervention under study.

After investigators review existing studies, they must establish appropriate methodologies for assessing their proposed research before the research commences. As indicated earlier in this chapter, there is a need to select appropriate methodologies for each alternative medical procedure or system being researched and to develop new ones if present methods are inadequate. Correspondingly, it is clear that alternative medical practitioners and researchers must interface directly with methodologists and experienced researchers to work out how to test the effectiveness of their systems.

Specific Recommendations

Implementation of the following recommendations would provide valuable methodological assistance for alternative medical research:

• OAM should sponsor, perhaps with the assistance of the National Institutes of Health Office of Medical Applications of Research, two or more methodology assessment conferences to begin the process of identifying or developing appropriate methodologies. These conferences should examine differing research assumptions and epistemological issues and should review available appropriate methodologies. Published proceedings from these conferences will be valuable to alternative and conventional investigators.
  
  

• Through the Field Investigations program, OAM should help practitioners to collect data in a scientifically valid manner, conduct retrospective reviews, assemble best case series, and conduct prospective pilot trials on existing therapies.
  
  

• OAM should implement systematic reviews, including meta-analyses, of the alternative medical therapies as necessary to assist in implementation of next-step clinical investigations in the various fields.
  
  

• OAM should develop and make available a list of research methodologists willing to collaborate or serve as resources for alternative medical investigators and for conventional investigators intending to study alternative practices.
  
  

• OAM should make resource materials to guide alternative researchers readily available, including guidelines on research methods and methodology bibliographies.

---

References

Aldridge, D. 1993. Music and Alzheimer's disease--assessment and therapy: a discussion paper. J. R. Soc. Med. 86(2):93-95.

Anthony, H.M. 1987. Some methodological problems in the assessment of complementary therapy. Stat. Med. 6:761-771.

Beal, M.C., J.P. Goodridge, W.L. Johnston, et al. 1982. Interexaminer agreement on long-term patient improvement: an exercise in research design. J. Am. Osteopath. Assoc. 81(5):322-328.

Bensoussan, A. 1991. Contemporary acupuncture research: the difficulties of research across scientific paradigms. Am. J. Acupunc. 19(4):357-365.

Best, J.W., and J.V. Kahn. 1986. Research in Education. Prentice Hall, New York.

Campbell, D.T., and J.C. Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Rand McNally, Chicago.

Cassileth, B.R., E.J. Lusk, D.S. Miller, et al. 1985. Psychosocial correlates of survival in malignant disease. N. Engl. J. Med. 312:1551-1555.

Cassileth, B.R., M.S. Soloway, N.J. Vogelzang, et al. 1989. Patients' choice of treatment in stage D prostate cancer. Urology 33 (Suppl.):57-62.

Cassileth, B.R., W.P. Walsh, and E.J. Lusk. 1988. Psychosocial correlates of cancer survival: a subsequent report 3-8 years after cancer diagnosis. J. Clin. Oncol. 6:1753-1759.

Cassileth, B.R., R.V. Zupkis, K. Sutton-Smith, et al. 1980. Information and participation preferences among cancer patients. Ann. Intern. Med. 92:832-836.

Charles, C.M. 1988. Introduction to Educational Research. Longman, Inc., White Plains, N.Y.

Diamond, G.A., and T.A. Denton. 1993. Alternative perspectives on the biased foundations of medical technology assessment. Ann. Intern. Med. 118:455-464.

Eisenberg, D.M., T.L. Delbanco, C.S. Berkley, et al. 1993. Cognitive behavioral techniques for hypertension: are they effective? Ann. Intern. Med. 118(12):944-972.

Goeble, M., G.W. Viol, and C. Orebaugh. 1993. An incremental model to isolate specific effects of behavioral treatments in essential hypertension. Biofeedback Self Regul. 18(4).

Harris, G.F., and J.J. Wersch. 1994. Procedures for gait analysis. Arch. Phys. Med. Rehabil. 75:216-225.

Jaeger, R., ed. 1988. Complementary Methods for Research in Education. American Educational Research Association, Washington, D.C.

Kassirer, J.P. 1994. Incorporating patient's preferences into medical decisions [editorial]. N. Engl. J. Med. 330(26):1895-1896.

Moss, R. 1989. The Cancer Industry. Paragon House, New York.

Narula, M. 1993. Effect of the six week Awareness through Movement Lessons--the Feldenkrais method on selected functional movement parameters in individuals with rheumatoid arthritis (M.S. thesis, Oakland University, Rochester, Mich.) (abstract).

Patel, M.S. 1987a. Evaluation of holistic medicine. Soc. Sci. Med. 24(2):169-175.

Patel, M.S. 1987b. Problems in the evaluation of alternative medicine. Soc. Sci. Med. 25(6):669-678.

Pavek, R. 1994. Personal communication.

Perry, J. 1992. Gait Analysis: Normal and Pathological Function. SLACK Inc., Thorofare, N.J.

Shellenberger, R., and J.A. Green. 1986. From the Ghost in the Box to Successful Biofeedback Training. Health Psychology Publications, Greeley, Colo.

Shellenberger, R., and J.A. Green. 1987. Specific effects and biofeedback vs. biofeedback-assisted self-regulation training. Biofeedback Self Regul. 12(3):185-209.

Sutherland, D.H., R.A. Olshen, E.W. Biden, et al. 1988. The Development of Mature Walking. Mackeith Press, London.

Wilson, T., et al. Psychological Science 4:322-325, cited in April 1994 in "Study of `studies' reveals surprising science bias" in Brain-Mind, A Bulletin of Breakthroughs, Interface Press, Los Angeles, Calif., April 1994, 19(7).

Sidebar

US Issues Guidelines That May Lead to Less Surgery for Enlarged Prostate

By Ron Winslow

U.S. officials issued new guidelines for doctors and patients that could transform treatment for an enlarged prostate, a condition that afflicts about half of men over age 50.

If the guidelines are widely followed, they would likely lead to less surgery and fewer diagnostic tests than are now performed in managing the ailment. They call for patients to take a primary role in deciding whether surgery, medication, or just monitoring symptoms without treatment is the best course to follow.

Known as benign prostatic hyperplasia, or BPH, an enlarged prostate restricts or obstructs the flow of urine from the bladder through the penis, causing frequent and urgent urination and related symptoms. It isn't related to or a cause of prostate cancer. The most common treatment is surgery, which was performed on more than 220,000 Medicare patients in 1992 at a cost of more than $1 billion. Both the cost and the fact that doctors vary widely in prescribing treatments prompted U.S. health officials to consider BPH as a major candidate for an initiative to develop guidelines for a variety of common diseases. At the same time, new treatment alternatives, including drugs marketed by Merck & Co. and Abbott Laboratories have recently been approved by U.S. regulators, adding to options for both doctors and patients.

"There's been a tendency to intervene with surgery too quickly, rather than consider options," said Jarrett Clinton, administrator of the Agency for Health Care Policy and Research, a division of the Department of Health and Human Services, which announced the guidelines.

Surgery `Most Effective'

"Our analysis clearly demonstrates that surgery is the most effective treatment for BPH," added Dr. John D. McConnell, chairman of urology at University of Texas Southwestern Medical Center, Dallas, and chairman of a 13-member expert panel that developed the guidelines. "But not all patients need or desire the most effective therapy."

The guidelines urge doctors to curtail use of two widely used diagnostic tests and rely instead on a new, seven-item questionnaire yielding a symptom score on a 35-point scale that indicates how severely BPH is affecting a patient's quality of life. Since some men live comfortably with symptoms that others consider severe, the guidelines say a focus on a patient's view of symptoms rather than clinical tests will lead to more appropriate treatment.

"The symptom score is a very pivotal part of the work-up and is one of the things that brings the patient into the process," said Dr. McConnell. "This is going to be a new concept for many physicians."

The new guidelines say patients with severe symptoms, including an inability to urinate, should be treated with surgery. But all others should fill out the questionnaire and use the results as a basis for discussing treatment strategies with their physicians.

"The best treatment is the one chosen by an informed patient," Dr. McConnell said. "A pill is less effective than surgery, but has less risk. We're trying to get doctors away from making that value judgment themselves. It's a patient's decision."

The guidelines also say that X-ray or ultrasound examination of the kidneys and cystoscopy, in which doctors look at the urinary tract through a scope, are of little use in deciding whether a patient needs treatment. A 1989 survey found that two-thirds of urologists routinely use those tests in examining a patient with BPH symptoms.

--Wall Street Journal, February 9, 1994, page B6

---

Peer Review

PANEL MEMBERS AND CONTRIBUTING AUTHORS

Carlton F. Hazlewood, Ph.D.--Cochair

Edward H. Chapman, M.D.--Cochair

R. James Barnard, Ph.D.

Myrin Borysenko, Ph.D.

Rosalin L. Bruyere, D.D.

James A. Caplan

Effie Poy Yew Chow, Ph.D., C.A.

Winston Franklin

Carol Hegedus, M.S., M.A.

Darrcy A. Loveland, J.D.

Gladys T. McGarey, M.D.

Richard Pavek

William H. Philpott, M.D.

David M. Sale, J.D., LL.M.

Savely L. Savva

Marilyn Schlitz, Ph.D.

Gertrude Schmeidler, Ph.D

Jeremy Waletzky, M.D.

Introduction

Peer review is the process of allowing the researcher's peers to evaluate the research credibility and potential value of researchers' work. Peers are chosen to participate in the evaluation process on the basis of their knowledge of or expertise in the area of scientific investigation being considered. The peer review process is widely used in such areas as education, publication, state licensing, and review of clinical outcomes on a case-by-case basis by physician review organizations mandated by Federal law. Though these areas all have tremendous relevance to alternative medicine, this report will chiefly focus on the National Institutes of Health (NIH) evaluation process for applications for research grants and research contracts.

Structure of the Peer Review System

Intake of Applications

At NIH, the peer review process is administered by the Division of Research Grants (DRG), to which the grant application is submitted._ An initial administrative review by the division leads to the assignment of the application to one of the various institutes within NIH--such as the National Cancer Institute or the National Heart, Lung, and Blood Institute--and also to 1 of some 100 standing initial review groups (IRGs), or study sections, within DRG. Each study section has a scientific review administrator plus 20 or so scientists, each of whom is a specialist in a given area.

If the application does not fit the scope of any of the established study sections, the scientific review administrator in the DRG, in consultation with the science administrator in the institute to which the application has been assigned, may convene a "special" (one-time) study section or an "ad hoc" (temporary) study section to undertake the review.

Review of Applications

In the review process, each grant is reviewed in detail by a primary and a secondary reviewer within the study section, who then present their findings to the entire study section. (The findings of these two reviewers are likely to shape the study section's decision.) After their review, study section members vote whether or not to recommend for further consideration and assign to the application a funding "priority score" ranging from 100 to 500. A "perfect " score is 100. The applicant receives a written response called a summary statement ("pink sheet") that briefly discusses the decision of the study section and the numerical score assigned to the application.

The actual funding of grants is made after a secondary review of the application by an advisory council or board of those NIH centers and institutes that have funding authority. With current levels of funding, it is likely that only grants with scores lower than 150 will receive funding.

Appeals

An applicant may investigate an unfavorable recommendation by contacting the science administrator within the institute to which the application was assigned and the scientific review administrator of the regular study section. The applicant may request additional information and even appeal. If sufficient reasons are identified by the applicant, he or she may request that the appropriate branch chief within the institute review the decision. The appeal may then be carried up the chain of command to the director of the institute.

This appeals process is, however, not frequently employed. A major concern for applicants who choose to appeal is the potentially negative effect that requesting an appeal may have on their professional reputations. Furthermore, as it currently stands, the appeal process is perceived by a number of scientists that have actively used it to be an exercise in futility (Hazelwood, 1992, 1993; Ling, 1992).

Potential Concerns

The NIH peer review process in alternative medicine is not expected to raise methodological issues substantially different from those encountered with other emerging fields of scientific endeavor with which this division may have been initially unfamiliar. Rather, the major challenges are intellectual ones for the reviewers involved in the peer review process.

Alternative medicine challenges established scientists and institutions by proposing models and patterns that differ from what is medically familiar. Peer review in alternative medicine will require reviewers to confront issues of potential bias stemming from differences in basic assumptions about health and disease that may reflect the limitations of current scientific knowledge (Ling, 1992).

Historically, new ideas in science have rarely been readily accepted by proponents' peers. In fact, established scientists are major contributors to the resistance to new ideas. The peer review system, which matured during the latter half of this century, may be contributing to the resistance to acceptance of legitimate scientific discoveries that are out of the mainstream (Horrobin, 1990). This is particularly so in the case of alternative medical practices.

In general, there are two approaches to research: innovations, which directly challenge existing models and patterns; and improvements, which refine and advance the results stemming from innovative research. For example, Thomas Edison's research in the 19th century led to an innovation, the light bulb, while later research resulted in various improvements of the light bulb, such as the "Miser." Alternative medicine usually involves innovative research, particularly because research developments in this field are so new. Some peer review bodies may inadvertently fail to recognize innovative research in study proposals they review (Horrobin, 1990).

Inherent resistance to innovative research must be recognized and acknowledged before much can be done to install a peer review system that supports innovation and shortens the time between scientific discovery, creation of new effective treatments, and availability to the general public (Ling, 1992).

Because many alternative medical applications do not fit within the scope of existing study sections, the ad hoc review mechanism must be used until enough experience has been obtained with these applications to allow the formation of new standing review groups. Moreover, since standing study sections primarily serve NIH institutes and centers, the status of OAM as an office, rather than a center or institute, complicates the review process.

The following points summarize specific concerns that are likely to arise in an examination of the peer review process:

Grant proposals are usually evaluated in terms of established priorities. Peers are chosen to review established categories of specialization, and proposals must generally fit into one of those categories. This process tends to promote research in long-established directions and discourages efforts toward new synthesis and new concepts.

Peer reviewers are not neutral parties; rather, they are specialists in specified areas, and they also may have vested interests. Reviewers may make their living teaching or practicing the very ideas that are called into question by the new research thinking or the new practices being proposed.

It appears that NIH does not consider it a conflict of interest for a scientist to serve on an appeal board when he or she holds a perspective that conflicts with the premises of an appellant. Proponents of alternative medicine are concerned that this practice presents the greatest difficulty of the NIH review process. The issue is especially important for approaches such as homeopathy and biofield therapeutics, whose basic assumptions challenge conventional scientific assumptions.

Shortages of research funds intensify the problem of bias against innovative research and alternative medical approaches.

The appeals system as it now functions is not viewed as effective. There is a belief that too few win appeals without repercussions on their careers; this, in turn, discourages others from participating.

1993 Grant Review Process

Grant applicants have made various complaints about the grant review process resulting from OAM's first RFA (see sidebar). Some complaints concerned misassignments, category and procedural errors, and possible violations by the peer review groups of the rules of the RFA process.

Recommendations

The optimal peer review process for proposed alternative medical research projects would rely on one or more standing study sections that meet regularly and are made up of experts from a broad spectrum of alternative medicine practices, and basic and clinical scientists who have balanced perspectives and can offer an objective opinion on methodological issues related to applications from practitioners of alternative medicine. Consequently, DRG should be asked to set up a model for the peer review process that allows a balanced evaluation of science that is outside of traditional or mainstream thinking.

Specific steps to improve the review process include the following:

OAM should recruit experts within the alternative medical community who are willing to review and give advice on applications prior to their being formally submitted.

Ad hoc study section members should be selected by the criteria already established by NIH for regular review groups, with these additional provisions:

For alternative medicine practitioners the "equivalent" to a doctoral degree can include board certification in their field.

Where there is no board, special approval on the basis of expertise and experience may be needed.

The presence of healthy skeptics is essential to adequate review, but it is necessary to screen out obviously antagonistic reviewers.

During the initial phases of OAM's operation, its staff should participate in decisions concerning the referral process. To achieve this, OAM and DRG staffs will need to cooperate in deciding either to send each proposal to the OAM initial peer review panel or, if feasible, to route it through the regular NIH grant process. When a program analyst sends the alternative medical grant proposal to DRG, the analyst could assist the process within the division--for instance, by making recommendations for appropriate reviewers to participate in the ad hoc study section.

For the first few years, an initial OAM review panel (i.e., a special or ad hoc study section) should function to review grant applications that do not fit the criteria for existing DRG study sections.

On the basis of the experience of the initial review section, NIH should then establish an official ad hoc panel within either DRG or OAM. It is anticipated that the ad hoc panel would function for up to 5 years.

Ultimately, an initial review group (or study section) should be established for OAM within DRG. If the OAM evolved to center status, initial review groups could be established within the Center for Alternative Medicine to review training grants, center grants, RFAs, and so on.

The reviewers of any application should include at least one expert in the discipline specified in the application.

The entire review section should be at least 50 percent composed of clinicians or researchers from the various types of alternative medical practice groups, such as bioelectromagnetics, acupuncture, homeopathy, and lifestyle interventions.

Because it is possible that some errors did occur in the 1993 RFA review process, it is recommended that an ad hoc review committee be drawn from members of OAM's new mandated advisory council and from methodologists knowledgeable in alternative practices to review a randomized sample of the studies that were proposed. If this effort reveals that errors did occur, the committee should then develop guidelines to help future peer review committees dealing with alternative medical subjects.

Since the problem of appropriate peer selection is ongoing, a select committee should be established to continually review the makeup of the peer review panels. (This committee is a logical outgrowth of the ad hoc review committee proposed in the preceding paragraph.)

OAM should encourage DRG to develop a workable and effective NIH appeals process--one that allows due process.

References

Hazelwood, C.F. 1992. Personal communication.

Hazelwood, C.F. 1993. Personal communication.

Horrobin, D.F. 1990. The philosophical basis of peer review and the suppression of innovation. JAMA 263(10):1438-1441.

Ling, G.N. 1992. A Revolution in the Physiology of the Living Cell. Krieger Publishing Co., Melbourne, Fla.

Sidebar

The 1993 Research Grants Application and Review Experience*

The purposes of the request for applications (RFA) sponsored by the Office of Alternative Medicine (OAM) and released in spring 1993 were to foster alternative medicine-researcher collaborations and provide opportunities to obtain preliminary or pilot data that might justify additional support for research. NIH received some 800 letters expressing interest in the RFA and 463 applications for the 30 grants for which there was funding. Only 10 or fewer applications did not meet the RFA's requirements. DRG deemed the response overwhelming.

Several senior staff members of DRG who participated in the application review process attended an orientation seminar presented by OAM. Eight review panels were set up for the six categories of subjects to which applications were assigned: Diet/Nutrition/Lifestyles; Mind/Body Control; Traditional and Ethnomedicine; Structural and Energetic Therapies; Bioelectromagnetic Applications; and Pharmacological and Biological Therapies. The division worked with OAM to identify alternative medical practitioners to serve, along with more conventional scientists, on the panels. The division also kept statistics on the nature of the applicants, noting, for example, that nearly two-thirds held a Ph.D. or an M.D. and that very few applicants were members of underrepresented minority groups, even though some members of these groups do use alternative medical practices.

The eight panels conducted the initial review of the grant applications. OAM originally intended that the applications then be reviewed by members of the Ad Hoc Advisory Panel on Alternative Medicine; however, this step turned out to be disallowed by Government regulations. Instead, the second review was carried out by OAM staff members. OAM then submitted a list of recommended grantees to the National Advisory Research Resources Council; this is the advisory council of NIH's National Center for Research Resources, which has grantmaking authority and which is accustomed to dealing with a broad range of subjects. The advisory council accepted OAM's recommendations, and the 30 grant awards were announced in fall 1993.

__________

*Based on presentations by D. Eskinazi (OAM) and F. Calhoun (DRG) to the July 14-15, 1993, meeting of the Ad Hoc Advisory Panel to the Office of Alternative Medicine, and on followup interviews.

---

Public Information Activities

by Richard Pavek

According to Section 404E (d) (1) of the June 1993 amendments to the Public Health Service Act, the Office of Alternative Medicine (OAM) is to "establish an information clearinghouse to exchange information with the public about alternative medicine." This chapter of the report deals with implementation of that directive and other public information activities._

As vital as research databases are to the researcher, information libraries are of equal importance to the clinician, physician, and patient; without information there is no way a physician or patient can make truly informed choices. (Obviously, databases for consumers and practitioners differ from research databases in scope, language, complexity, and intent even though they may share many of the same materials.) An accessible database of alternative medicine information is a vital need for the American public. Except for current efforts at OAM, the National Institutes of Health (NIH) does not currently maintain a special information service for consumers wanting information on alternative medicine._

Nevertheless, several institutes at NIH, as well as some other Federal agencies, include alternative medical practices in some of their information. The National Institute of Neurological Disorders and Stroke includes acupuncture and psychological techniques; the National AIDS Information Clearinghouse, information on nutrition strategies; the National Institute of Mental Health, information on biofeedback; and the National Cancer Institute, information on chaparral tea and other medicinal herbs. Outside NIH auspices, the Science and Technology Division, Reference Section, of the Library of Congress has reference guides to acupuncture and medicinal plants, among others.

A national clearinghouse would provide a clear, concise message for the broader health care community, as well as interested members of the lay public, about the benefits of alternative medicine based on a body of scientific information that is current, accurate, and complete. Thus, a National Library of Medicine (NLM) alternative medicine clearinghouse would provide a gateway for knowledge transfer to several audiences: health care practitioners, policymakers, educators, and the public at large.

Current Information Sources on Alternative Medicine

Doctor's Office

Unfortunately, most health information currently available to consumers comes from the conventional medical practitioner's office in the form of leaflets provided as a "public service" by the American Medical Association or by drug companies. The rare citation of any alternative medical practices usually mentions biofeedback or massage. A few medical practitioners with alternative interests do provide a range of relevant material.

Health Newsletters

Some major medical schools publish "health newsletters," which are sold by subscription to the public. Although these newsletters often carry prestigious faculty names under the masthead, the newsletter is usually edited and published by independent publishing houses. If alternative medical practices are mentioned, it is usually in the context of urging caution in using them because they are unresearched. Some newsletters are stridently against alternative medicine.

Directories and Guides

A number of books available on the retail market index and describe alternative medical practices. These books vary in comprehensiveness and usability._

Magazines and Newspapers

Consumer Reports recently published a three-part 1994 series, "Alternative Medicine: The Facts" (volume 59, January, March, and June). Other magazines of general interest frequently publish similar articles, and certain specialty magazines are even more likely to do so, including, for example, New Age Journal, East West Journal, and Yoga Journal. The New York Times and other newspapers sometimes publish articles on alternative medicine, especially on diet, supplements, and mind-body approaches.

Private Sector Databases

Several private sector databases in use in the United States cover alternative medical practices. The databases are organized in different formats. Some of the best known are listed here:

Wellnet is an electronic allied health professional database.

IBIS is a hypertext database that includes information on conditions and treatment approaches used by alternative practitioners in such fields as homeopathy, naturopathy, nutrition, oriental botanicals, osteopathy, physical medicine, psychosocial therapies, Western botanicals, and biofield therapies.

Herbalgram is a consumer-and physician-oriented reference library on herbal botanicals.

The Brain-Mind Bulletin maintains a reference library that includes listings on a wide range of subjects, most of which are alternative medicine issues.

Alternative Medical Connection is a new journal of alternative medicine with an online database available to consumers and clinicians.

Inappropriate Regulations

Regulations sometimes keep important information about the potential benefits of alternative agents from consumers. For example, the present policy of the Food and Drug Administration (FDA) is to allow the sale of herbal medicines provided that no information is included on the conditions for which they are to be used. Although this policy addresses the technicality of current laws, it leaves the consumer vulnerable to misapplication of herbal medications.

Development of an Information Clearinghouse

OAM will have to collaborate with appropriate NIH agencies as well as others, such as the Agency for Health Care Policy and Research and the National Center for Health Statistics, to collect information on the extent and pattern of utilization and on cost-effectiveness of alternative medical practices. This cooperation may be implemented by adding alternative medical questions to existing statistical instruments, or it may involve a separate effort.

Information will have to be gathered from a wide range of sources. To assess the extent of consumer use of, and satisfaction with, over-the-counter interventions, practitioner-independent approaches may be helpful. For example, patients fill out questionnaires to assess treatments offered through "buyers clubs," such as those operated for persons with HIV and AIDS and chronic fatigue syndrome (also known as chronic fatigue immune dysfunction syndrome). Practitioner-independent approaches may be in widespread use in the community but have never received investigation in controlled clinical trials.

Accessing Information

Information will have to be made available to consumers through various means. Electronic access through America OnLine, CompuServe, Genie, and Prodigy will likely be very useful.

However, since not all consumers have access to home computers, other means will be required to provide information to the general public--for example, through print information, CD-ROM disks at public libraries, and other community outlets.

Media Activities

A noteworthy shift in media attitudes toward alternative medicine since the early 1990s is attributable to the establishment of OAM. NIH should be supported in its endeavors in this area, and OAM should be encouraged to continue its efforts to increase awareness of alternative medicine among the scientific and lay media.

Among other duties, the OAM public information officer should

initiate and maintain regular mailings of worth to selected medical journals of note;

notify interested associations and groups of impending OAM activities;

promptly circulate minutes of the meetings of the OAM Advisory Council to interested associations and individuals; and

function as a speakers bureau by maintaining and coordinating a list of available speakers, including members of the OAM staff, members of the OAM Advisory Council, and other known alternative medicine spokespersons.

Recommendations

OAM should convene a committee of advisory panel members; OAM staff members; experts on database development; technical, organizational, and legal experts; and others to develop a workable plan to implement the public information clearinghouse mandated by Congress. To coordinate efforts and avoid expensive redundancy, the committee should include a member of the parallel committee that will plan the research database associated with NLM.

Since many institutes at NIH have clearinghouses, it is advisable to survey several of them to discover the methods they have found appropriate. Representatives from appropriate ones should be invited to meet with the committee.

The committee should address issues such as the following:

Scope of available resources.

Inclusion of future data.

Search and indexing mechanisms suitable for the general consumer (for electronic databases).

Intellectual property and copyright issues.

Fees for inclusion of material.

User fees.

Hard-copy retrieval methods.

Electronic access routes.

Comparative costs of possible approaches.

Ethical issues.

Qualitative review for inclusion.

Costs of qualitative reviews.

Specifically, OAM should do the following:

• Develop a consumer-oriented computerized inquiry system devoted to alternative medicine. This system would be similar to health consumer subsections of America OnLine and CompuServe, such as the Cancer Forum. Initially it might be based on an existing public information database, such as that operated by Brain-Mind Bulletin.
  
  

• Develop hard-copy (i.e., printed) materials for distribution to the public.
  
  

• Maintain a source within OAM to disseminate alternative medical materials and to field questions from the media and others.
  
  

• Supply treatment information for herbal medicinals, which (in accordance with current FDA policy) are currently packaged without use instructions. (Including use information on the package is a violation of current FDA rules.) Such a reference document, which is legal, would be of considerable benefit to the public.
  
  

• Such issues are more difficult to solve than acute illness, as was mentioned in the introduction to this report.
  
  

• Current projects related to alternative medicine supported by the various NIH institute include the following: National Heart, Lung, and Blood Institute: transcendental meditation in the control of hypertension, research by Dean Ornish (see the "Diet and Nutrition" chapter) in cardiac rehabilitation. National Institute of Arthritis and Musculoskeletal and Skin Diseases: refocus of the research agenda for treating fibromyalgia to include alternative clinical treatments. Division of Cancer Prevention of the National Cancer Institute: nutritional approaches to cancer prevention. National Institute on Drug Abuse: acupuncture in the treatment of substance abuse. National Institute on Allergy and Infectious Diseases: acupuncture for peripheral neuropathy in AIDS. National Institute on Aging: use of tai chi for movement disorders in the elderly. National Institute of Mental Health: biofeedback, hypnosis, and Navajo spirituality. There are also 43 projects listed in the NIH grants and contracts database related to spirituality and religion.
  
  

• From "The Need for a New Medical Model: A Challenge for Biomedicine," Science (April 1977) 196:129.
  
  

• The following general points are relevant to alternative as well as conventional medical research for which funding is sought. A proposed basic research project should address a significant fundamental question; incorporate appropriate controls; employ appropriate tests of statistical significance and power; provide adequate characterization of the treatment used and the background context; present evidence to indicate how any results with nonhuman biological systems would apply to humans; and be based on testable hypotheses. Besides having similar characteristics, a proposed clinical study should address questions of effectiveness and/or safety; offer benefits commensurate with the risks involved for patients; allow questions of effectiveness to be decided within a predictable timeframe; and when appropriate, include comparisons to other medical approaches. Appendix F, "A Guide for the Alternative Researcher," provides additional material and references on how to plan and conduct research.
  
  

• Internal validity is the certainty that the treatment or regimen under study, rather than something else, produced the study results.
  
  

• Wilson and his colleagues provided descriptions of six fictitious research studies to their research subjects. The studies were actually identical apart from the topic under investigation; noticeable methodological errors were included. One set of study subjects linked the effects of alcohol to heart disease (important) or heartburn (less serious), and another linked fast food to either cholesterol levels or acne. The persons reviewing "important" studies were considerably more likely to consider them publishable than those who reviewed "trivial" subjects.
  
  

• The use of independent evaluators and objective data collection is relevant in all research studies of methods in which the teacher or practitioner plays a major role. Besides learning situations, these methods include, for example, biofeedback therapy and manual healing methods.
  
  

• Potential alternative medical researchers can explore various approaches using the reference list provided in appendix F and using MEDLINE as a resource (see the "Research Databases" chapter as well as app. F section on the National Library of Medicine, which provides an introduction to MEDLINE). Relevant key terms can be selected for the MEDLINE search, such as patient choice and patient participation.
  
  

• Some institutions have their own panels for grants review. These standing study sections are usually concerned with evaluating training grants, center grants, and requests for applications that bear directly on specific missions of the institute.
  
  

• This report recognizes that researchers in alternative medicine have different information needs from the general public. Recommendations to address these needs are discussed in the "Research Databases" chapter.
  
  

• OAM has begun to conduct an informal clearinghouse, responding to the hundreds of phone call inquiries it receives each week requesting information about alternative therapies and research.
  
  

• Examples of recent books: (1) British Holistic Medical Association. 1986-1990. Holistic Medicine, 5 vols. Wiley, Sussex, England. (2) Burroughs, K., and M. Kastner. 1993. Alternative Healing: The Complete A-Z Guide to Over 160 Different Alternative Therapies. Halcyon, La Mesa, Calif. 356 pp. (3) Drury, N. 1981. The Healing Power: A Handbook of Alternative Medicine and Natural Health. Frederick Muller, London. 231 pp. (4) Hafen, B.Q., and K.J. Frandsen. 1983. From Acupuncture to Yoga: Alternative Methods of Healing. Prentice-Hall, Englewood Cliffs, N.J. 135 pp. (5) Linde, S., and D.J. Carrow, eds. 1985. Directory of Holistic Medicine and Alternate Health Care Services in the U.S. Health Plus, Phoenix, Ariz. 262 pp. (6) Lyng, S. 1990. Holistic Health and Biomedical Medicine: A Countersystem Analysis. State University of New York Press, Albany, N.Y. 268 pp. (7) Olsen, K. 1991. The Encyclopedia of Alternative Health Care. Platkus, London. 330 pp. (8) Weil, A. 1983. Health and Healing: Understanding Conventional and Alternative Medicine. Houghton Mifflin, Boston. 296 pp.

Return to EXPANDING MEDICAL HORIZONS