Editorial ~ JAMA 1999 (Mar 24); 281 (12): 1129-1130
A Commentary on:
Accuracy of Data in Medical Abstracts of Published Research Articles
JAMA 1999 (Mar 24); 281 (12): 1110–1111
Margaret A. Winker, MD
abstract is, aside from the title, the most frequently read and most
easily accessed portion of an article reporting original biomedical
research. The abstract provides an irreplaceable resource for busy
clinicians, researchers, and authors searching for pertinent material
in the source journal or in computerized databases. JAMA began
publishing abstracts with articles on January 7, 1956, predating
MEDLINE by a decade; other journals followed suit. 1 Structure was not added to JAMA abstracts until
1991 2 when the structured format developed by Haynes et al 3 and tested 4 and evolved 5 in Annals of Internal Medicine with the help of Edward J. Huth, MD, then Annals' editor, was adopted. Reading the abstract has never been a substitute for reading the article: crucial details of
the study, such as patient selection and follow-up, definition of
outcome measures, and study limitations, receive short shrift in the
terse style of the abstract. 6, 7 A simple and
straightforward abstract may obscure a more complex (and realistic)
story within the text. These limitations aside, however, the abstract
provides the reader with an efficient summary of the study that
facilitates scanning many articles to find those that are the most
pertinent to the reader's interests and needs.
The structure of abstracts is based on the components that are
essential elements of teaching critical appraisal of the
literature. 5 Structure appears to have improved the quality and usefulness of the abstract. 8-10 Of course,
"quality" of an abstract depends on what the reader wants to know;
one abstract format cannot serve the needs of all
readers. 11, 12 However, the author can convey the key
elements of the study, just as a journalist conveys the key
elements of a news story: the abstract's sections Context, Objective, Design, Setting, Patients, Intervention, Main Outcome Measures,
Results, and Conclusions should tell the reader why, what, how, where and when, who, what was done, what was measured, what was found, and
what it means. Within these categories, certain information should be
included that provides the reader with essential information and
accurately reflects the material in the text. 5
Accurately reflecting the material in the text may seem the most
basic requirement for an abstract. However, in this issue of THE JOURNAL, Pitkin and
colleagues 13 find an astounding 18% to 68% of 264 abstracts in 6 large general medical journals Annals of Internal
Medicine, BMJ, CMAJ, JAMA,
Lancet, and New England Journal of Medicine had data in the abstract that were either inconsistent with or absent from the
main body of the article. In addition to the astonishingly high number
of deficiencies, the frequency of errors varied significantly from
journal to journal. This result was especially troubling because
abstracts are widely used, often separate from their text, as in
MEDLINE and other databases, and data taken from the abstract may be
reported and disseminated in other works, in other formats, and in the
When preliminary data demonstrating this problem were presented at the
Third International Congress on Peer Review in Biomedical Publication
in September 1997, along with the results of another study by Pitkin
and Branagan 13 demonstrating that authors did not improve the quality of their abstract in response to specific instructions,
JAMA took this study as a mandate to develop and implement
abstract quality control procedures, which began with the January 1,
1998, issue. We developed quality criteria (Table 1), with a focus not only to improve accuracy, the bare minimum that should be expected of an
abstract, but also to improve the quality of what was reported. Since
then we have used these criteria to review and edit the abstracts of
all articles accepted for publication.
The quality criteria were developed using evidence wherever possible,
built on work of previous authors and supplemented by common sense.
Item 1 reflects the importance of the structured abstract categories
for reporting specific components of a study. 5 Items 2 and 3 are based on the findings of the study by Pitkin and
Branagan. 14 Item 4, the years in which the study was conducted and the length of follow-up, provides the reader with the
currency and scope of the study. Item 5 is included to improve reporting of negative as well as positive results. 5 Item 6 emphasizes that results should be quantified, preferably with confidence intervals, 5 because verbal expressions of
frequency are interpreted differently by physicians, the general
public, and physicians with different native languages. 15
Absolute differences rather than relative differences, item 7, provide
the reader with a more accurate understanding of the effect of treatment for a population of
patients. 5 Item 8 states that, for
randomized trials, the intent-to-treat analysis should be specified and
included so that results are not influenced by differential dropout
rates. 16 For surveys, the response rate, item 9, is an
important measure of quality. When potentially confounding factors have
been controlled for in a model, they should be specified to enable the
reader to determine whether uncontrolled confounding may remain (item
10). Finally, item 11 stipulates that conclusions should be consistent
with the study results, and the stated implications of the study should
be reasonably circumspecta subjective call, but a frequent debate.
Have our efforts improved abstract quality? A preliminary assessment of
abstracts before and after the quality criteria were implemented was
reassuring. While our baseline data were consistent with the disturbing
results of the study by Pitkin et al 13 more than half of
the 21 original research articles published in November 1997 had some
discrepancy between the abstract and the text no discrepancies between the abstract and text were identified in the 27 articles published in
November 1998 after the quality improvement step was instituted
(J. C. Lantz, MLA, ELS, unpublished data, November 1998).
While the data presented should be accurate and consistent for every
abstract, other components of the quality criteria are more subjective.
We use our quality criteria as guidelines, and perfect adherence may
not be achieved. Even with constant attention to abstract quality,
since a primary goal of the abstract is to provide a concise summary of
the study, the quality criteria sometimes must be compromised to
prevent the abstract from duplicating the text.
Many important issues remain that have not been addressed by Pitkin et
al or by our quality criteria. Are abstracts as accessible as they
should be to both professionals and consumers? The Cochrane abstract
guidelines aim to make abstracts as "readable as possible without
compromising scientific integrity" (P. Middleton, BSc, written
communication, February 11, 1999). Shouldn't medical journals take a
similar tack? Why do discrepancies occur between the abstract
text? What form of abstract reporting is most helpful to
physicians and others practicing evidence-based medicine? 17
Are different types of abstracts for a single article necessary to meet
the differing needs of readers? And most important for editors and
authors to address and researchers to evaluate, is essential
information in the article missing from the abstract? 5, 12
While this question is the most difficult to assess, if the answer is
yes, for the reader the abstract may cause more harm than good.
Regardless of the answers to these questions, authors, reviewers, and
editors should pay increased attention to abstracts. The abstract must
truly reflect the study, both in terms of specific data and overall
message. After reviewing the abstract, the reader who does not have
time to refer to the text should have an accurate impression of the
study and should obtain useful information. Based on the findings of
Pitkin and colleagues, 13 readers should be cautious and not assume that information reported in the abstract accurately reflects
that in the text. However, the concrete responsibility belongs to
editors and authors alone: they must improve the quality of abstracts
to help ensure that studies achieve the maximum possible benefit for
patients, physicians, and the research community.
Author Affiliation: Dr Winker is Deputy Editor of
Corresponding Author and Reprints: Margaret A. Winker, MD,
JAMA, 515 N State St, Chicago, IL 60610 (e-mail: email@example.com).
Editorials represent the opinions of the authors and THE JOURNAL and not those of
the American Medical Association.
Acknowledgment: I thank Jane C. Lantz, MLA, ELS, for her thoughtfulness and diligence in the pursuit of better abstracts.
N Engl J Med.
Rennie D, Glass RM.
Structuring abstracts to make them
Ad Hoc Working Group for Critical Appraisal of the
A proposal for more informative abstracts of
Ann Intern Med.
Structured abstracts for papers reporting
Ann Intern Med.
Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ.
More informative abstracts revisited: a progress report.
Froom P, Froom J.
Deficiencies in structured medical
J Clin Epidemiol.
Narine L, Yee DS, Einarson TR, Ilersich AL.
abstracts of original research articles in CMAJ in 1989.
Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR.
Quality of nonstructured and structured abstracts of
original research articles in the British Medical Journal, the
Canadian Medical Association Journal and the Journal of
the American Medical Association.
Trakas K, Addis A, Kruk D, Buczek Y, Iskedjian M, Einarson TR.
Quality assessment of pharmacoeconomic abstracts of
original research articles in selected journals.
Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB.
Preliminary assessment of the effect of more informative (structured)
abstracts on citation retrieval from MEDLINE.
The structured sonnet.
Structured abstracts: a modest dissent.
J Clin Epidemiol.
Pitkin RM, Branagan MA, Burmeister LF.
Accuracy of data
in abstracts of published research articles.
Pitkin RM, Branagan MA.
Can the accuracy of
abstract be improved by providing specific instructions? a randomized
Nakao MA, Axelrod S.
Numbers are better than words:
verbal specifications of frequency have no place in medicine.
Intention to treat analysis: implications
for quantitative and qualitative research.
Int J Epidemiol.
Naylor CD, Williams JI, Guyatt G.
of proposals for clinical and epidemiological studies.
Return to the EVIDENCE–BASED PRACTICE Page