A point of view
On the Nature of Evidence
François Auclair MD
------------------------------------------------------------------------------
Origins of the Concept
We are told that medicine should be founded on evidence. Like a hard science
such as physics, medicine has found its own way of eliminating hearsay,
authority, and opinion to adopt a deductive-nomological (ruling) model. The
model states that if H (hypothesis or theory) implies E (evidence) in
specified conditions, and if this relation holds most of the time (and we
give it a five per cent chance of error), then we can justifiably deduce that
if H is true, then E will occur, and we can explain E by its relation to H.
We call H disease mechanisms, health determinants, or quality-of-life
measurements, and for E, we give values such as tests results, drug studies,
surgical interventions, or effects of treatment and care. Physicists chose O
for observation instead of E, but evidence is more suggestive of clarity, so
we physicians adopted it.
Cicero invented the word "evidentiam" in translating "energeia," a term that
the Greeks used when referring to brightness or clarity. Evidence then meant
that no proof was needed, it did not need explanation, it was self-evident.
This idea persisted for centuries. Descartes used evidence in the sense of
clear and distinct, so much so that it was immune to doubt, such as the
sentence "I think." Evidence in those days applied to a few things or
sentences. It was not until the 17th century that evidence came to mean
something pointing beyond itself: a sign, a symptom, or today a clinical
study. What it is pointing to is a hypothesis. But what evidence has gained
in the number of classes it could be applied to, it lost in clarity. All
types of things can be evidence for all sorts of things, as lawyers know. We
may help resolve a few quandaries about the nature of evidence in medicine
with a definition.
Defining Evidence
One definition states that evidence is the best clinically relevant research
on diagnosis, prognosis, therapy, or prevention.1 These terms qualify as
synonyms for best evidence but do not instruct us on their common nature. We
can also look at the use that we make of evidence since use confers meaning.
We have established criteria: grade I evidence is represented by a randomized
clinical trial, grade II would be a non-randomized trial, and grade III is
illustrated by observations and reports from expert committees.2 These
represent formal features of evidence. Obtaining certain forms of evidence
such as a randomized trial is desirable because it represents a generally
reliable process, but is it a necessary or sufficient criterion to use
evidence?
In 1981, the first cases of Pneumocystis carinii infection in homosexual men
were reported.3 The evidence was an astute observation of five patients. This
resulted in the most important search for answers of the last decades in
medicine. Physicians who read the report of this observation had to look for
similar cases and then act on the basis of such evidence with diagnostic
tests for Pneumocystis carinii, something that they would not have done
before. Proponents of the importance of formal features of evidence would
concede that an exceptional observation is likely significant but that for
clinical practice, the categories of evidence remain the most significant
criteria.
In early 1983, a viral etiology of acquired immunodeficiency syndrome (AIDS)
was not yet documented. Despite this, experts, using an analogy with
hepatitis B, declared that the probability of an infectious cause was high
and that there should be keen awareness of blood-product associated cases.4
The only evidence for blood transmission available at the beginning of 1993
consisted of isolated reports involving patients who had received blood
products and could have secondarily developed the disease.5-7 This led to the
statement that the available medical and scientific evidence that AIDS could
be spread in blood components was incomplete.8 Based on incomplete evidence,
measures were taken by U.S. blood banks to screen blood from high-risk
groups.8 At the same time in Canada, the Red Cross considered that the
probability of developing AIDS after blood transfusion was low, which was
true, and a less aggressive approach was taken to screen blood. In 1997, the
Commission of Inquiry on the Blood System in Canada concluded that the Red
Cross should not have required "conclusive evidence" before taking action to
reduce the risk of AIDS.8 It is a matter of what difference we believe the
evidence can make and not its form that will result in us accepting to act on
it.
A reliable clinical trial may also be insufficient. By 1993, several
randomized clinical trials had documented the benefit of treating
hypertension.9 Despite this, American and British experts formulated
different recommendations. Treatment of hypertension with the diastolic blood
pressure over 90 was more universally recommended in the United States. The
approach that was subsequently recommended takes into consideration the fact
that reduction in cardiovascular risk by treatment of high blood pressure is
a matter of degree depending on associated risk factors.9 This conclusion did
not come from new empirical data but from inference from previous trials
including the Framingham study.10 It was a matter of the difference the
evidence can make that was of prime consideration.
So we use evidence according to the differences that we perceive. If the
perceived difference is that the benefit is high compared with the risk, we
act as if the probability of the hypothesis is high. If a treatment has a low
probability of cure for a previously incurable disease, we act as if the
hypothesis is higher because there is much to gain if the risk is low. On the
other hand, if a treatment has a high probability of cure for a benign
condition, we act as if the hypothesis is lower if the risk associated with
the treatment is significant. In the practice of medicine, we attribute an
apparent probability to the hypothesis, a subjective probability.11 This is
the link between evidence and the hypothesis. To be coherent in his or her
action, a physician should consider the hypothesis to be probable to a degree
x given the evidence if it is preferable that everyone acts as if the
hypothesis had such a degree of probability. We now have, based on its use,
the first part to a definition of evidence: any observation that changes the
subjective probability of a hypothesis in such a way that it is preferable to
act based on it. But how can I justify my belief in the observation itself?
Methods to evaluate the validity of evidence include appraisal of
publications by evaluating diagnostic, prognostic, and therapeutic reports.
The evidence can then be accepted or rejected after this appraisal. The
problem with this method is that there is a lot of evidence. In a review of
randomized clinical trials on 19 cardiology topics, results supportive of the
hypothesis were obtained in 44 instances, while conflicting results were seen
in 52 cases.12 Not to accept contradictory evidence, one has to possess the
knowledge of the field supporting the hypothesis. Recently, discussions on
meta-analysis yielded the conclusion that not to accept wrong treatment or
not to reject useful treatment, physicians should look at the included
studies, evaluate the consistency of their results, or appraise each trial
separately.13 Considering the time that this would require, this is
unrealistic for a non-expert physician.
In the 1980s, clinical trials showed that the addition of vancomycin in the
treatment of immunocompromised hosts with neutropenia and fever resulted in
more rapid resolution of fever with less morbidity.14,15 Official
recommendations suggested to use regimens containing vancomycin at the onset
of fever.16 A non-expert physician would have been justified to use
vancomycin. Other experts were not favoring such use because of the marginal
benefits and the theoretical risk of inducing vancomycin-resistant
organisms.17 Only experts in the treatment of febrile neutropenic patients
could detect these incoherences and choose the most preferable hypothesis,
which was to recommend against the use of vancomycin at the onset of fever.
Expertise is needed to evaluate evidence, lawyers know that also. We are now
directed to evidence-based medicine journals, which are a collection of
interpretations by experts in clinical studies.1 This leads us to the second
part of our definition of evidence: any observation that has survived the
tribunal of expertise.
Consequences
To qualify as a basis for medicine, evidence should have two characteristics
that together are necessary and sufficient: it should consist of an
observation that has been assessed as valid by experts knowledgeable in the
field, and it should impart such a degree of probability on the hypothesis
that it is preferable that we act on it. This definition places evidence in
logical relationship within a scientific model and draws boundaries to its
vagueness. The implication of the first part of our definition is simple;
physicians should be wary of accepting or rejecting evidence on the sole
examination of its internal structure without expert knowledge. We still need
a good textbook. The Red Book is an example and the Cochrane library is an
emerging form of textbook. The implication of the second part is more
complex. Evidence confers subjective probability to a hypothesis based on
assessments of differences (for example, risks versus benefits). The
assessment of differences is the domain of judgment. Judgment in medicine is
the ability to perceive differences, the ability to discriminate between a
banal observation and a significant one, the ability to perceive degrees, the
ability to estimate the degree of coherence of a set of theses. If we want to
improve how we practise medicine, we have to improve our ability to judge.
Practice in decision analysis and assessment of probabilities is a start.
There may be no shortcuts, however, to the acquiring of sound clinical
judgment; perhaps "judgment is the understanding that comes with age."18
References
Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine.
How to practise and teach EBM. London: Churchill Livingstone, 1997.
Hayward RSA, Laupacis A. Initiating, conducting, and maintaining guidelines
development programs. Can Med Assoc J 1993;148:507-12.
Centers for Disease Control. Pneumocystis pneumonia, Los Angeles. Morb Mortal
Weekly Rep 1981;30:250-2.
Curran JW, Essex M. Epidemic acquired immune deficiency syndrome:
epidemiologic evidence for a transmissible agent. J Natl Cancer Inst
1983;71:1-4.
Centers for Disease Control. Pneumocystis carinii pneumonia among persons
with hemophilia A. Morb Mortal Weekly Rep 1982;31:365-7.
Centers for Disease Control. Update on acquired immune deficiency syndrome
(AIDS) among patients with hemophilia A. Morb Mortal Weekly Rep 1982;31:644-
52.
Centers for Disease Control. Possible transfusion-associated acquired immune
deficiency syndrome (AIDS), California. Morb Mortal Weekly Rep 1982;31:652-4.
Krever H. Commission of inquiry on the blood system in Canada. Final report,
volume 1. Ottawa: Minister of Public Works and Government Services Canada,
1997.
Kaplan NM. Clinical hypertension, seventh edition. Baltimore: Williams &
Wilkins, 1998.
Jackson R, Barham P, Bills J, et al. Management of raised blood pressure in
New Zealand: a discussion document. BMJ 1993;307:107-10.
Ramsey FP. Truth and probability (1926). In: Mellor DH, editor. Philosophical
papers. Cambridge: Cambridge University Press, 1990:52-96.
Horwitz RI. Complexity and contradiction in clinical trial research. Am J Med
1987;82:498-510.
LeLorier J, Grégoire G, Benhaddad A, Lapierre J, Derderian F. Discrepencies
between meta-analyses and subsequent large randomized, controlled trials. N
Engl J Med 1997;337:536-42.
Karp JE, Hick JD, Angelopulos C, et al. Empiric use of vancomycin during
prolonged treatment-induced granulocytopenia. Am J Med 1986;81:237-42.
Shenep J, Hughes WT, Robertson PK, et al. Vancomycin, ticarcillin, and
amikacin compared to ticarcillin-clavulanate and amikacin in the empirical
treatment of febrile, neutropenic children with cancer. N Engl J Med
1988;319:1053-8.
Hughes WT, Armstrong D, Bodey GP, et al. Guidelines for the use of
antimicrobial agents in neutropenic patients with unexplained fever. J Infect
Dis 1990;161:381-96.
Rubin M, Hathorn JW, Marshall D, Gress J, Steinberg SM, Pizzo PA. Gram-
positive infections and the use of vancomycin in 550 episodes of fever and
neutropenia. Ann Intern Med 1988;108:30-5.
Kant I. Anthropology from a pragmatic point of view (1800). Gregor MJ,
translator. The Hague: Martinus Nijhoff, 1974.
Address for reprints: F. Auclair, Division of Infectious Disease, Ottawa
Hospital, Civic Campus, 1053 Carling Ave., Ottawa ON K1Y 4E9.
© The Royal College of Physicians and Surgeons of Canada
