Introduction

Evidence-Based Medicine clinicians face two major challenges in accessing medical knowledge: (1) finding the most relevant and reliable information to address a clinical problem at hand and (2) keeping abreast of important new developments in the expanding mass of published research.

If this were not challenge enough, time constraints dictate the need for extremely efficient strategies to meet these ends. Fortunately, all this information is now available online at one's fingertips. But, since almost anyone with a server can post any type of material, we need to find a way to identify information that is truly evidence-based and efficiently navigate through that which is not.

Chapter 6 of Evidence-Based Medicine: A Framework for Clinical Practice and this companion Web page are devoted to searching the Internet to help you efficiently retrieve high quality medical information and keep up-to-date with new developments. In addition, this site will help you stay on your evidence-based medicine toes by ensuring that you are well equipped to identify, retrieve, appraise and apply the most relevant information to the care of your patient.

This site comprises 4 sections. The first presents a search strategy, the second, an approach for keeping yourself current, and the third recommends Evidence-Based Medicine resource sites to help you exercise your EBM skills. The final section summarizes the Web addresses referred to in the first two sections to provide you with a single handy resource that may serve as your base for retrieving medical information.

Links to Sections of This Page

1) A SEARCH STRATEGY

2) AN APPROACH FOR KEEPING UP-TO-DATE

3) EVIDENCE-BASED MEDICINE RESOURCE SITES

4) SUMMARY TABLES

 

1) A SEARCH STRATEGY

The goal of our search strategy is to retrieve the most reliable information in as short a time as possible. Priority is given to sites that are most likely to yield valid information and which cover large amounts of research on a given topic. Therefore we first search for reliable sources of integrative literature. If these time saving resources do not contain the information we seek, we next search the "gold standard" (but more time consuming) medical database, MEDLINE. If we are still unsuccessful, we then perform specific medical Internet searches. Finally, if we have still not found the required information we may "dredge the Net" by inputting any related search term into one of the many general search engines, with escalating caution as to the quality of data we are likely to retrieve.

In Evidence-Based Medicine we frame the clinical question, identify and retrieve all the relevant information, appraise it, and integrate it so that it may ultimately be applied to the care of our patient. Each of these steps takes time - far more than is available in the ten minute managed care visit. While this may feel like an impossible task, a number of sites present information in which this sequence of steps has already been completed for us. This information, known as "integrative literature," provides summary findings or recommendations in the form of overviews, meta-analyses, practice guidelines, or consensus statements.

But before we rush to use these efficient resources, we need to ensure that each step has indeed been rigorously performed; just because the literature has been integrated does not guarantee it is valid. When integrative literature is retrieved, it should be critically appraised. Guides, such as those presented in Evidence-Based Medicine: A Framework for Clinical Practice, Chapter 10 or the User's Guide to the Medical Literature, can assist this endeavor.

While critical appraisal takes time, there are groups that assemble many reviews or guidelines using a reliable uniform methodology that is often available as a supporting article for our scrutiny. Just reading the group's supporting article once enables us to assess their entire body of work. If we are satisfied that their method of assembling literature is rigorous, we can be fairly confident that the review or guideline we are reading (often, in abstract form) is evidence-based, and that its host site is likely to be an efficient and reliable location for us to initiate our search.

A number of sites do not produce integrative literature, but rather collect it from a variety of sources. Some of these sites appraise the quality of information they present to help us ensure that the literature is reliable. If we cannot appraise the integrative literature or it has not been assembled by an organization that we know and trust, we should proceed with caution.

The following are examples of sites that we may find useful in our search for valid integrative literature.

The Cochrane Collaboration
(http://www.cochrane.org/cochrane/revabstr/mainindex.htm)

The Cochrane Database of Systematic Reviews (CDSR) is a rapidly growing collection of regularly updated systematic reviews. Its contributors are an international group of individuals and institutions dedicated to the ambitious endeavor of summarizing all randomized controlled trials relevant to health care. New reviews are added with each issue of "The Cochrane Library". The process of assembling reviews is rigorous; each review focuses on a particular clinical question, and goes to great lengths to retrieve all the relevant trials, each of which is critically appraised. The studies meet specific quality criteria are then summarized quantitatively, whenever possible, using the meta-analysis. Otherwise the summary findings are presented qualitatively.

DARE
(http://144.32.228.3/scripts/WEBC.EXE/NHSCRD/start)

The Database of Abstracts of Reviews of Effectiveness comprises a comprehensive collection of reviews from around the world (including those from the Cochrane Collaboration). Each article is critically appraised by reviewers at the National Health Service Centre for Reviews and Dissemination at the University of York, England. For good quality reviews, detailed structured abstracts describing the methodology, results, and conclusions are produced. Comments on the overall quality of reviews and their implications for health care are also included. Of particular note is that this vast database is indexed not only by keywords, but also by the official Medical Subject Headings, or MeSH terms of the MEDLINE database which enables us to improve the specificity of our search. (MeSH terms are discussed in detail in the MEDLINE section below.)

NIH: Health Services Technology Assessment Text
(http://text.nlm.nih.gov/)

HSTAT, developed by the National Library of Medicine, provides access to the full-text of documents useful in health care decision making. It includes a host of clinical practice guidelines, quick-reference guides for clinicians, consumer brochures, and evidence and technology assessment reports all sponsored by the Agency for Health Care Policy and Research; numerous consensus development conference and technology assessment reports sponsored by the National Institutes of Health; as well as the entire text of the U.S. Preventive Health Services Task Force Guidelines from 1996.

National Guidelines Clearinghouse
(http://www.guidelines.gov/index.asp)

Operated by the U.S. Department of Health and Human Services, Agency for Health Care Policy and Research (AHCPR), in partnership with the American Medical Association (AMA) and the American Association of Health Plans (AAHP), this site disseminates a wide range of clinical practice guidelines from more than 100 institutions and organizations. Structured abstracts are provided to facilitate critical appraisal. If our search retrieves more than one guideline, the abstracts describing the process of guideline assembly may be displayed in a table allowing side-by-side comparisons of relevance, rigor and generalizability. Links are provided to the major recommendations and, where available, to the full-text of each guideline.

The American Heart Association
(http://www.americanheart.org/)

This site contains extensive information on heart disease and stroke for patients and providers of care. Clinicians will find a multitude of guidelines, consensus statements and biostatistical fact sheets relating to the various aspects of cardiovascular and cerebrovascular disease. Patients will find helpful information on warning signs and risk assessment as well as a comprehensive A to Z list for topics related to these highly prevalent diseases. 

The National Cancer Institute: CancerNet
(http://cancernet.nci.nih.gov)

CancerNet provides a tremendous resource of cancer information for health care providers, patients, and researchers. Updated monthly, it is continually reviewed and revised by oncology experts and is based on the latest research in the field. At the heart of this site is PDQ, the NCI's comprehensive cancer database, containing peer-reviewed summaries on cancer staging, treatment, screening, prevention, and supportive care; a registry of clinical trials from around the world; and directories of physicians and organizations that provide cancer care. In addition, this site provides a link to CANCERLIT Topic Searches that provides citations and abstracts from the NCI's bibliographic database.

Centers for Disease Control
(http://www.cdc.gov/)

While the CDC site covers a broad range of topics, it provides two links that provide especially helpful information on infectious disease. The Morbidity and Mortality Weekly Report (MMWR) provides a searchable index of a wide array of "Recommendations and Reports" including specific advice on immunizations, antibiotic choices for specific infections, and information on the diagnostic criteria for a number of infectious diseases. You can subscribe (for free) to a weekly mailing list that will send you the electronic version of MMWR. The link to Traveler's Health provides extensive information regarding specific infectious disease prophylaxis according to the region that will be visited.

If we are unable to find the information we require in the above integrative literature sites, or if we believe the information we require is too specific to be found there in the first place (e.g., the incidence of rash with a specific drug), we need to invest more time to search MEDLINE.

MEDLINE, the National Library of Medicine's (NLM) premier bibliographic database, has been free to all users since June 1997. It contains over 11 million citations dating back to the mid-1960s and includes author abstracts from more than 4000 biomedical journals published in the United States and 70 other countries.

The NLM has developed a powerful interface for searching MEDLINE called PubMed. Other Internet sites may allow us to search MEDLINE, but in general, their search capabilities are not as powerful as PubMed's.

PubMed via the National Library of Medicine
(http://www.ncbi.nlm.nih.gov/entrez/query)

PubMed has excellent "Help" and "Tutorial" links to optimize use of this user-friendly search tool — within a couple of hours of using these resources you will be able to search with ease and proficiency. In addition, you may find it helpful to read Chapter 5 of Evidence-Based Medicine: A Framework for Clinical Practice, which is entirely devoted to the subject of searching MEDLINE. What follows are the essentials of using PubMed.

The Key to Successful PubMed Searching

While we can search for articles on PubMed by using keywords, title words, and author names, more powerful searches rely on sophisticated MEDLINE-specific designations, known as MeSH (Medical Subject Headings). MeSH are terms assigned by indexers at the NLM to identify topics discussed in an article. These terms are then entered into the database along with the citation. For example, authors writing about lung cancer may use one of these phrases in the title or the abstract: "lung cancer," "pulmonary cancer," or "pulmonary neoplasms." The MeSH term for all of these phrases is "lung neoplasms," thus enabling us to find all the articles on this topic, even when authors have used other terms to discuss it.

MeSH terms may be further modified using MeSH subheadings, which enable us to make the search more specific. For example, if we are interested in lung cancer screening, the MeSH term can be modified with the subheading, "prevention and control."

While successful searches are frequently determined by the use of appropriate MeSH terms, these terms are often not intuitive. Who would guess that the key MeSH term to search for articles on lung cancer would be "lung neoplasms," or that the subheading for screening would be "prevention and control"? We need assistance to incorporate the appropriate MeSH terms and their subheadings in our searches. PubMed provides three approaches that enable us to harness the full power of MeSH:

1. The PubMed Mapping Feature

If we simply type a word or phrase into the PubMed search line and click on "Go," PubMed uses a sophisticated mapping function to translate our search into more appropriate terms. In addition to using our original text words, the mapping feature invisibly adds other terms by sequentially processing our search through four separate databases.

First, the mapping feature tries to match our search terms with the appropriate MeSH term in a MeSH Translation Table using an index and the Unified Medical Language Structure (UMLS) Metathesaurus, an "electronic Rosetta stone," that contains over a million synonyms for specific MeSH terms and their subheadings. If the MeSH Translation Table does not make a match, PubMed will then try to map our search terms to a Journal Title Index. Next, the system will check a Phrase List of several thousand phrases. Finally, PubMed will try to match our search to names in the Author Index.

After performing a search, we can view how PubMed mapped our search to MeSH and other terms by clicking the "Details" button on the search retrieval screen. Sometimes it's helpful to view the particular terms selected by the mapping feature — we may use them to build a more powerful search.

Our search is profoundly influenced by the use of the Boolean operators, AND, OR, and NOT. (Note: PubMed only recognizes these operators if they are typed in caps). The Boolean operators you will use most often when searching MEDLINE are AND and OR. The AND operator is used to combine two or more topics and retrieve citations to articles that discuss all topics together. For example, if we want to retrieve citations on the use of "Cholecystectomy" for "Cholelithiasis," we can combine the two using AND: "Cholecystectomy AND Cholelithiasis." If we want to be more comprehensive in our subject search, we might add synonyms for "Cholelithiasis" (such as "Gallstones") to our search strategy. In this case we would use the OR operator: "Cholelithiasis OR Gallstones." We will now retrieve all citations that use the term, "Cholelithiasis," all citations that use the term "Gallstones," and all citations that use both terms.

If we do not include a Boolean operator in our search, and the mapping function does not recognize a string of words as a MeSH term, journal title, phrase, or author name, the individual words are automatically linked by the Boolean operator, AND.

2. Using the MeSH Browser

If we want to begin building our search with the most appropriate MeSH terms, but are not sure what they are, we may click on the "MeSH Browser" link located under the PubMed Services column on the left hand side of the page.

For example, if we are interested in the treatment of gallstones and enter "gallstones" into the query box and click on the "Go" button, we are informed, "Gallstones is not a MeSH term, but it is associated with the MeSH term Cholelithiasis."

The MeSH Browser also shows us where cholelithiasis lies in the MeSH tree structure, a hierarchy of MeSH terminology where concepts proceed from the general to the specific. For example, articles on "Common Bile Duct Calculi" lie beneath the more general term "Cholelithiasis" on the MESH tree.

PubMed automatically "explodes" (includes) all subjects below a MeSH term when performing a search. Thus, a search on the more general term, "cholelithiasis," will also retrieve articles on the subordinate term, "common bile duct calculi." This naturally makes our search more comprehensive. We can also choose not to explode "cholelithiasis," but then we only retrieve articles that are indexed at this level in the tree.

We can easily perform a more focused search on the treatment of Cholelithiasis by appending a subheading to the chosen MeSH term; we simply click on the "Detailed display" link and are provided with a checklist of all possible subheadings for this MeSH term. In this example, we would check the "therapy" box. We are also given the option to check the "Restrict Search to Major Topic Headings Only" box. This further improves the specificity of our search because it instructs PubMed to retrieve only articles where our selected MeSH term and Subheading are tagged as a major topic of the article. By clicking on the "Add" button, we are effortlessly returned to a window where we find "cholelithiasis/therapy [MAJR]" typed in for us in the appropriate search format. All we have to do now is click on the "PubMed Search" button to run the search.

(Note: While PubMed adds citations to the MEDLINE database as soon as they are received from journal publishers, it takes, on average, 4-6 weeks for the citations to be indexed with MeSH terms. Thus, MeSH Browser searches will miss these pre-indexed articles.)

3. Using PubMed's Clinical Queries Feature

PubMed has a unique feature called the "Clinical Queries" page (also located under the PubMed Services column on the left hand side of the page). It's particularly helpful for retrieving articles on issues of therapy, diagnosis, etiology, or prognosis. With a simple click of a button we may designate any one of these study categories to activate built-in "research methodology filters." These filters consist of particular strings of search terms (including predetermined MeSH terms, subheadings, and publication types) that have been pre-tested to optimize the retrieval of articles on each study category. Further, we can request that our search be more sensitive (ie, includes most relevant articles but probably some less relevant as well) or more specific (ie, includes mostly relevant articles but probably also omits a few). These search terms are largely based on the work done by Haynes RB et al. (see table).

After using any of the three above approaches to run our search, we may process it further by applying limits, displaying it in various formats, identifying related articles, collecting articles of interest on a clipboard, and saving articles to a file. Finally, we may print citations and abstracts and retrieve the full text of articles.

Applying Limits to a Search

Even if we perform a relatively focused subject search, we often retrieve more articles than are useful. MEDLINE allows us to limit our search further by clicking the "Limits" link below the PubMed search window. Doing so enables us to restrict our search by language, study group (human or animal), gender, age group, journal, date, and publication type.

Of particular note, the publication type limit allows us to narrow a search to Meta-Analyses, Reviews, and Practice Guidelines. Thus, if our search retrieves many articles we can search most efficiently by first reviewing the integrative literature. Next, before reviewing a slew of studies, letters to the editor, and editorials, etc., we can retrieve studies with the most methodologically rigorous designs by clicking on the publication type limit, 'Randomized Controlled Trials.'

Displaying Articles

PubMed displays the first 20 citations of our search in a "Summary" format that includes: author name(s), title, journal source, and the language if the article isn't English. We can display more than 20 items per page by clicking on the "Show" pull-down menu and selecting a larger number, up to 500 citations per page. This allows for continuous scrolling without repetitious page click-throughs or download waits.

A number of other display options are available. We can use the pull-down menu to the right of the "Display" button to view them:

"Brief" — Shows only the author's name and the start of an article's title.
"Abstract" — Shows the information in the Summary display, plus the abstract.
"Citation" — The same as the Abstract display, but also includes the MeSH terms.
"MEDLINE" — Displays two-letter code tags for each field in the MEDLINE citation. These codes are read by software like EndNote and may be used to load MEDLINE records into personal bibliographic databases.

Retrieving Related Articles

To the far right of each citation on the search retrieval screen we find a link called "Related Articles." Clicking this link will retrieve a set of articles that PubMed feels are closely related in topic to that particular citation. Related article sets are pre-calculated by PubMed and usually retrieve 100 to 150 additional citations. These sets are created using an algorithm that looks at words in a citation's title, abstract, and MeSH fields. The Related Articles feature is useful for finding additional articles in MEDLINE, particularly when we are having difficulty locating anything of relevance.

Collecting Articles on the Clipboard

The search retrieval screen also features a button labeled "Add to Clipboard." At any time when searching PubMed, we can select and save citations to be printed or downloaded at a later time by clicking the box to the left of the citation, and then clicking the "Add to Clipboard" button. The selected citations are added to the PubMed Clipboard and held there for up to one hour of inactivity before being cleared off the PubMed system. To enter the clipboard at any time and view what has been saved, we click on the "Clipboard" link that appears under the PubMed search window.

Printing and Saving Articles

To print or download search results in PubMed, we first select a desired "Display" format in which we want to print or save the citations.

To print, we click the "Text" button to the right of the display options. This displays our search retrieval in a text format, without all the graphics on the regular PubMed screens. We can then print the citations using the print options in our Web browser.

To save citations to a file on our hard disk, we click the "Save" button to the right of the display options. A pop-up window then asks us to select a location on the computer to which to save the file where it can later be read by a word processor. (Note: If you are planning to load saved citations into EndNote, or other bibliographic management software, remember to save the citations in the "MEDLINE" format.)

Retrieving the Full text of Articles

After retrieving the best list of citations/abstracts to answer the particular search question, we can decide which articles to retrieve in full. PubMed now provides convenient links to the full text of 1800 journal titles via the publishers' Web site, where, often for a fee, the article can be downloaded. We can also order articles though "Loansome Doc" and, for a fee, have them delivered to a library where we can pick them up or arrange to have them forwarded to us. Finally, if you have the time and inclination, a trip to the library stacks is always an option, and is free.

If we have not found the information we require searching high grade integrative literature sites or MEDLINE, or if the information is unlikely to be found there in the first place (for example, we are looking for multimedia presentations, educational tools, or patient support services), we need to search more broadly on the Internet. But before we turn to the more familiar Internet search engines in Step d below, we first search using methods that are more specifically tailored to retrieve medical information from the Web. These sites search the Internet using MEDLINE MeSH terms, specific "medical dictionaries," or browse Web pages organized by organ system, specialty, or disease process.

However, since we have descended to the third step of our search strategy, we should also appreciate that the information we retrieve will less frequently be rigorously assembled, peer-reviewed, or produced by highly trusted institutions or organizations. Escalating caution is warranted.

CliniWeb at Oregon Health Sciences University
(http://www.ohsu.edu/cliniweb/)

CliniWeb is an index and table of contents organized using the Medical Subject Headings (MeSH) disease and anatomy classifications, with access provided by both searching and browsing. Currently nearly 10,000 URLs are indexed, and the pages selected contain clinical information for health care education or practice, targeted at the level of students or providers. Searching may be entered in five different languages (English, German, French, Spanish, and Portuguese), though articles are currently all retrieved in English. We can only search using MeSH terms. For example, searching "gallstones" retrieves not a single Web page, but searching "cholelithiasis" retrieves fourteen sites. Alternatively, we may browse the index by "Anatomy" or "Diseases" if we are not sure of the specific MeSH term to use. The site also links back to PubMed, enabling us to perform "canned" searches on reviews, therapy articles, disease articles, or all articles, related to our MeSH term.

Current Medical Diagnosis & Treatment - Companion Web Page
(http://www.pbg.mcgraw-hill.com/medical/lange/cmdt/)

This well-known medical textbook is arranged by discipline, organ system, and disease. Its companion Web page provides links to selected sites for each of the book's chapters. Each of the sites is rated according to whether it is peer reviewed, doctor oriented, non-commercial, multimedia enhanced, and whether it has patient resources.

Hardin Meta Directory
(http://www.lib.uiowa.edu/hardin/md/index.html)

As the name "Meta Directory" implies, Hardin MD is a comprehensive "list of lists." The lists are arranged according to organ system and specialty. It includes subject listings in large catalogue sites, such as MedWeb, Medmark, Yahoo:Health, Karolinska Institute MIC-KIBIC MeSH Index, Medical Matrix, Martindale's Health Science Guide, Tward's Multimedia Medical Reference Library, and EiNet Galaxy, as well as independent discipline-specific lists developed by people within the particular field. These lists are well known and frequently cited within the specific field, but are generally not well known outside of it.

MedHunt at the Health On the Net Foundation
(http://www.hon.ch/MedHunt/)

The Health On the Net Foundation is a not-for-profit organization headquartered in Geneva, Switzerland that helps us find health-related information using MedHunt. The search is processed in two separate databases: (1) an automatically indexed database where information is retrieved using the Marvin health sites retrieval robot (2) and the Health on the Net's "HONoured" database that manually catalogues and provides short descriptions of numerous health-related sites. The Health On the Net Foundation's Homepage provides a link to a Search Assistant that also allows us to search the HONoured database by anatomy and some diseases. It further provides access to a MeSH browser and the Media Gallery.

Medical World Search
(http://www.mwsearch.com)

Medical World Search incorporates the Unified Medical Language System (UMLS) Metathesaurus, from the National Library of Medicine, to perform MeSH term searches on the Internet. The use of Booleans and information available regarding synonyms, definitions, and the structure of relevant MeSH trees help us perform relatively specific Internet searches. The Medical World Search staff and collaborators' index sites using a selection process to enhance the quality of information retrieved. While the search process is relatively specific, it is also comprehensive, allowing users to search online medical sites such as the Merck Manual, Oncolink, Medscape, and the Virtual Hospital, as well as specific health directories at Alta Vista, Infoseek, HotBot, WebCrawler, or Healthgate's MEDLINE.

Medscape
(http://www.medscape.com/)

Designed for health care professionals, Medscape provides a compendium of medical resources for obtaining health-related information. Registration is required but free. Medscape pools information from clinical journals, electronic books, medical news providers, medical education programs, and material created expressly for use at this site. Current and archived articles, organized by medical specialty or clinical topic, may also be searched by keywords. This site further provides useful pharmaceutical information through the National Drug Data File at First DataBank, access to the Merriam Webster's Medical Dictionary, and a catalogue of patient resources.

Office of Rare Diseases
(http://cancernet.nci.nih.gov/ord/index.html)

This National Institutes of Health site is useful for both clinician and patient. It helps us find information on more than 6,000 rare diseases, including current research, publications from scientific and medical journals, completed research, ongoing studies, and patient support groups. The "Help and More" link leads to "Search Strategies," which assists us in retrieving the information we need.

If we still have not found the information we require, we may explore the vast expanse of the World Wide Web by searching one of the general search engines (such as presented in Table 1 below). Unlike medical specific Internet search engines, like Medical World Search and CliniWeb mentioned in the step above, the search engines in this step are far less discriminating in the sites they index. This means we should expect to find a lot of "noise", (ie, an abundance of low-grade information that tends to obscure the true gems we seek).

Search engines usually take one of two approaches to deal with the noise. They may index the full text of a large number of sites that may be retrieved using "word statistics," which leaves the user to filter out noisy results. Alta Vista, Lycos, and Infoseek are examples of such systems. While likely sensitive, this approach lacks specificity for high quality information. The other approach is to choose the major sites manually, index them with specific descriptions and search or browse over this filtered selection, instead of searching the full text. Yahoo! is an example of such an engine. Its approach most likely improves the specificity of a search while sacrificing some degree of sensitivity.

Most search engines enable us to perform keyword, title word and phrase searches (the latter indicated by surrounding quotation marks) Booleans, such as "and," "or," "not," or "+" and "-" signs may be used to refine our search. Many engines also allow the insertion of an asterisk that acts as a wildcard. For example, "anchor*" retrieves anchors, anchored, anchoring, etc. Information explaining the specific commands may be found in the particular search engine's help files. The Spider's Apprentice is also an extremely helpful site. It presents an overview and strategy towards Web-based searching, and a detailed analysis elucidating how each of the seven most popular search engines work. The site also grades each engine and indicates the best one to use when one has any one of a number of outcomes in mind.

If we do not retrieve the information we need with the general search engines, or, if the information we require is fairly obscure, or, if we just want to view the results from many different search engines all at once, we use metasearch engines. As their name implies, metasearch engines retrieve information by searching multiple engines simultaneously. Table 2 presents a few highly rated metasearch engines. You may find more information about these and other engines like them at SearchIQ.

Whichever engine we decide to use, we should be aware that we have entered cyberspace, a place for which no government, institution or organization is responsible. With no oversight, Web page content ranges from interesting to irrelevant, and opinions and anecdotes look just as official as well thought-out reviews and even journal articles. Even Web pages at major university teaching hospitals have not necessarily been evaluated for content by any academic or administrative body. Much of what is opinion is not always labeled as such. However, many sites do clarify their source of information. Some highlight this by indicating they abide by the Health On the Net Foundation Code of Conduct (HONcode). While the HONcode does not rate the quality of information, it defines a set of rules intended to clarify the source of evidence on the Web site. Developing critical appraisal skills using approaches such as those outlined in Chapters 8-10 of Evidence-Based Medicine: A Framework for Clinical Practice, or the Users' Guide to the Medical Literature at McMaster University may help you better discern the quality of information you have retrieved.
 
Table 1. General Search Engines

 

 
Table 2. Metasearch Engines

 

 

2) AN APPROACH FOR KEEPING UP-TO-DATE

The overall strategy for using the Internet to keep current follows that outlined in Chapter 7 of Evidence-Based Medicine: A Framework for Clinical Practice. The clinician is encouraged to read the abstracts of a select few well respected, peer-reviewed journals in the specific field of interest, further expand the scope of surveillance by reading pre-reviewed summaries of articles culled from many journals, and read medical updates for the latest developments reported by a variety of news groups. In addition, the clinician may deepen any specific areas of knowledge by reading a reliable up-to-date review, guideline, or consensus statement (as mentioned in section 1a above).

Many of the major medical journals have Web sites. Most provide free access to the study abstracts; a few also provide free access to the full text and figures (otherwise usually accessible to you for a subscription fee). They can often be found using search engines and the name of the journal, but if your journal has an electronic site, it will almost certainly be found at e.journal (http://www.edoc.com/vjo.htm). Because these sites are updated on the day of or before publication, information may be available here before the articles are indexed in MEDLINE. Most good sites have their own search engines with which you can search their electronic libraries.

You might consider scanning two or three journals within your particular field of interest and one or two general medical journals. The following are examples of premiere journals that broadly cover medicine. We suggest you bookmark any additional journals you plan to scan regularly.

Annals of Internal Medicine
(http://www.acponline.org)

British Medical Journal (BMJ)
(http://www.bmj.com)

Journal of the American Medical Association (JAMA)
(http://www.ama-assn.org)

The Lancet
(http://www.thelancet.com)

 The New England Journal of Medicine
(http://www.nejm.org)

Reading pre-reviewed summaries from publications that have scanned many journals for key articles can be a very efficient use of time for the busy clinician to broadly review the medical literature. Three publications are particularly helpful:

ACP Journal Club
(http://www.acponline.org/journals/acpjc/jcmenu.htm?wni)

Published bimonthly by the American College of Physicians, this resource abstracts key articles from a number of "core" journals focused on internal medicine (American Journal of Medicine, Annals of Internal Medicine, Archives of Internal Medicine, BMJ, Circulation, Diabetes Care, JAMA, Journal of General Internal Medicine, Journal of Internal Medicine, Lancet, and New England Journal of Medicine). Articles are selected according to their perceived importance and rigor as judged by specific criteria based on the User's Guides to the Medical Literature. Each article summarized is reviewed by an expert in the content area. A commentary is added to describe any important methodological problems and provide the context of previous knowledge as well as recommendations for the clinical application of the study findings. While only a select group of reviews from this bimonthly publication is available online for free, the entire list of reviews is available with payment of a subscription fee.

Evidence-Based Medicine
(http://www.acponline.org/journals/ebm/ebmmenu.htm?wni)

A close relative of ACP Journal Club, Evidence-Based Medicine is generated using identical procedures, is published bimonthly, and also requires a subscription fee. This resource, however, has a broader focus, reviewing more than 100 journals covering family practice, surgery, psychiatry, pediatrics, and obstetrics and gynecology. In addition, Evidence-Based Medicine publishes approximately half of the abstracts reviewed in ACP Journal Club. (The abstracts not published will be listed, by their declarative titles, in the section titled Additional Articles Abstracted in ACP Journal Club.)

Journal Watch
(http://www.jwatch.org)

Journal Watch began in 1987 in both an online and newsletter format. An eleven-member physician editorial board prepares concise summaries of the most important research appearing in fifty journals covering medical literature across all specialties. A subscription fee is required.

While news updates are by no means rigorous sources of evidence-based information, they provide easily accessible updates of the latest breaking developments in medicine. This information should not serve as the basis for any clinical practice, but rather may point the clinician in the direction of the particular study, review, or guideline on which the news may have been based. Only critical appraisal of these primary resources allows the clinician to truly consider whether the finding or recommendation is applicable to patient care.

Sources of news updates may be found on the home page of many medical associations or organizations, such as the American College of Physicians and the American Medical Association. Also, most of the search engine home pages (in the Table 1 above) have specific sections devoted to "Health News." Yahoo! even allows you to set up a news ticker in the taskbar, so that you can see all the latest news, tailored to your interests, right on the desktop.

A number of other sites also provide informative medical news updates:

Doctor's Guide to the Internet
(http://www.docguide.com/)

This site has daily news items related to health, a weekly email update program, information on new drugs or new indications, a thorough list of medical conference information, and a comprehensive list of links to medical Web sites.

Health On the Net Foundation
(http://www.hon.ch/)

This site, mentioned in 1c above, also provides a library with daily news and a list of conferences and events.

InteliHealth
(http://ipn.intelihealth.com/ipn/)

This joint venture between Aetna U.S. Healthcare and the Johns Hopkins University Hospital and Health System provides a compendium of medical resources including daily news items and customized email updates. In addition, under "Health Resources" we find free excerpts from Journal Watch and free access to the U.S. Pharmacopeia database.

Medscape
(http://www.medscape.com/)

This site, mentioned in 1c above, also provides email updates, daily medical news items, summaries from the latest medical conferences, links to newly published clinical practice guidelines, treatment updates, and even free online CME examinations and credits.

 

3) EVIDENCE-BASED MEDICINE RESOURCE SITES

The principle steps of Evidence-Based Medicine (EBM) are:

• Framing the clinical question
• Retrieving the relevant information
• Critically appraising the relevant information
• Applying the relevant information to the care of the patient

Evidence-Based Medicine: A Framework for Clinical Practice uses medical decision-making techniques to both help frame the clinical question and apply the relevant information. It uses the "Search Strategy," as described above, to search for the required information, which is then critically appraised using two novel guides to evaluate either studies or integrative literature.

It is important to recognize that EBM is a global movement and an evolving process; there is no single "correct" strategy to perform each of its steps. Many groups have different approaches to frame the clinical question, and to retrieve, appraise, and apply the relevant information. Something can be learned from each different approach. Ultimately, each clinician's EBM skill will develop as a synthesis of these many approaches.

Over the last three years there has been a dramatic increase in the number of Web sites devoted to EBM. Probably the most thorough compilation of these sites is available on the Netting the Evidence Web page at the School of Health and Related Research (ScHARR), University of Sheffield, England. This extensive list provides an abundance of resources that helps refine our skills in performing each individual step of EBM. You may also find the email discussion list helpful for finding additional information relating to evidence-based health care.
 

 

4) SUMMARY TABLES