Search Article 
Advanced search 
Official publication of the American Biodontics Society and the Center for Research and Education in Technology
Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 

 Table of Contents  
Year : 2015  |  Volume : 6  |  Issue : 4  |  Page : 134-140

Implications and applications of systematic reviews for evidence-based dentistry and comparative effectiveness research: A sample study on antibiotics for oro-facial cellulitis treatment

1 Student, Division of Oral Biology and Medicine, Section of Oral Biology, UCLA School of Dentistry, Los Angeles, California, USA
2 Student, Division of Oral Biology and Medicine, Section of Oral Biology; Evidence-Based Decision Practice-Based Research Network, UCLA School of Dentistry, Los Angeles, California, USA

Date of Web Publication27-Nov-2015

Correspondence Address:
Francesco Chiappelli
School of Dentistry, Division of Oral Biology and Medicine, UCLA Center for the Health Sciences 63-090, 10833 Le Conte Avenue, Los Angeles - 90095-1668, California
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/2155-8213.170639

Rights and Permissions

Introduction: Comparative effectiveness and efficacy research for analysis and practice (CEERAP) was performed to assess the effects of penicillin-based versus erythromycin-based antibiotic treatments in patients with skin and soft tissue infections (SSTIs) including cellulitis, impetigo, and erysipelas. Because SSTIs, especially orofacial cellulitis, are volatile infectious diseases of a life-threatening nature, research on the most efficacious remedies is necessary. Methods: The stringent bibliome yielded three systematic reviews, which were examined for quality of research synthesis protocol and clinical relevance. Results: The sample size of three, rendered the statistical analyses and cumulative meta-analysis problematic. Conclusion: The systematic review outlined here should aid in increasing clinical awareness, improving patient health literacy, and promoting consensus of the best evidence base (BEB) to mitigate the threat of sepsis and potential death caused by cellulitis infections.

Keywords: Best evidence base (BEB), comparative effectiveness and efficacy research for analysis and practice (CEERAP), erythromycin, evidence-based dentistry (EBD), orofacial cellulitis, penicillin, skin and soft tissue infections (SSTIs), systematic review

How to cite this article:
Bach Q, Kasar V, Chiappelli F. Implications and applications of systematic reviews for evidence-based dentistry and comparative effectiveness research: A sample study on antibiotics for oro-facial cellulitis treatment. Dent Hypotheses 2015;6:134-40

How to cite this URL:
Bach Q, Kasar V, Chiappelli F. Implications and applications of systematic reviews for evidence-based dentistry and comparative effectiveness research: A sample study on antibiotics for oro-facial cellulitis treatment. Dent Hypotheses [serial online] 2015 [cited 2023 Mar 22];6:134-40. Available from:

  Introduction Top

Rarely do conditions of a dental nature require urgent attention; cellulitis is one of these rare conditions. Cellulitis is a skin and soft tissue infection (SSTI) that presents as diffuse, erythematous swelling, resulting from the spread of infectious bacteria - commonly Streptococcus and Stapholococcus - across epidermal, dermal, and subcutaneous tissues. [1] Orofacial cellulitis, in particular, can have origins from dental surgeries such as improperly performed root canal procedures or inadequate posttreatment care. These and other poor surgical outcomes may lead to cracks or cavities in the tooth that result in the formation of a dental abscess, a pocket of pus at the root tip. [2] If untreated, this abscess can ultimately lead to the spread of infection to the gums, the maxillofacial tissue, and throughout the body, inducing systemic effects. [2] The infection can also spread to other regions of the patient's body and cause a variety of diseases such as gangrene, meningitis, lymphangitis, and Ludwig's angina. [3],[4] Ludwig's angina is a type of orofacial cellulitis that causes swelling at the lymph nodes; this requires immediate attention as airways are at high risk of becoming obstructed, as is the case with most orofacial cellulitis infections. [3] Difficulty in swallowing (dysphagia) or pain during swallowing (odynophagia) are two of the principal symptoms of orofacial cellulitis that may indicate an urgent need for examination, draining, and subsequent antibiotic therapy. [5]

A disease of such volatile and life-threatening nature necessitates adequate research on the most effective and efficacious remedies; yet, there seems to be a gap in knowledge on this subject. This topic was researched to understand more about the remedies of SSTI diseases with the intent to ultimately inform clinicians and patients about the best available treatments and associated risks.

Comparative effectiveness and efficacy research for analysis and practice (CEERAP) seeks to obtain the consensus of the best evidence base (BEB) by means of the research synthesis design as the fundamental protocol of the systematic review. This approach was applied here to compare and contrast the effectiveness of penicillin-based versus erythromycin-based antibiotics in the treatment of cellulitis. In order to perform CEERAP, the quality of systematic reviews on the topic needed to be assessed by performing a "complex" systematic review, i.e., a systematic review of systematic reviews.

This study's Population, Intervention, Comparator, Outcome, Timeline, and Setting (PICOTS) question was drafted to examine specifically the following:

  • Antibiotic treatments administered orally in males and females of all ages 2 weeks after completion of the antibiotic regime.
  • Patients were diagnosed with either cellulitis or similar SSTIs, erysipelas and impetigo.
  • The clinical outcome of interest was the elimination of threat to health from sepsis caused by cellulitis, as measured by the reduced presence of laboratory markers of inflammation and infection.
  • The clinical outcome was measured 2 weeks after antibiotic treatment and in a hospital-dentistry setting.

In brief, the working hypothesis of the study stated that erythromycin-based antibiotics would be more efficacious as compared to penicillin-based antibiotics when administered orally in male and female patients of all ages, who had been clinically diagnosed with cellulitis, erysipelas, or impetigo. The results are measured in a hospital dentistry setting and assessed for the presence of laboratory markers of inflammation and infection 2 weeks after the completion of antibiotic treatment.

This paper aims to elucidate the process behind performing a systematic review. Understanding this protocol will then provide a foundation to perform a complex systematic review and CEERAP. Altogether, these steps provide the tools necessary to devise a clinical intervention that is evidence-based, effectiveness-focused, and patient-centered.


The systematic review process is delineated by the following steps:

  1. Preparing the topic.
    • Topic refinement.
    • Developing an analytic framework.
    • Search, bibliome, and publication bias.
  2. Study eligibility and relevance.
    • Identify study eligibility criteria.
    • Search for relevant studies.
    • Select evidence for inclusion.
  3. Extracting data from studies.
    • Grade strength of evidence: Expansion of the Grading of Recommendations Assessment, Development, and Evaluation (EX-GRADE), Revised Assessment for Multiple Systematic Reviews (R-AMSTAR), Risk of Bias, Appraisal of Guidelines and Research and Evaluation, Europe (AGREE).
  4. Analyzing and synthesizing studies.
    • Quantitative consensus.
    • Qualitative consensus.
  5. Reporting the systematic review.

  1. Preparing the topic.

    This step sets the stage for the entire systematic review and consists of three main components: Topic refinement, analytic frameworks as well as search, bibliome, and publication bias. After developing a topic based on its appropriateness, importance, and feasibility, the topic can be refined to a PICOTS question. Modeling these components graphically in an analytical framework can present the information in such a way that helps to visually link factors such as interventions and stakeholder investment to final outcomes, and everything in between [Figure 1]. [6] When later searching and refining the bibliome, that is, in the process of eliminating irrelevant evidence after examining all the available literature on a topic, an issue becomes apparent: A degree of publication bias exists. It is important to keep this bias in mind as the process continues, as it is an inevitable issue that may have an effect on the outcome. [7]
    Figure 1: Schematic representation of the systematic review process. The systematic review protocol involves 5 essential steps, each delineated by its own set of subprocesses and criteria

    Click here to view
  2. Study eligibility and relevance in pursuance of the bibliome.

    The PICOTS question derived in the previous step will enable the formulation of key questions, literature search, inclusion and exclusion criteria, and data extraction. When determining the eligibility criteria, one effectively controls how wide or narrow the scope of a systematic review will be. The criteria for study eligibility may include such factors as study design, studies in foreign languages, gray literature, and year of publication. After defining the eligibility criteria, the next step is to search for relevant studies, i.e., randomized controlled trials (RCTs) that are within the inclusion and exclusion criteria. To perform this literature search, a set of keywords or Medical Subject Headings (MeSH) words should be utilized with search engines that produce curated results such as PubMed and MEDLINE databases [Figure 2]. The overarching goal of selecting studies for eligibility and relevance is to reduce bias and increase the applicability of study results to the issue at hand.
    Figure 2: Framework of literature search. This figure shows the process by which the 3 resulting systematic reviews were chosen for qualitative analysis

    Click here to view
  3. Extracting data from studies.

    After selecting relevant studies, one must proceed to grade the strength of evidence extracted from these studies by using assessment tools. Grading often and ideally provides the researcher with a quantifiable value, which can subsequently function as an indicator of quality and/or completeness. Evaluating the strength of evidence is crucial to allow the use of papers from authors with a variety of perspectives, and to reach a consensus of the best available evidence base. Among the wide array of grading systems presently at the disposal of evidence-based dentistry researchers, the most recommended ones grade the strength of evidence based on the risk of bias, consistency, directness, and precision. [7] Many practical guidelines have been developed by our group, which include but are not limited to the following: EX-GRADE, [8] R-AMSTAR, [9] Risk of Bias, [7] and AGREE. [10] The steps of each assessment tool are described in further detail below.
    1. EX-GRADE is an expansion of the Grading of Recommendations Assessment, Development, and Evaluation. This assessment is based on eight questions each worth 4 points, which assesses the report based on the evidence provided and the strength of the recommendation made in the report. [8] The four domains in this assessment include:
      • Study design.
      • Study quality.
      • Consistency.
      • Directness.
    2. R-AMSTAR or Revised Assessment for Multiple Systematic Reviews provides an 11-item checklist to validate the content and methodological quality of the systematic reviews. This assessment quantifies systematic review quality with a score out of maximum 44, with 44 indicating the lowest possible risk of bias. [9] The R-AMSTAR considers 11 criteria:
      • A priori design.
      • Duplicate study selection and data extraction.
      • Comprehensive literature search.
      • Status of publication.
      • Inclusion and exclusion criteria.
      • Relevant characteristics of the population (age, sex, and socioeconomic data).
      • Scientific quality of the included studies.
      • Utilization of included studies in the formation of conclusions.
      • Statistical methods of combining data or conclusions.
      • Likelihood of publication bias.
      • Conflicts of interest.
    3. A risk of bias tool was created (Cochrane Risk of Bias), which covered six specific domains *:
      • Sequence generation.
      • Allocation concealment.
      • Blinding of participants and researchers.
      • Incomplete outcome data.
      • Selective outcome reporting.
      • Other sources of bias.

      * Within each domain, assessments of one or more of the following criteria were evaluated:
      • Selection bias.
      • Performance bias.
      • Detection bias.
      • Attrition bias.
      • Reporting bias.
      • Other research protocol-related biases. [7],[11]
    4. AGREE stands for Appraisal of Guidelines and Research and Evaluation, Europe * The AGREE grading tool utilizes a system of 23 criteria points, which are evaluated from points 1-4. The six main domains for this instrument are:
      • Scope and purpose.
      • Stakeholder involvement.
      • Rigor of development.
      • Clarity and presentation.
      • Applicability.
      • Editorial independence.
    *AGREE was revised to the AGREE II, which was expanded to include a 7-point scoring system instead of 4, and some of the criteria were improved upon to make better the power and sensitivity of the tool. [10]
  4. Analyzing and synthesizing studies.

    When the researcher has finished compiling evidence, he or she can analyze the results in two distinct ways: Quantitative or qualitative consensus. A meta-analysis is a statistical analysis designed to compare and combine results from different studies to identify patterns among the results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies. [12] In brief, meta-analysis provides valuable information to better understand alternative intervention efficacy, consistency of study effects, and statistical heterogeneity. [12] The key to quantitative consensus is reaching a quantifiable conclusion, which can provide valuable results after applying statistical analysis methods. By contrast, qualitative consensus can be used when obtaining a quantitative conclusion is not possible such as in situations of research designs with insufficient data and small sample sizes. The qualitative consensus method utilizes open-ended interview questions, contextual evidence, and consensus among the research group by taking in multiple opinions.

  5. Two qualitative consensus methods generally used include the Delphi process and the RAND expert panel:
    • The RAND expert panel is a multidisciplinary consensus panel, which combines both scientific evidence and clinical experience in order to create the best clinical procedure. The process begins by a thorough systematic review of the literature for the clinical procedure in question. A set of indications are created for that procedure and independently rated by nine expert panelists. The scores are discussed in a group in a face-to-face fashion, and rerated until a consensus is reached on the clinical procedure that is brought to a clinical setting. [13]
    • The Delphi process is another communication method, which utilizes a panel of experts. In this technique, the panelists are kept anonymous, even from one another, as they respond to multiple rounds of questionnaires. A moderator disseminates the responses in a statistical form. The overall goal of this method is to reach an expert consensus while minimizing the potential issues arising from group dynamics from a face-to-face discussion, as seen in the RAND expert panel. [8]
  6. Reporting the systematic review.

    The final step in performing an effective systematic review is reporting the findings. When reporting a comparative effectiveness review, a standard template should be used to ensure high clarity. The template should include an abstract, introduction, methods, results, and discussion. If a review is not reported accurately, readers may come across issues when evaluating the strength of evidence.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is the set of standards recently developed for the purpose of verifying the quality of meta-analyses and by extension, systematic reviews. The PRISMA 2009 Checklist of 27 criteria can be used to organize the information in a standardized, complete format. The checklist serves as a way to guarantee that all the information that stakeholders would need is present in the systematic review. The PRISMA Checklist currently exists as a binary (yes/no) scoring system. In the future, the strength of grading of the PRISMA Checklist can be expanded into a quantifiable assessment, e.g., with a grading rubric of 1-4 per item. By quantifying the PRISMA Checklist, authors and publishers can more accurately assess and improve the reliability of their reports before publication in a curated journal, therefore providing stakeholders with comprehensive systematic reviews, which they can then use to understand, evaluate, and construct clinical interventions. [14]

  Results Top

With this knowledge base of systematic reviews, the protocol described above was followed in order to perform a complex systematic review - a systematic review of systematic reviews - in order to obtain enough data to subsequently perform CEERAP. The extensive literature search generated a bibliome of three systematic reviews that fit within the selection criteria:

  • Interventions for cellulitis and erysipelas.
  • Systemic antibiotics for symptomatic apical periodontitis and acute apical abscess in adults.
  • Interventions for impetigo. [2],[15],[16]

Three readers calibrated for the uniformity of critical review [7],[8],[9],[10],[11] and graded the bibliome independently with R-AMSTAR. All three systematic reviews scored above 39.6 out of 44.0 (90%), qualifying them for inclusion within the acceptable sampling pool based on confidence intervals. Data extraction did not yield consistency of outcome measures across the systematic reviews in the bibliome, thus precluding data homogeneity and preventing cumulative meta-analysis.

Qualitative consensus of the bibliome led to the conclusion that most recommendations should not be considered reliable because outcomes within each systematic review were determined based on very few studies and compared different treatment subgroups, e.g., oral versus intravenous administration of antibiotics, antibiotic treatment with versus without surgical intervention, or antibiotic treatment with versus without analgesics.

  Discussion Top

The small bibliome available yielded inconclusive analyses in the comparative effectiveness research of the effects of erythromycin-based versus penicillin-based antibiotics in SSTI and oral cellulitis. Qualitative consensus by the Delphi and the RAND protocols were not stringent enough to establish a clinically relevant difference between oral versus intravenous administration of antibiotics, antibiotic treatment with or without surgical intervention, or antibiotic treatment with or without analgesics.

The search for the most effective SSTI antibiotic remedies continues. But in the meanwhile, the importance of identifying mechanisms to best disseminate and utilize research results should be discussed. Protocols can be implemented to do so, including the following [Figure 3]: [17]
Figure 3: Dissemination and Utilization of Research Results [Moura et al. 2013] The following mechanisms and protocols were discussed in relation to their potential to improve dissemination and utilization of the best available evidence

Click here to view

  • A RAND expert panel, which will define and promote the necessity to have timely and cost-effective clinical interventions. [8]
  • Health care institutions should have a committee in place that reviews the RAND expert panels clinical consensus and decides if it is financially feasible to enact the new procedure.
  • Improvement in the use of laboratory test results and patient communication via training with model of care used in medical schools. [17]
  • Improvement of patient literacy regarding pre- and postoperative procedures.
  • Training of clinicians to determine consensus using BEB.

Mechanisms that can be utilized for dissemination are as follows:

  • Media: If a change or clinical intervention is significant to the extent that more patients would benefit from being aware of it, the media should be alerted to disseminate the information via advertisements, news articles, or social health magazines.
  • Surveys assessing the following outcome measures can be implemented:
    1. Patient satisfaction.
    2. Health literacy.
    3. Incentive or motivation to.
      Go forth with treatment.
      Adhere to treatment specifications by the prescribing doctor.
      Complete the treatment.
      Incentive and motivation values could be determined and defined systematically by Weiner's attribution theory of motivation. [18]

These outcome measures should be determined quantitatively for the highest degree of effectiveness and applicability. Overall, patient-centered care is the most important goal of health care services. With the growing base of knowledge, it is as much essential to provide mechanisms to disseminate pertinent information as it is to seek these scientific findings to begin with.

  Conclusion Top

In conclusion, the intent of this study is to provide an insight and disseminate knowledge on a prevalent dental issue, orofacial cellulitis, because any emerging symptoms of cellulitis necessitate urgent care. These potentially fatal infections are avoidable by applying the protocols discussed above.

By delineating the steps to understand and perform a systematic review, more research of this nature may arise, allowing researchers and clinicians to one day reach a consensus on ongoing cellulitis and SSTI research. Moving forward with this research, clinical practitioners need to stress the importance of patient literacy and proper dental care, especially pre- and postoperative treatments, and a RAND expert panel should be consulted to develop an improved protocol against the threat of cellulitis from surgical procedures. If all of these steps to disseminate the BEB are taken, not only will patient care improve but so too will the subsequent actions taken by the patient to be compliant with a clinician's instructions - this will be a direct result of higher motivation due to enhanced patient literacy. [18] At the very core of translational science in dentistry, systematic reviews and CEERAP are fundamental to provide the best available evidence base and a method to compare that evidence to reach a conclusion that is evidence-based, effectiveness-focused, and patient-centered.

Abbreviations: Evidence-Based Dentistry (EBD); Comparative effectiveness and efficacy research for analysis and practice (CEERAP), skin and soft-tissue infections (SSTI); Population, Intervention, Comparator, Outcome, Timeline, Setting (PICOTS); Expansion of the Grading of Recommendations Assessment, Development, and Evaluation (Ex-GRADE); Revised Assessment for Multiple Systematic Reviews (R-AMSTAR); Appraisal of Guidelines and Research and Evaluation, Europe (AGREE).


The authors would like to thank the mentors, postdoctoral, graduate, and undergraduate predental students who have actively contributed toward the pursuance of evidence-based dentistry at the University of California, Los Angeles (UCLA) (Dr. Chiappelli's research group) over the years. The authors declare no conflict of interest.

Financial support and sponsorship


Conflicts of interest

Francesco Chiappelli has editorial involvement with Dent Hypotheses.

  References Top

Quirke M, Boland F, Fahey T, O′Sullivan R, Hill A, Stiell I, et al. Prevalence and predictors of initial oral antibiotic treatment failure in adult emergency department patients with cellulitis: A pilot study. BMJ Open 2015;5:e008150.   Back to cited text no. 1
Cope A, Francis N, Wood F, Mann MK, Chestnutt IG. Systemic antibiotics for symptomatic apical periodontitis and acute apical abscess in adults. Cochrane Database Syst Rev 2014;6:CD010136.   Back to cited text no. 2
Ehrlich SD. Cellulitis. University of Maryland Medical Center. University of Maryland School of Medicine, 26 May 2014. Available from: [Last accessed on 2015 Jul 15].   Back to cited text no. 3
Kusne S, Eibling D, Yu V, Fitz D, Johnson J, Kahl L, Ellis L. Gangrenous cellulitis associated with gram-negative bacilli in pancytopenic patients: Dilemma with respect to effective therapy. Am J Med 1988;85:490-4.  Back to cited text no. 4
Boscolo-Rizzo P, Da Mosto MC. Submandibular space infection: A potentially lethal infection. Int J Infect Dis 2009;13:327-33.  Back to cited text no. 5
The EHC Program Slide Library. (n.d.). Agency for Healthcare Research and Quality. Available from:   Back to cited text no. 6
Barkhordarian A, Pellionisz P, Dousti M, Lam V, Gleason L, Dousti M, et al. Assessment of risk of bias in translational science. J Transl Med 2013;11:184.  Back to cited text no. 7
Phi L, Ajaj R, Ramchandani MH, Brant XM, Oluwadara O, Polinovsky O, et al. Expanding the grading of recommendations assessment, development, and evaluation (Ex-GRADE) for evidence-based clinical recommendations: Validation study. Open Dent J 2010;6:31-40.  Back to cited text no. 8
Kung J, Chiappelli F, Cajulis OO, Avezova R, Kossan G, Chew L, et al. From systematic reviews to clinical recommendations for evidence-based health care: Validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J 2010;4:84-91.   Back to cited text no. 9
Chiappelli F. Fundamentals of Evidence-Based Health Care and Translational Science 2014. New York: Springer.  Back to cited text no. 10
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al.; Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration′s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.  Back to cited text no. 11
Greenland S, O′Rourke K. Meta-analysis, in modern epidemiology. Philadelphia: Lippincott Williams & Wilkins; 2008. p. 652-82.  Back to cited text no. 12
Coulter ID, Shekelle PG, Mootz R, Hansen DT. The use of expert panel results: The RAND panel for appropriateness of manipulation and mobilization of the cervical spine. Top Clin Chiropr 1995;2:54-62.  Back to cited text no. 13
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med 2009;6:e1000100.  Back to cited text no. 14
Kilburn SA, Featherstone P, Higgins B, Brindle R. Interventions for cellulitis and erysipelas. Cochrane Database Syst Rev 2010;CD004299.   Back to cited text no. 15
Koning S, van der Sande R, Verhagen AP, van Suijlekom-Smit LW, Morris AD, Butler CC, et al. Interventions for impetigo. Cochrane Database Syst Rev 2012;1:D003261.   Back to cited text no. 16
Moura Jde A, Costa BC, de Faria RM, Soares TF, Moura EP, Chiappelli F. Improving communication skill training in patient centered medical practice for enhancing rational use of laboratory tests: The core of bioinformation for leveraging stakeholder engagement in regulatory science. Bioinformation 2013;718-20.  Back to cited text no. 17
Bailey B, Hare D, Hatton C, Limb K. The response to challenging behaviour by care staff: Emotional responses, attributions of cause and observations of practice. J Intellect Disabil Res 2006;199-211.  Back to cited text no. 18


  [Figure 1], [Figure 2], [Figure 3]


Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article
Article Figures

 Article Access Statistics
    PDF Downloaded919    
    Comments [Add]    

Recommend this journal