INTRODUCTION
Evidence-based healthcare involves solving health problems by basing decisions on the best available evidence 1 . Clinical Guidelines (CG) are based on systematically developed recommendations to mediate among health policies, best practices, government funding, local contexts, and patient choice 2 , 3 . Thus, adopting CG helps qualify professional practices at a time when evidence-based health care has become consolidated as a standard to guide both continuing education and care 4 . The World Health Organization (WHO) recommends the development of national CG as one of the main strategies for promoting the rational use of medicines, with scientific guidance on diagnoses and treatments 5 .
The quality of guidelines is decisive regarding the potential benefits provided by their use 4 , 6 . Key attributes of high-quality guidelines include validation, reliability, reproducibility, clinical applicability, adaptation to the healthcare setting, clarity, multidisciplinarity, evidence review, and documentation 6 . Guideline quality is highly variable, often failing to meet basic methodological standards, and thereby discrediting a mechanism that is fundamental to the implementation of evidence-based health care 2 , 4 , 6 . If numerous guidelines with different methods and objectives are developed for the care of the same disease, they can create unnecessary competition and a complex system of conflicting practices and interventions 4 , 7 . Inadequate methodologies and inconsistent strategies in the CG development process can hinder the implementation of recommendations 2 , 4 . This is particularly critical for conditions or diseases of considerable frequency and incidence that trigger organic reactions and tend to occur without harming the patient, including some infectious diseases such as herpes labialis 8 . Cold sores are a common, highly contagious infectious disease that affects the orolabial region (most commonly the outer part of the lower lip), caused by Herpes Simplex Virus Type 1 (HSV-1), and less frequently, Herpes Simplex Virus Type 2 (HSV -2) 9 , 10 . It is estimated that about 90% of the world population is exposed to HSV-1 11 , 12 . The incidence of HSV is 70% to 80% in populations with low socioeconomic status, and 40% to 60% in those with higher status 11-14 . Herpes labialis cases have increased globally over the past two decades, with over 23 million new cases per year, becoming a significant public health problem, with 15 to 40% of the population experiencing recurrent symptomatic outbreaks12,14-16. Most people with recurrent HSV-1 infection have fewer than two episodes per year, but 5% to 10% of affected people have at least six recurrences per year 9 , 10 , 14 , 16 . HSV-1 recurrences appear to be precipitated by several factors that can compromise an individual’s immune status, including prolonged exposure to ultraviolet (UV) light, use of immunosuppressive medications, HIV infection, stress, premenstrual tension, and surgery 10 , 13 , 14 , 16 . Some researchers claim that a diet rich in arginine-containing foods (e.g., chocolate, nuts and seeds) may be associated with the reactivation of herpes labialis 17-19 .
Cold sores, a public health concern inherently linked to sexually transmitted infections (STI), constitute an incurable condition, requiring specific approaches for reducing complications
and managing symptoms 1 , 9 , 13 , 14 , 16 , 20 . The relevance
of cold sores as a public health issue underscores the urgent need for continuous attention and education for proper prevention and management. The interconnection between cold sores and STI emphasizes the importance of comprehensive strategies encompassing prevention, diagnosis and treatment. It is crucial to recognize that, given their recurrent nature and high contagiousness, cold sores require essential therapeutic recommendations, especially considering the significant increase in HSV infections12,13,16-21.
The scarcity of studies reviewing clinical recommendations for cold sore management underscores the need for additional research. There is a clear need for continuous improvement of patient care based on systematic, reliable, unbiased information 2 , 3 . It is therefore crucial to adopt a comprehensive approach that considers public health perspectives to produce effective therapeutic guidelines to optimize cold sore management in the context of STI 10-14 . This study evaluated the methodological quality of clinical guidelines for the care of the population affected by fever blisters, checking their compliance with evidence-based health standards and guidelines.
MATERIALS AND METHOD
Identification and selection of guidelines
A search was made for CG on labial herpes from June 2022 to March 2023, in the databases of the Medical Literature Analysis and Retrieval System Online (Medline), through Pubmed® (with filter for CG), Google Scholar, Biblioteca Virtual em Saúde do Brasil (BVS) (with filter for guidelines prepared by the Ministry of Health) and websites of institutions/professional categories, through the descriptors: “herpes simplex” or “herpes labialis” or “labial herpes” or “oral herpes” and their synonyms combined with Boolean operators, previously consulted on the websites Descriptors in Health Sciences (DeCS, from Brazil) 22 and Medical Subject Headings (MeSH, from MEDLINE-PubMed) 23 . Duplicate publications were excluded. There was no publication time restriction for identification in the databases consulted 13 , 20 . The search was performed by peers, and any publication on whose inclusion they disagreed was included, given the limited number of specific CG for labial herpes.
Quality was assessed in documents containing guidelines for the care of individuals with fever blisters without other comorbidities. CG with interventions based on specific treatments (e.g., laser) and special care for specific patients (such as athletes, transplant patients, pregnant women, and newborns with or without the presence of comorbidities) were not included.
Guideline Evaluation Tool
The quality of the selected CG was assessed using the AGREE Instrument ( Appraisal of Guidelines for Research & Evaluation ), 2nd edition. AGREE is an assessment tool developed from reviews of more than 100 selected guidelines independently evaluated by more than 200 reviewers from different countries 24-27 . It is used as part of a protocol for quality assessment of CG to improve healthcare by WHO and several technology assessment agencies around the world3,25-27. Its latest edition (AGREE II), used in this paper, contains 23 key items organized into six quality domains 22 . Four independent experts evaluated the Clinical Guidelines selected for this study according to the instructions in AGREE II 3 , 22 .
Data extraction, management, and evaluation
The AGREE II Instrument includes six quality domains: (i) scope and purpose; (ii) stakeholders; (iii) development rigor; (iv) clarity of presentation; (v) applicability; and (vi) editorial independence3,24-26. Data related to the six quality domains were collected and recorded on a score sheet with ratings from 1 (strongly disagree) to 7 (strongly agree) for each of the 23 items provided by the instrument 3 , 24 . Finally, a percentage of adequacy was calculated for each of the six domains, with values from 0% to 100%, in which the score obtained by each evaluator and the maximum possible score for the domain were used, following the suggestions in the AGREE II instrument 22 .
Although it is not suggested in the AGREE II instrument, the Kappa statistical test was used in the current study to analyze inter-rater agreement and avoid randomness or poor agreement 3 , 21 . The Kappa coefficient of agreement describes the agreement between two or more researchers performing a nominal or ordinal evaluation of the same sample. Kappa coefficients of moderate agreement (Kappa>0.4) were considered preferable for this type of study3,27-28. For the agreement analysis, the raters jointly decided that assessment scores of 1 and 2 would be considered “low,” scores of 3 to 5 would be “intermediate” and scores of 6 and 7 “high”.
The AGREE II instrument does not define a standard indicating whether the guideline should be recommended 3 , 26 . Thus, to make the evaluation less subjective in this regard, it was agreed in the current study that domain (iii) “developmental rigor” would be the primary standard for the overall guideline evaluation. Following the criteria suggested by other authors3,19-21, the evaluations defined 50% as the minimum score for “developmental rigor”. Thus, for a CG to be considered “recommended,” it should score above 50%. The CGs that scored between 30% and 50% were considered as “developmental rigor requiring modifications,” and any that scored less than 30% were considered as “not recommended” 3 .
RESULTS
Guideline Characteristics
The study included CG on labial herpes published in different countries. Initially, 168 publications were retrieved and screened by reading titles and abstracts. Application of the established inclusion and exclusion criteria led to the exclusion of 149. In the guideline pre-selection stage, a greater diversity of available publications on clinical management for treating and diagnosing genital herpes was observed, including an international guideline published by the WHO 29 . After the CG pre-selection and selection processes, 12 publications remained, which comprised the final sample in this study ( Fig. 1 , Table 1 ).
The most common type of recommendation in CG was treatment (62%), followed by recommendations on diagnosis (24%) and care (15%). Guidelines from the United States of America (USA), United Kingdom, Spain and Brazil agreed on the use of nucleoside antivirals (acyclovir, valacyclovir and famciclovir) as the drugs of choice in the treatment of herpes labialis ( Table 2 ). In most guidelines, the oral route of administration was recommended more than the topical route. There was no consensus in the guidelines on the recommendation of topical antiviral therapy. CG 4, 9 and 10 listed some adjuvant treatments, such as oral analgesics and topical anesthetics, based on clinical severity, cautioning to consider individual need 30-33 . The results of all primary studies and their combination characteristics addressed in the guidelines were summarized and systematized ( Table 3 ).
Evaluation of Guideline Quality: Overall Assessment
Evaluation of the 12 CG in this study produced quality scores (0 to 100) for each domain ( Table 4 ) . The effectiveness of the method used in this study is demonstrated by the accuracy of the AGREE II methodology in identifying the weaknesses of different types of guidelines in the evaluation of comprehensive issues necessary for any type of care guidance 26 .
Acronym | Selected guideline | Year of publication | Institution | Origin | Version | Population | Recommendation Rating System |
---|---|---|---|---|---|---|---|
CG 1 | Oral Herpes simplex | 2016 | NICE | UK | Updated Adult / Child | None | |
CG 2 | Herpes simplex infections | 2015 | PCDS | UK | Original Adult / Child | None | |
CG 3 | Management guidelines for herpes simplex | 2016 | HVA | International | Original Adult / Child | None | |
CG 4 | Oral herpes - Clinical Guideline | 2020 | MSF | International | Original Adult / Child | None | |
CG 5 | Recurrent Herpes Simplex Labialis: Selected Therapeutic Options | 2003 | JCDA | CA | Original Adult / Child | None | |
CG 6 | Common Oral Lesions: Part I. Superficial Mucosal Lesions | 2007 | AAFP | USA Original Adult SORT | |||
CG 7 | Congenital Herpes Simplex Virus | 2010 | AAFP | USA Original Adult SORT | |||
CG 8 | Guidelines For The Management Of Community-Acquired Infections | 2013 | NHS | UK Original Adult / Child CEBM | |||
CG 9 | Management of infection guidance for primary care for consultation &local adaptation | 2014 | PHE | UK Original Adult / Child CEBM | |||
CG 10 | Pharmaceutical Indication Protocols and Derivation Criteria for Minor Symptoms | 2008 | GFS | ES | Original Adult / Child | None | |
CG 11 | Infectious and Parasitic Diseases | 2010 | MH | BR | Original Adult / Child | None | |
CG 12 | Dermatology In Primary Health Care | 2002 | MH | BR | Original Adult / Child | None |
There was a lack of transparency and methodological rigor in producing most of the CG, which made them inconsistent in reliability and reproducibility, and in turn may compromise decision-making by health professionals 30-31 . Only two of the 12 CG evaluated (CG 1 and CG 5) declared conflict of interest. There is reason to believe that the production, dissemination and application of evidence in providing healthcare may often be influenced by conflicts of interest, especially financial 26 .
Two of the 12 CG evaluated, “Oral Herpes simplex” (CG 1) and “Infection management guidance for primary care for consultation and local adaptation” (CG 9), were considered “recommended”, one, “Non-genital Herpes Simplex Virus” (CG 7) was considered “recommended with modifications”, and nine (CG 2-6,8,10-12) were considered “not recommended”.
Inter-rater agreement square-weighted Kappa was 0.62, reflecting substantial agreement3,27-28.
Evaluation of Guideline Quality: Domain 1 -Scope and Purpose
This was the domain with the highest scores. “Oral Herpes Simplex” (CG 1) had the highest score (93%). It defines the target population, actions and health context using an objective scientific essay of questions answered with inclusión and exclusión criteria and is well updated 29 .
Clinical Guidelines | Pharmacological Treatment | Non-pharmacological treatment | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
Acyclovir | Valacyclovir | Famciclovir | Penciclovir | Docosanol | Chlorhexidine solution | Hydrogen Peroxide | Copper Sulfate | Vaseline | Povidone iodine | |
CG 1 | YES | YES | YES | - | - | |||||
CG 2 | YES | YES | YES | - | - | - | - | - | - | |
CG 3 | YES | YES | YES | - | - | - | - | - | - | |
CG 4 | YES | - | - | YES | - | YES | - | - | - | YES |
CG 5 | YES | YES | YES | YES | - | - | - | - | - | |
CG 6 | YES | YES | YES | - | - | |||||
CG 7 | YES | YES | YES | YES | YES | - | - | - | - | - |
CG 8 | YES | - | YES | - | - | - | - | - | ||
CG 9 | - | - | - | - | - | YES | YES | - | - | |
CG 10 | YES | - | - | - | - | - | - | YES | YES | |
CG 11 | YES | YES | - | - | - | - | - | - | - | |
CG12 | YES | YES | - | YES | - | - | - | - | - |
Acronym | Scope and Purpose | Stakeholder involvement | Rigor of development | Clarity of presentation | Applicability | Editorial independence |
---|---|---|---|---|---|---|
CG 1 | 93% | 83% | 91% | 85% | 82% | |
CG 2 | 39% | 7% | 1% | 35% | 2% | |
CG 3 | 46% | 13% | 3% | 51% | 10% | |
CG 4 | 63% | 24% | 5% | 47% | 2% | |
CG 5 | 75% | 39% | 21% | 78% | 11% | |
CG 6 | 44% | 28% | 28% | 71% | 4% | |
CG 7 | 57% | 31% | 36% | 78% | 39% | |
CG 8 | 64% | 19% | 16% | 32% | 13% | |
CG 9 | 82% | 64% | 61% | 56% | 15% | |
CG 10 | 86% | 64% | 17% | 60% | 17% | |
CG 11 | 83% | 46% | 15% | 51% | 21% | |
CG 12 | 79% | 43% | 6% | 61% | 11% |
Study Origin | Type of design | Interventions | Outcomes | Adverse effects | Conclusions | |
---|---|---|---|---|---|---|
Rooney JF et al. 34 | USA | RCT, doubleblind, placebocontrolled, crossover (n = 22). | Acyclovir 400 mg (twice daily) orally or placebo for four months. | 1. Number of relapses per patient. 2. Number of relapses with positive HSV culture per patient. | Not reported. | Treatment with acyclovir 400 mg (twice daily) resulted in a 53% reduction in clinical relapses and 71% in HSV-positive culture compared to placebo therapy. |
Baker et al. 35 | USA | RCT, doubleblind, placebocontrolled (n = 311) | Valacyclovir 500 mg (once daily) orally or placebo for four months. | 1. Number of relapses per patient. 2. Average time to first relapse. | The most common adverse effect in both groups was a headache, reported five times among three patients in the valacyclovir group and twice in the placebo group. | Valacyclovir 500 mg orally once daily for four months is effective and well tolerated to prevent recurrent herpes labialis. |
Spraunce et al. 36 | USA | RCT, doubleblind, placebocontrolled, multicenter (n = 49) | Valacyclovir 500 mg (once daily) orally or placebo for four months. | 1. Time to healing of the lesion. 2. Time to resolve pain and/or discomfort. | Not reported. | The time to heal the lesion and resolve pain and/or discomfort was statistically reduced with valacyclovir compared to placebo. |
Spraunce et al. 37 | USA | RCT, doubleblind, placebocontrolled (n = 701) | Famciclovir 1500 mg (once daily) or 750 mg (twice daily) for one day or placebo within 1 hour after the onset of prodromal symptoms of an episode of herpes labialis. | 1.Healing time of the lesion. | Not reported. | The single dose of famciclovir reduced the healing time of herpes labialis lesions by approximately two days compared to placebo. |
Raborn et al. 38 | USA, UK, and NA | RCT, doubleblind, placebocontrolled, multicenter (n = 4,273) | Penciclovir (1%) cream or placebo applied topically (six times a day) during the first day and every 2 hours for four consecutive days. | 1.Healing time of the lesion. 2. Time to resolve pain and/or discomfort. | Not reported. | Penciclovir cream significantly outperformed placebo in healing classic lesions and resolving the pain of recurrent herpes labialis. The effectiveness was apparent if the therapy was started “early” (stage of prodrome injury or erythema). |
Spraunce et al. 39 | USA | RCT, doubleblind, placebocontrolled, multicenter (n = 699) | Acyclovir (5%) cream or placebo topically (five times a day) for four days, starting within 1 hour of the beginning of a recurrent episode. | 1.Healing time of the lesion. 2. Time to resolve pain and/or discomfort. | Adverse effects were mild and uncommon. | Acyclovir cream did not prevent the development of classic lesions (progression to vesicles, ulcers and/or crusts). |
Sacks et al. 40 | USA | RCT, doubleblind, placebocontrolled, multicenter (n = 370) | Docosanol (10%) cream or placebo (five times a day) until complete healing of the lesions. | 1.Healing time of the lesion. 2. Time to resolve pain and/or discomfort. | Not reported. | Docosanol applied five times a day is safe and effective in treating recurrent herpes labialis. |
Rahimi et al. 41 | USA | Systematic review and meta-analysis | Topical and systemic antivirals | Prevention of recurrent herpes labialis. | Report of pain and nausea as the only adverse effects in 5% of patients using systemic antivirals. | Using acyclovir and systemic valacyclovir is safe and effective in preventing recurrent herpes labialis. |
Evaluation of Guideline Quality: Domain 2 -Stakeholder Engagement
Scores in this domain varied widely across guidelines (7%-83%). Overall, the evaluation identified two main weaknesses: little use of collaborative multidisciplinary practices, and lack of investigation of patients’ opinions and preferences.
The guideline “Herpes Simplex Infections” (CG 02) had the lowest quality score in the domain (7%) because it was prepared by only one dermatologist, with no report of other health professionals or methodologists collaborating by supervising the search and analyzing evidence. Other CGs were authored and reviewed by infectious disease physicians, dermatologists, and epidemiologists, but did not include non-medical health professionals such as dentists, pharmacists and nurses.
Among the 12 guidelines evaluated in this study, only “Oral herpes simplex” (CG 1) reported active patient participation in its development through information obtained from the literature review on patients’ experiences and individual and group consultations of stakeholders. Neither of the two guidelines evaluated by the Brazilian Ministry of Health (MS) was previously reviewed by the National Committee for the Incorporation of Technologies (CONITEC) or submitted for public consultation.
Evaluation of Guideline Quality: Domain 3 -Development Rigor
This domain was one of the most significantly divergent from the quality standards assessed. Scores were lower than 36% in all guidelines except CG 1 and CG 9, which scored higher than 61%. Among the 12 guidelines evaluated, “Herpes simplex infections” (CG 02) had the lowest score in this domain (1%). It was found that most of the guidelines evaluated neglected some critical issues for quality assessment, such as a complete description of search methods, selection criteria and evidence, strengths and limitations of the evidence, consistency between the formulation of recommendations and evidence, consideration of benefits and drawbacks, and external review criteria for updating the guideline.
Evaluation of Guideline Quality: Domain 4 -Clarity of presentation
This was the domain with the second-highest scores. In general, all the evaluated CG were written in simple language with descriptions of the recommendations typed in bold or italics, presented in topics, flowcharts and/or summary tables.
Evaluation of Guideline Quality: Domain 5 -Applicability
This domain scored low in all the guidelines, with scores below 39%, except for “Oral herpes simplex” (CG 1), which scored 82%.
Evaluation of Guideline Quality: Domain 6 -Editorial independence
Of all the domains, this one diverged the most from the quality standards established by AGREE II.
All guidelines scored 0%, except for “Oral Herpes Simplex” (CG 1) and “Recurrent Herpes Simplex Labialis: Selected Therapeutic Options” (CG 5), which scored 100% and 75%, respectively.
Evaluation of Guideline Quality: International Comparison
In order to analyze the results from an international point of view, the quality scores per domain were compared to the findings of a systematic review of studies from around the world that gathered 625 different guidelines published since 2003 34 . In general, CG for herpes labialis were not precisely aligned with international standards, as they had lower quality scores in all domains assessed by AGREE II. The main discrepancies were in the domains ‘developmental rigor’ and ‘editorial independence’. reinforcing the findings that these documents had problems of methodological rigor and transparency, since the methodological criteria were questions about evidence-based health, and the procedures did not describe an unbiased process 26-31 .
Strength of recommendation and level of evidence of clinical guidelines for herpes labialis
In general, among the 12 CG for fever blisters, there was no grading of the level of evidence underlying the 83 recommendations identified. The strength of the recommendation rating system was covered by only four guidelines according to the SORT (Strength of Recommendation Taxonomy) and CEBM (Center for Evidence-Based Medicine) criteria. Only eight of the 83 recommendations were assigned a recommendation strength. Approximately 10% were based on recommendation strengths A and B (3.6% and 6.0%, respectively). The other recommendations were not ranked simply because the guidelines no longer use a recommendation ranking system.
DISCUSSION
Effective implementation of the recommendations requires the adoption of consistent methodologies in the development process 3 , 4 , 6 . There is reason to believe that methodological information may be challenging to find because it appears in separate documents or appendices not specified in the CG. In the current study, relevant literature may have been overlooked since review procedures, including previous publications, do not work in many of the evaluated CG. The developmental approach suggested presenting a rigorous evaluation by the authors and supervisors of the CG in examining the clinical conduct and transparency of the scientific evidence of the herpes labialis recommendations 4 , 26 . There is often a normative character in the applicability/implementation of CG 42 , 43 . This study revealed that most of the evaluated CG neglected to describe aspects of the potential implications for resources arising from the recommendations, validation procedures, follow-up criteria, and suggestions on how the recommendations can be put into practice. Furthermore, the evaluated guidelines showed a lack of a systematic, transparent approach to make judgments about the quality of evidence and the strength of recommendations in most of the unrated recommendations, which not only may impair care guidance for health professionals, population, and managers, but may also result in low reliability of the recommendations 1 , 4 , 43 . Regarding treatment, nucleoside antivirals (acyclovir, valacyclovir, and famciclovir), the main class of drugs recommended in CG, are synthetic analogs of acyclic purine (or guanine analogs), highly specific substrates for viral thymidine kinase and effective inhibitors of deoxyribonucleic polymerase (DNA) against HSV-1 and HSV-2 9 . It was also observed that few CG provided specific recommendations for diagnosing and screening labial herpes, which were grouped for cases of HSV infection. This promotes gaps in clinical relevance regarding differential diagnosis, risk stratification, and evolution of the patient with labial herpes. It is thus useful to have a classification that indicates the confidence level of the evidence quickly and practically. The results of the current study show that CG with adaptations of reliable graded recommendations on specific treatment and care for the management of fever blister are essential to fill the knowledge gaps 44-46 .
Regarding non-pharmacological measures, the development of CG for fever blisters could prioritize greater reliability of the scientific evidence of the recommendations provided to ensure greater methodological rigor, transparency, a classification system, and the participation of more oral health professionals and patients with the disorder 15 . Furthermore, it was found that 10 of the total CG reviewed provided no clear, certified statement demonstrating that the opinions or interests of funders or competing bodies did not influence the final recommendations. “Oral herpes simplex” (CG 1) was the only guideline that fully contemplated supplementary documents and terms that defined procedures and responsibilities for each stakeholder group and included declarations of no conflict of interest signed by all parties. CG should follow this procedure to contribute to unbiased final recommendations and authoritative publications3,28,30-32.
The analysis revealed that the healthcare setting in most initial descriptions of CG for fever blisters was primary healthcare30-33,37. Thus, patients seeking primary healthcare services are more likely to receive the actions recommended in the guidelines in that setting 43 .
Including other health professionals such as dental surgeons and pharmacists in the CG development team may contribute not only to greater integration of professionals in the evidence review’ but also to the diversity of target users qualified for caring for patients with the disorder 6 . It is essential to specify the professionals involved, as there is global difficulty due to issues related to the closed structure of the Health Technology Assessment (HTA) area and the suspicion that CG developers may be influenced by pharmaceutical companies 3 , 31 .
It has been observed that including patients in developing the guidelines is also crucial to improving recommendation quality. According to
Van et al. 37 and Cluzeau et al. 38 , doing so enables identification of points of disagreement between professionals and patients, priority needs from the users’ perspective, and aspects that are not well observed by guideline supervisors and can help improve guideline adherence and implementation strategies 44-46 .
One of the limitations of this study is the number of CG used, based on the inclusion and exclusion criteria. The study was nevertheless performed, and the results should be considered because they can be used to support the scientific community in developing and updating CG for labial herpes. Any further studies should use different tools to analyze the CG so that they can be compared to the results obtained in the current study.
The findings showed that there is consensus regarding the treatment of fever blister herpes using nucleoside antivirals - acyclovir, valacyclovir and famciclovir - as the drugs of choice. However, there are limitations in the vast majority of CG, especially concerning the methodology for diagnosis and screening. A multidisciplinary team should be involved in the preparation of CG for labial herpes, to ensure complementarity among the health knowledge areas. All recommended parameters for preparing CG should be followed in order to avoid publications of low quality and contradictions among the different CG that address the same subject.