This document was reviewed and approved by the Board of Governors of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Jul 2022.
Authors
Geoffrey P. Kohn1,2, Ziad T. Awad3, Mazen R. Al‑Mansour4, Jennifer Salluzzo5, Emily Miraflor6, Uretz Oliphant7, Bethany J. Slater8
1 Department of Surgery, Monash University Eastern Health Clinical School, Melbourne, Australia
2 Melbourne Upper GI Surgical Group, Cabrini Hospital, Malvern, Australia
3 Department of Surgery, University of Florida College of Medicine-Jacksonville, Jacksonville, FL, USA
4 Department of Surgery, University of Florida College of Medicine, Gainesville, FL, USA
5 Department of Surgery, Virginia Commonwealth University, Richmond, VA, USA
6 Department of Surgery, University of California San Francisco, East Bay, Oakland, USA
7 University of Illinois College of Medicine, Urbana, IL, USA
8 Department of Surgery, University of Chicago, Chicago, IL, USA
Abstract
Background: The SAGES Guidelines Committee has implemented processes for Quality Assessment of SAGES-endorsed guidelines, with the aim of improving the quality of published guidelines.
Methods: We provide details of the processes developed, using standardized tools for assessing the methodological quality of practice guidelines. As an example, we describe the application of our processes to the recent multi-societal GERD consensus guideline.
Results: Assessment of the multi-societal GERD consensus guideline by the iterative processes of SAGES Quality Assurance taskforce improved the quality of the final manuscript in all domains of appraisal. These processes are easily applicable to future guidelines.
Conclusions: Such systems will increase the confidence in SAGES recommendations and increase the implementation of SAGES guidelines. By demonstrating the rigor of Quality Assessment, this confidence also extends to a further increase in the assurance of the publications of the Surgical Endoscopy journal.
Keywords Quality Assurance · Clinical Practice Guidelines · AGREE tool · Recommendations
The Society of American Gastrointestinal and Endoscopic Surgeons’ mission is to innovate, educate, and collaborate to improve patient care [1]. The Society aims to be at the forefront of best practices in laparoscopic and endoscopic surgery by researching, developing, and disseminating clinical practice guidelines (CPG).
The manuscript “Multi-Society Consensus Conference and Guideline on the Treatment of Gastroesophageal Reflux Disease (GERD)”[2] is an example of such collaboration to develop impactful guidelines. Members of the SAGES Guidelines Committee have worked together with representatives of the American Society for Gastrointestinal Endoscopy, American Society for Metabolic and Bariatric Surgery, European Association for Endoscopic Surgery, Society for Surgery of the Alimentary Tract, and The Society of Thoracic Surgeons to develop this guideline.
SAGES has previously reported their Standard Operating Procedure of CPG creation to ensure high-quality guidelines [3], a process which has been followed in development of these guidelines. Both the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) [4] standards and Grading of Recommendations Assessment, Development and Evaluation (GRADE) [5] process are utilized by the SAGES Guidelines Committee to both ensure accurate reporting of included systematic review evidence and to evaluate the quality of the evidence and strength of the recommendations made. Implementation of these procedures has resulted in clinical practice guidelines rated highly by external reviewers such as the Emergency Care Research Institute (ECRI) ECRI’s TRUST (Transparency and Rigor Using Standards of Trustworthiness) Scorecard [6] which assesses against Institute of Medicine standards [7].
Working in parallel with the Guidelines working group, the SAGES Guidelines Committee has also formed a Quality Assurance (QA) taskforce, whose methods are described below. The role of the QA taskforce is to ensure all CPG endorsed by SAGES remain of the highest quality, meeting current international best practices. The QA taskforce ideally consults with the authors before guidelines development is commenced, with early emphasis on ensuring adequate involvement of all stakeholders, including patients, from the outset. The taskforce will then review multiple interim drafts of CPG manuscripts and provide feedback to the authors. The intent is that any deficiencies can be addressed during drafting, and consequently the final document will be of better quality. The final version of the manuscript will also be reviewed.
The SAGES QA taskforce is composed of multiple volunteer appraisers from the SAGES Guidelines Committee. Appraisers assess the drafts against the criteria of both the AGREE-II tool [8] and the newer AGREE-S extension [9]. These tools have been developed by an international consortium of methodologists, a biostatistician, a patient representative, a national authority representative and in the case of the AGREE-S, as well as surgeons with experience in guidelines development and representatives from various surgical subspecialties. A summary of the appraisers’ assessments, calculated according to the equations within the AGREE-II User’s Manual [10], is reported to the manuscript’s authors. Any member of the QA taskforce who is also an author on the Guidelines is excluded from appraisal.
The SAGES QA taskforce has been asked to provide editorial commentary of the Quality Assessment of these and future multi-society guidelines. The taskforce hopes that such commentary will highlight the rigorous methodological processes employed in the development of CPG with which SAGES is associated in order to reassure the reader that the recommendations can be accepted with confidence.
The QA assessment of “Multi-Society Consensus Conference and Guideline on the Treatment of Gastroesophageal Reflux Disease (GERD)” commenced with review of early drafts assessed using the AGREE-II checklist, with the six domains of assessment being Scope & Purpose (initial appraiser summary score of 80%), Stakeholder Involvement (61%), Rigor of Development (63%), Clarity of Presentation (74%), Applicability (56%), and Editorial Independence (50%). Suboptimal early scores resulted from Confl of Interest which were not adequately addressed in early drafts, nor were recommendations for implementation or auditing. Patient involvement had not been sought from the outset and this deficiency could not be rectified by the manuscript writing phase. Not all management options were initially clearly presented, particularly endoscopic treatment options, and the role of diagnostic testing was unclear. Lack of contribution to guidelines development by patients was noted. These concerns were noted by the appraisers to be common among other guidelines with which they were familiar and were all deemed by the appraisers to have only small eff on the overall quality of these particular guidelines. Nonetheless this early review allowed for clear recommendations to be made to the authors for improvement.
By the time of the next draft, a marked improvement in summary scores was noted, with Scope & Purpose (97%), Stakeholder Involvement (84%), Rigor of Development (95%), Clarity of Presentation (97%), Applicability (93%), and Editorial Independence (55%). Still, some areas for improvement were identified, such as lack of clarity regarding definitions of the health question and the explicit target population. Additionally, side effects and harms of interventions were not adequately described.
The authors addressed all suggestions and the final manuscript was submitted for publication. Review by AGREE-II criteria again showed high scores in all domains. Between reviews, the newer surgical guidelines-specific AGREE-S instrument had been published, enhancing assessment of guidelines on surgical interventions. It is an ongoing project of the QA taskforce to compare result correlation between AGREE-II assessment and AGREE-S. Of the six AGREE-S domains, which differ slightly from those of AGREE-II, all summary scores are now 94% or above (Scope & Purpose 94%, Stakeholder Involvement 100%, Evidence Synthesis 100%, Development of Recommendations 94%, Editorial Independence 100%, and Implementation & Update 98%).
It is acknowledged that there exists a degree of subjectivity in the appraisers’ scoring, less so with AGREE-II wherein the User’s Manual guides the appraisers through the scoring than perhaps in AGREE-S, but the multiple iterations performed by multiple appraisers familiar with Guideline development lead to a robust Quality Assessment.
With positive results demonstrated in this guideline’s development, SAGES will continue with this newly developed process to improve the quality of future SAGES-endorsed clinical practice guidelines. SAGES will continue to report on the final assessment of guidelines and we expect to see external validation of the high quality of our product.
We hope that such a process will increase the confidence in which the target audiences may have with SAGES recommendations and increase the implementation of the guidelines. This confidence relates to the individual guidelines, SAGES as a society, and by demonstrating the rigor of Quality Assessment, and also extends to an increase in the assurance of the publications of Surgical Endoscopy. This continued focus on high-quality CPG is of benefit to all.
Declarations
Disclosures Geoffrey Paul Kohn: Boston Scientific (speaker honorarium), Bethany Slater: Hologic (consultant), Cook Medical (consultant), not relevant for this manuscript. Mazen Al-Mansour: Intuitive Surgical (education payments), Medtronic (speaker fees), Gore (general payments), CONMED (general payments), Ziad Awad, Jennifer Salluzzo, Emily Miraflor, Uretz Oliphant: have no conflict of interest or financial ties to disclose.
References
- Society of American Gastrointestinal and Endoscopic Surgeons. Sages Mission Statement. Available at: https://www.sages.org/. Accessed 8 Aug 2022
- Slater BJ, Collings A, Dirks R, Gould J, Qureshi A, Juza R, Rodríguez-Luna MR, Wunker C, Kohn GP, Kothari S, Carslon E, Worrell S, Abou-Setta A, Ansari MT, Athanasiadis DI, Daly S, Dimou F, Haskins IN, Hong J, Krishnan K, Lidor A, Litle V, Low D, Petrick A, Soriano IS, Thosani N, Tyberg A, Velanovich V, Vilallonga R, Marks JM (2022) Multi-society consensus conference and guideline on the treatment of gastroesophageal reflux disease (GERD). Surg Endosc 2022 Dec 18
- Rogers AT, Dirks R, Burt HA, Haggerty S, Kohn GP, Slater BJ, Walsh D, Stefanidis D, Pryor A (2021) Society of American gastrointestinal and endoscopic surgeons (SAGES) guidelines development: standard operating procedure. Surg Endosc 35:2417–2427
- Moher D, Liberati A, Tetzlaff J, Altman DG, Group P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. BMJ 339:b2535
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, GRADE Working Group (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336:924–926
- ECRI Guidelines Trust®. Available at: https://www.ecri.org/solutions/ecri-guidelines-trust Accessed 1 Dec 2021
- Institute of Medicine (2011) Clinical practice guidelines We Can Trust. Washington, DC: National Academies Press
- Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Little-johns P, Makarski J, Zitzelsberger L (2010) AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 182:E839-842
- Logullo P, Florez ID, Antoniou GA, Markar S, López-Cano M, Silecchia G, Tsokani S, Mavridis D, Brouwers M, Antoniou SA (2022) AGREE-S: AGREE II extension for surgical interventions—United European Gastroenterology and European Association for Endoscopic Surgery methodological guide. United European Gastroenterol J 10:425–434
- Consortium ANS (2017) The AGREE II Instrument [Electronic version]. Available at: http://www.agreetrust.org. Accessed 8 August 2022
This document was reviewed and approved by the Board of Governors of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Jul 2022.
For more information please contact:
11300 West Olympic Blvd., Suite 600
Los Angeles, CA 90064
- Tel:
- (310) 437-0544
- Fax:
- (310) 437-0585
- Email:
- publications@sages.org
Guidelines for clinical practice are intended to indicate preferable approaches to medical problems as established by experts in the field. These recommendations will be based on existing data or a consensus of expert opinion when little or no data are available. Guidelines are applicable to all physicians who address the clinical problem(s) without regard to specialty training or interests, and are intended to indicate the preferable, but not necessarily the only acceptable approaches due to the complexity of the healthcare environment. Guidelines are intended to be flexible. Given the wide range of specifics in any health care problem, the surgeon must always choose the course best suited to the individual patient and the variables in existence at the moment of decision.
Guidelines are developed under the auspices of the Society of American Gastrointestinal and Endoscopic Surgeons and its various committees, and approved by the Board of Governors. Each clinical practice guideline has been systematically researched, reviewed and revised by the guidelines committee, and reviewed by an appropriate multidisciplinary team. The recommendations are therefore considered valid at the time of its production based on the data available. Each guideline is scheduled for periodic review to allow incorporation of pertinent new developments in medical research knowledge, and practice.