• Skip to primary navigation
  • Skip to main content
  • Skip to footer

SAGES

Reimagining surgical care for a healthier world

  • Home
    • Search
    • SAGES Home
    • SAGES Foundation Home
  • About
    • Who is SAGES?
    • SAGES Mission Statement
    • Advocacy
    • Strategic Plan, 2020-2023
    • Committees
      • Request to Join a SAGES Committee
      • SAGES Board of Governors
      • Officers and Representatives of the Society
      • Committee Chairs and Co-Chairs
      • Full Committee Rosters
      • SAGES Past Presidents
    • Donate to the SAGES Foundation
    • SAGES Store
    • Awards
      • George Berci Award
      • Pioneer in Surgical Endoscopy
      • Excellence In Clinical Care
      • International Ambassador
      • IRCAD Visiting Fellowship
      • Social Justice and Health Equity
      • Excellence in Community Surgery
      • Distinguished Service
      • Early Career Researcher
      • Researcher in Training
      • Jeff Ponsky Master Educator
      • Excellence in Medical Leadership
      • Barbara Berci Memorial Award
      • Brandeis Scholarship
      • Advocacy Summit
      • RAFT Annual Meeting Abstract Contest and Awards
    • “Unofficial” Logo Products
  • Meetings
    • NBT Innovation Weekend
    • SAGES Annual Meeting
      • 2024 Scientific Session Call For Abstracts
      • 2024 Emerging Technology Call For Abstracts
    • CME Claim Form
    • Industry
      • Advertising Opportunities
      • Exhibit Opportunities
      • Sponsorship Opportunities
    • Future Meetings
    • Related Meetings Calendar
  • Join SAGES!
    • Membership Benefits
    • Membership Applications
      • Active Membership
      • Affiliate Membership
      • Associate Active Membership
      • Candidate Membership
      • International Membership
      • Medical Student Membership
    • Member News
      • Member Spotlight
      • Give the Gift of SAGES Membership
  • Patients
    • Healthy Sooner – Patient Information for Minimally Invasive Surgery
    • Patient Information Brochures
    • Choosing Wisely – An Initiative of the ABIM Foundation
    • All in the Recovery: Colorectal Cancer Alliance
    • Find a SAGES Member
  • Publications
    • SAGES Stories Podcast
    • SAGES Clinical / Practice / Training Guidelines, Statements, and Standards of Practice
    • Patient Information Brochures
    • TAVAC – Technology and Value Assessments
    • Surgical Endoscopy and Other Journal Information
    • SAGES Manuals
    • SCOPE – The SAGES Newsletter
    • COVID-19 Annoucements
    • Troubleshooting Guides
  • Education
    • Wellness Resources – You Are Not Alone
    • OpiVoid.org
    • SAGES.TV Video Library
    • Safe Cholecystectomy Program
      • Safe Cholecystectomy Didactic Modules
    • Masters Program
      • SAGES Facebook Program Collaboratives
      • Acute Care Surgery
      • Bariatric
      • Biliary
      • Colorectal
      • Flexible Endoscopy (upper or lower)
      • Foregut
      • Hernia
      • Robotics
    • Educational Opportunities
    • HPB/Solid Organ Program
    • Courses for Residents
      • Advanced Courses
      • Basic Courses
    • Fellows Career Development Course
    • Robotics Fellows Course
    • MIS Fellows Course
    • Facebook Livestreams
    • Free Webinars For Residents
    • SMART Enhanced Recovery Program
    • SAGES OR SAFETY Video
    • SAGES Top 21 MIS Procedures
    • SAGES Pearls
    • SAGES Flexible Endoscopy 101
    • SAGES Tips & Tricks of the Top 21
  • Opportunities
    • NEW-Area of Concentrated Training Seal (ACT)-Advanced Flexible Endoscopy
    • SAGES Fellowship Certification for Advanced GI MIS and Comprehensive Flexible Endoscopy
    • Multi-Society Foregut Fellowship Certification
    • SAGES Research Opportunities
    • Fundamentals of Laparoscopic Surgery
    • Fundamentals of Endoscopic Surgery
    • Fundamental Use of Surgical Energy
    • Job Board
    • SAGES Go Global: Global Affairs and Humanitarian Efforts
  • Search
    • Search All SAGES Content
    • Search SAGES Guidelines
    • Search the Video Library
    • Search the Image Library
    • Search the Abstracts Archive
  • OWLS
  • Log In

A Novel Video-Based Performance Rating System for Asynchronous Pinpoint Feedback Loops, Reduced Cycle Time, and Improved Patient Outcomes

Vivek Agrawal, MS1, Jean-Sebastien Legare, MSc1, Daniel Naab, BS1, Justin P Wagner, MD2, Juan Espinosa, MD3, Patrick J Colp, MSc1, David C Chen, MD2. 1Comment Bubble, Inc., 2David Geffen School of Medicine at University of California, Los Angeles, 3Children’s Hospital Los Angeles

Objective

As video recording in the operating room has become more prevalent, there is significant potential to revolutionize clinical training and improve patient outcomes.

The landmark study of patient outcome and surgical proficiency assessed using video by Brickermeyer, et al. (N Engl J Med 2013; 369:1434-1442) showed a strong correlation. The bottom quartile of surgical skill was associated with higher complication rates (14.5% vs. 5.2%, P<0.001) and higher mortality (0.26% vs. 0.05%, P=0.01) than the top quartile, suggesting peer rating as an effective strategy for assessing surgical proficiency towards improved mortality rates.

The American Board of Surgery (ABS) recommends the Operative Performance Rating System (OPRS) as a model evaluation system, but as a paper-based tool, there are limitations. We propose a novel web-based analytics tool for pinpoint quantitative scoring and pinpoint qualitative feedback integrating video into the OPRS model.

By reducing cycle time between surgery and feedback, and providing a pinpoint analytics platform that can be superimposed on any video, a new standard can be established for assessment and feedback of operative performance for surgical residents and, potentially even practicing surgeons, resulting in higher satisfaction and improved outcomes.

Soon, surgeons will be able to review their performance in the operating room and adjust just like a professional athlete can review their performance and adjust after every game.

Description

Through collaboration between Comment Bubble, Inc., a Silicon Valley-based company at the forefront of video content analytics, and UCLA, we have designed web technology based on the patent-pending CommentBubble.com platform. Multiple feedback types include pinpoint numerical ratings, textual feedback, video feedback, and audio feedback tagged to moment of assessment in the video, adding new levels of reliability and value to the OPRS model.

Integrated analytical tools include interactive user interface with graphs of feedback vs. time, display of comments, databases, and reports. Chapter markers are created such that numerical assessments of procedure-specific criteria will be valid, along with global assessment metrics as designed in the conventional OPRS.

We focus on operative evaluation and skill acquisition in surgical residency, but it is applicable for all aspects of competency-based learning from patient interactions to skills assessment.

Preliminary results

CommentBubble.com was released in 2014 as a qualitative video analytics tool and is used worldwide in secondary and post-secondary education. Additionally, it is being used to coach and provide expert feedback. Students and faculty find the interface convenient, the analytics insightful, and the ability to automatically cue the video to the moment of assessment profound. The medical video OPRS software in development is building on these best practices.

Future directions

With the beta release of the Comment Bubble video OPRS in February 2015, metrics will be established based on ABS-approved criteria and residents will be assessed and given feedback. The satisfaction will be surveyed and compared to existing evaluation systems, along with time savings. Additionally, we will explore asynchronous international surgical collaboration and training with this tool. Long-term studies will include comparison of patient outcomes before and after using Comment Bubble.

30

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Pinterest
  • WhatsApp
  • Reddit

Related

Hours & Info

11300 West Olympic Blvd, Suite 600
Los Angeles, CA 90064
1-310-437-0544
[email protected]
Monday - Friday
8am to 5pm Pacific Time

Find Us Around the Web!

  • Facebook
  • Twitter
  • YouTube

Important Links

SAGES 2024 Meeting Information

Healthy Sooner: Patient Information

SAGES Guidelines, Statements, & Standards of Practice

SAGES Manuals

 

  • taTME Study Info
  • Foundation
  • SAGES.TV
  • MyCME
  • Educational Activities

Copyright © 2023 Society of American Gastrointestinal and Endoscopic Surgeons