Educational Implications for Surgical Telementoring: a Current Review with Recommendations for Future Practice, Policy, and Research

This document was reviewed and approved by the Board of Governors of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Jun 2017.

M. Augestad • H. Han • J. Paige • T. Ponsky • C. M. Schlachta • Dunkin • J. Mellinger

Received: 6 June 2017 / Accepted: 20 June 2017 / Published online: 27 June 2017

Abstract

Background: Surgical telementoring (ST) was introduced in the sixties, promoting videoconferencing to enhance surgical education across large distances. Widespread use of ST in the surgical community is lacking. Despite numerous surveys assessing ST, there remains a lack of high-level scientific evidence demonstrating its impact on mentorship and surgical education. Despite this, there is an ongoing paradigm shift involving remote presence technologies and their application to skill development and technique dissemination in the international surgical community. Factors facilitating this include improved access to ST technology, including ease of use and data transmission, and affordability. Several international research initiatives have commenced to strengthen the scientific foundation documenting the impact of ST in surgical education and performance.

Methods: International experts on ST were invited to the SAGES Project Six Summit in August 2015. Two experts in surgical education prepared relevant questions for discussion and organized the meeting (JP and HH). The questions were open-ended, and the discussion continued until no new item appeared. The transcripts of interviews were recorded by a secretary from SAGES.

---

Surgical education is at a critical juncture today, threatened by diminished funding, competing service obligations, mounting paper work, and the need to do more in less time with less support. However, it is only by maintaining effective teaching and mentoring of surgical skills, concepts, and precepts that our profession will remain strong. NJ Soper; SAGES 2001 presidential address [1].

---

Results: In this paper, we present a summary of the work performed by the SAGES Project 6 Education Working Group. We summarize the existing evidence regarding education in ST, identify and detail conceptual educational frameworks that may be used during ST, and present a structured framework for an educational curriculum in ST. Conclusions The educational impact and optimal curricular organization of ST programs are largely unexplored. We outline the critical components of a structured ST curriculum, including prerequisites, teaching modalities, and key curricular components. We also detail research strategies critical to its continued evolution as an educational tool, including randomized controlled trials, establishment of a quality registry, qualitative research, learning analytics, and development of a standardized taxonomy.

Keywords: Telementoring • Telemedicine • Surgical education • Surgical training • Surgical technology • Health policy

Education is the cornerstone of surgical skill development. A key component of the educational paradigm in medicine and surgery involves the progressive transmission of knowledge, skills, and attitudes from a more experienced surgeon to the trainee. From the early 1900s, there has been an ongoing discussion about the best way to train and educate physicians in general [2] and surgeons in particular [3, 4]. The Flexner Report, published in 1910, was the first US medical and surgical workforce report and continues to be referenced in the ongoing surgical workforce conversation [2]. Significantly, Flexner recognized the need for ongoing adaptation of the training paradigm in medicine and argued for a continuous assessment and adjustment to new developments, including changes in pedagogics, as well as society in general.

Surgical education is under intense pressure in the recent era of duty hour reform and the related and appropriate focus on patient safety and quality in health care delivery. Surgical trainees struggle to balance the acquisition of clinical judgment, technical skill, professionalism, and continuity oriented commitment to the patient against the challenges of fatigue, resident well-being, and related patient safety concerns [5]. After years of debate, there exists no consensus on the optimal structure of surgical training, including the most appropriate balance between patient care and service activities and structured didactics [5].

Alongside these training challenges, the surgical workforce is diminishing despite an increasing need for surgery in the population [6, 7]. Several reports have forecast a future surgeon shortage, with an attendant high impact on maldistribution of healthcare, especially affecting rural areas [7]. An 18% decrease of surgeons has been predicted based on these trends; it has been estimated that the US would have had to train more than 100,000 surgeons by 2030 to maintain access for the population, with a total cost of $37 Billion [8]. Compounding these pressures, the time required to train a surgeon is considerable. Furthermore, the ongoing introduction of new surgical techniques, technology, and treatment methods continues to increase the curricular compendium for the surgical trainee and fuels the ongoing trend toward surgical subspecialization.

Given these pressures, there is a need to modernize surgical education in line with societal need. Part of this adjustment will involve the incorporation of new pedagogical methods and teaching technologies. One such technology is surgical telementoring (ST). It has been described as a natural fit in surgical education, due to the skill orientation and visual components of surgical training, and in view of ST’s ability to overcome geographical barriers [9–13].

ST has been assessed in numerous surgical trials, including intra-hospital ST, inter-hospital ST, and inter- continental venues [14–19]. In addition, the technology used for ST has gone through a tremendous development over the past 50 years, from a complex technology with low audio–video (AV) transmission bandwidth, to a user-friendly, low-cost technology with high bandwidth.

‘‘Telementoring is a telemedicine technique that involves the remote guidance of a treatment or a procedure where the caregiver has no or limited experience with the featured technique’’ [20]. This definition of telementoring provides a general meaning of telementoring. However, surgical telementoring (ST) has different attributes that become a basis of educational program. ST focuses on developing specific surgical techniques of trainees  who may not be able to access expert surgeons in the area. Trainees in ST are not novice learners but practicing surgeons who do not have a targeted surgical technique. While ST aims at developing surgical techniques, ST is heavily relying on a learning relationship among participants [21]. As the laparoscopic surgical technology has rapidly been adopted, video representation of surgical objects becomes a crucial medium for surgical work process. This video representation of surgical objects and two-way audio transmissions through telecommunication technology between a mentor and a mentee are also an essential part of ST.

In 2015, SAGES initiated the Surgical Telementoring Project Six (P6), a term which was adopted from the military aviation industry where a more experienced pilot ‘‘watched the back’’ of a less experienced fighter pilot. This SAGES initiative resulted in the P6 Summit, and the pre- sent report is a summary of the Surgical Telementoring Educational Committee meeting that was part of that Summit.

International experts of ST were invited to the SAGES Project Six Summit in August 2015.

Two experts in surgical education prepared relevant questions for discussion and organized the meeting (JP and HH). The questions were open-ended, and the discussion continued until no new item appeared. The transcripts of interviews were recorded by a secretary from SAGES.

The objective of the committee’s work was to provide: (1) working definitions of terms, (2) a brief literature review to establish the current state of ST, (3) an outline of relevant educational frameworks, and (4) broad recommendations from the group, including [5] future directions for ST research.

Working definitions

Surgical telementoring

SAGES defines telementoring as a relationship, facilitated by telecommunication technology, in which an expert (Mentor) provides guidance to a less experienced learner (Mentee) from a remote location [22].

Mentor

An expert surgeon who undertakes to impart his/her clinical knowledge and skills in a defined setting to a mentee. The mentor must be appropriately privileged, skilled, and experienced in the procedure(s) and or technique(s) in question. In order to serve as a mentor in a specific procedure or technique, the surgeon (mentor) must be a recognized authority (e.g., publications, presentations, extensive clinical experience) in the particular field of expertise [23].

Telementor

An expert surgeon who undertakes to impart his/her clinical knowledge and skills in a defined setting to a student. The telementor, by definition, does not have the ability to physically intervene onsite in the primary activity without the telecommunications interface [21].

Mentee

A surgeon with appropriate basic knowledge and experience seeking individual training in skills and/or procedures not previously learned in prior formal residency or fellowship training. The mentee must have appropriate background knowledge, basic skills, and clinical experience relevant to the proposed curriculum. The mentee should be board eligible or certified in the appropriate specialty or possess equivalent board certification from outside the United States [23].

Videoconferencing (VC)

VC is defined as a real-time, live, interactive program in which one set of participants is at one or more locations and the other set of participants is at another location. The VC permits interaction, including audio and/or video, and possibly other modalities, between at least two sites [21].

Telestration

A technique enabling a remote mentor to perform drawing of freehand sketches over still image or video. These sketches are observed by a mentee at another location [24, 25].

Literature review of current status

Educational effectiveness of ST

Although most studies looking at ST report improved perceptions regarding usefulness of ST, few surveys examine the educational outcomes of ST in a structured manner. In a recent review, seven ST papers with an educational outcome were identified [26]. These surveys assessed the educational outcomes with different methods, i.e., number of times verbal advice is given vs. taking over the operation [27]; time of the procedure [28, 29]; path lengths, mode of grasping, cutting, clip applying, and suturing [30, 31]; recognition of anatomical landmarks [32]; and scoring scales [5, 28, 33]:

Taking over the operation 

Byrne and colleagues included 34 cases of laparoscopic cholecystectomies in a ST trial [27]. They measured the number of times intervention was needed during the surgical procedure (no intervention, verbal advice, scrub in). No intervention was necessary in 68% of cases, verbal advice was given in 26% of cases, and, in two cases, the mentor had to come to the OR from their remote location to scrub in and take over the case. They concluded that ST could be an important step forward in defining the transition from competence under direct supervision to competence for an unsupervised performance.

Recognition of anatomical landmarks 

In a study by Rafiq and colleagues, 25 thyroidectomy explorations were monitored and transmitted to remotely located surgical trainees [32]. They were asked to confirm seven anatomical landmarks during surgery, which were achieved in more than 90% of the tasks. This study supports the feasibility of ST as a tool for teaching and mentoring a remote audience. To our knowledge, this is the only ST study where the experts were located in the OR, whereas the mentees were remotely located (reverse mentoring).

The quality of mentoring

In a French study by Sereno and colleagues, forty surgeons were assisted onsite and remotely [33]. They conclude that there exists a superiority of onsite mentoring compared to remote mentoring, especially early in the mentoring program. The quality of the mentoring (i.e., scoring scale of interaction with mentor, quality of teaching) was higher at the first mentor session, when onsite mentoring was compared to remote mentoring. The remote mentors were, however, dependent on a robotic arm to demonstrate the task, which may have introduced technological obstacles during mentoring. They conclude that remote mentoring is a useful adjunct to local mentoring where the mentor gradually withdraws to a more remote location.

Time of procedure

Valentino and colleagues explored in a study from 2005 the feasibility of ST in distant teaching [29]. Forty-eight patients treated with endovascular aortic aneurysm repair were included, and procedural time was the educational outcome measurement. In this study, there were no significant differences between onsite mentored and remote mentored cases. They conclude that ST represents a model for teaching invasive procedures.

Scoring scales

To our knowledge, two trials have compared surgical performance with and without ST coaching. [28, 31] Eight general surgery residents performing three operative procedures, first without mentoring and then telementored by a surgical subspecialist. Overall mean performance scores improved in all scenarios when residents were remote proctored vs. when they were not proctored (p \ 0.001). Panait and colleagues showed that that telementoring could be an adjunct to surgical training [31]. Twenty medical students were assigned to simulator training (i.e., grasping, task performance time, clip applying, suture task performance time, and path length), with either a local mentor or a telementor. They showed that training with mentoring resulted in similar levels of surgical performance between locally mentored and telementored groups.

Other aspects related to surgical telementoring

Video coaching

An increasing body of evidence demonstrates that video coaching (i.e., retrospective coaching of a video performance of a learner) has a positive impact on surgical performances [34]. In a recent randomized trial by Singh et al., video-based coaching enhanced the quality of laparoscopic surgical performance on both simulated (i.e., using virtual reality [VR]) and porcine laparoscopic cholecystectomies (LC), although at the expense of increased time. They conclude that video-based coaching is a feasible method for maximizing performance enhancement from every clinical exposure [35, 36]. Other surveys have shown a similar positive impact of video coaching [37]. However, video coaching differs from ST in one fundamental way: ST is real-time coaching whereas video coaching is performed in retrospect.

Video assessment of surgical performance

There is a strong association between video-scored surgical performance and surgical complications. In a survey by Birkmeyer et al., expert surgeons assessed videos submitted by consultant surgeons performing bariatric procedures [38]. Interestingly, it was shown that after the scoring scale was matched with a quality registry, surgical skills were significantly associated with the rate of complications [38]. Rating scales for laparoscopic skills are commonly employed in assessing surgical performance and have a natural place in ST. The most commonly used rating scales are GOALS (The Global Operative Assessment of Laparoscopic Skills), OSATS (Objective Structured Assessment of Technical Skills), and OPRS (Operative Performance Rating System). Incorporating such rating scales systematically in trials assessing the educational aspects of ST would contribute to the strength and transferential capability of future studies [39–41].

Telestration as a ST teaching tool

Telestration is a necessary teaching tool during ST. Telestration enables the mentor to identify anatomical landmarks, identify planes of dissection, and identify anatomical danger zones. Furthermore, it enhances the teaching process as verbal instructions may be exemplified by drawings. It has been shown that telestration-supplemented guidance may reduce the duration of the telementored session by more than 30% [24]. There is an increasing amount of literature assessing the positive impact of telestration as a component of ST [25, 42].

In summary, the existing literature-based evidence of educational benefits of ST is weak (level IV) and consists mostly of observational studies performed on simulators, or with a limited number of patients (Table 1).

Relevant conceptual frameworks

ST practice is based on several theoretical frameworks including social learning theory [43], legitimate peripheral participation [44], community of practice [45], and transactional distance [46]. Knowledge is socially constructed through the learners’ gradual participation in the central practice of experts in a community of a profession. In virtual environments, physical distance may be perceived as a barrier. Meaningful instructional interactions, however, become an actual decisive factor of transactional distance between teachers and learners in virtual environments.

Several conceptual frameworks may be adapted in educational programs using ST, i.e., the ADDIE model, the GROW model, and SSTAR [47–50]. The frameworks may be used to (a) plan a ST educational program, (b) guide and structure coaching during ST, and (c) provide structured feedback to the mentors. However, further research is needed to define these conceptual frameworks and their ultimate utility in a structured ST program.

The ADDIE model: designing a structured ST program

The ADDIE model is a conceptual framework and is typically used in instructional and training development [47]. ADDIE may be used by mentors in a structured ST program, as a descriptive guideline for designing effective training in five phases:

Analysis phase

This phase clarifies the instructional problems and objectives, and identifies the learning environment, as well as the mentor’s existing knowledge and skills.

Design phase

This phase identifies learning objectives, assessment instruments, exercises, content, analysis of subject matter, lesson planning, and media selection.

Development phase

Designers create storyboards and graphics. If e-learning is involved, programmers develop or integrate technologies. The project is reviewed and revised according to feedback.

Implementation phase

This phase develops procedures for mentors and mentees.

Evaluation phase

The evaluation phase consists of two aspects: formative and summative. Formative evaluation is present in each stage of the ADDIE process, while summative evaluation is conducted on finished instructional programs or products.

The GROW model: providing structured feedback to mentees

The GROW (Goals, Reality, Options, Wrap-up) model is a useful tool that can be used in different mentoring scenarios such as ST. As described by Singh et al., performance mentoring is generally goal oriented [35]. The GROW model offers a way of structuring coaching sessions to facilitate a balanced discussion. The GROW model guides a mentoring conversation through four vital stages of goal-oriented coaching:

Goals

Focus on specific targets that the mentee wishes to achieve.

Reality

Exploration of the true nature of the problem (performance review).

Options

Formulation of effective solutions, particularly to the issues that prevent the mentees from achieving their goals.

Wrap-up

Development of an action plan for the mentee to move toward the originally stated goals and examination of potential obstacles.

The usefulness of the GROW model in a coaching program was shown by Singh et al. [35].

The SSTAR model: providing structured feedback to mentors

Recently, a framework for surgical teaching critique (The Structured Training Trainer Assessment Report, SSTAR) was presented by Wyles et al., which may be adopted to the ST setting [50]. The authors have developed a structured feedback tool that assesses training quality and structures feedback to surgical trainers. Semi-structured interviews of 29 surgical trainers, ten trainees, and four educationalists were performed, and through the Delphi process essential items pertaining to desirable trainer characteristics were determined. The different items within the three groups were then further subdivided into ‘‘teaching structure,’’ ‘‘teaching behavior,’’ ‘‘mentor attributes,’’ and ‘‘role model.’’ They conclude that SSTAR is a reliable, feasible, and acceptable evaluation tool for both trainers and trainees in surgical training programs. SSTAR has recently been successfully implemented into the English National Training Program for laparoscopic colorectal surgery [50]. The role of SSTAR in a structured ST program should be further validated and explored.

Non-technical skills and surgical telementoring

Given that ST inherently involves teamwork, it is important to investigate learning and performance at a team level as well as an individual level. Non-technical skills in the operating room, including self-awareness, communication, and team performance, have been recognized as an important factor for patient outcome [51]. It is unknown how ST impacts non-technical skills in the OR. ST impacts communication form, as the mentor by definition is not present in the OR. Audio and video delay may hinder optimal communication and create a source of miscommunication. Thus, synchronous virtual communication is important during telementoring, and protocols to enhance good communication routines during the telementoring session itself are critical. Similarly, asynchronous communication and relationship building between mentors and mentees is also crucial part for the educational process. That is, relationship and communication building activities distinct from the telementoring performance environment are foundational to the educational outcomes achieved through a ST experience.

Secondly, ST may impact the traditional hierarchal leadership model in the OR, where the operating surgeon is the team leader. In ST, the mentor will be a more experienced surgeon, but is remotely located. This will obviously influence the traditional leadership structure in the OR. Recently, Warth and colleagues launched a research initiative to further explore the impact of ST on communication, teamwork, and education [52, 53]. In this research, analyses of video recording of team performance will play a major role in exploring the underlying mechanisms and potential impact of this altered team structure.

In 2004, Vincent et al. created a framework of factors that influence clinical practice in the OR (organizational, work environment, team, individual, task, and patient factors) [51]. ST may impact all these factors as telementoring technology facilitates a feeling of being together (physical presence) and interfaces with all of these elements.

Recommendations

Components in a structured ST curriculum

We recommend that a structured ST curriculum consists of four main elements, namely prerequisites for entering the program, teaching modalities, curricular components, and methods of assessment. A framework for a structured training program for mentors (train the trainer) is shown in Table 2, whereas a structured training program for mentees is shown in Table 3.

Prerequisites

Given that a mentor/mentee in a ST program is not a pure novice, defining an entry level of performance for ST is important, and should be defined in terms of knowledge, skills, and leadership. The mentor has to prove excellence not only in the surgical procedure itself, but must also demonstrate high-level knowledge in pedagogical methodology. Mentors are the experts in this setting. They should be surgical experts and also process experts in ST, including virtual communication, risk management, and leadership in a virtual environment. Similarly, the mentees should have certain predefined surgical skills, they should be affiliated at an accredited institution, and they should have a letter of support from their institution.

Teaching modalities

Different teaching modalities are needed. For mentors/mentees in their index experience with ST, orientation or simulated sessions would be helpful. These sessions should reflect the different settings of ST, i.e., in the OR, inter- or intra-hospital, as appropriate to the planned application.

Curricular components

Principally, the objective of a ST curriculum does not differ from other surgical courses, i.e., to facilitate progression toward proficiency in a surgical technique/ method. However, the educational setting is fundamentally different from traditional mentoring, and the curricular components should focus on the technology including trouble shooting as well as communication obstacles and team integration. Languages and terms in telementoring can be different, and there is a need to develop a structured method to communicate during a ST session, including common vocabulary and patterns of communication.

Assessment methods

Video-based reviews may be a particularly efficient method to assess ST sessions. These reviews may be done either by video coaching, where the mentor and the mentee review the ST session in a structured manner, or by blinded review of submitted videos. Available tools such as GOALS and OPRS should be incorporated into this structure as appropriate. Other traditional assessment methods should also be used (360° feedback, pre- and post- course test, etc.)

Other reference points

Finally, a training program in ST may benefit from the experience and design of other surgical training programs, or from already published and pertinent guidelines. Examples of such training programs are the ‘‘The National Training Program in Laparoscopic Colorectal Surgery’’ (http://lapco.nhs.uk/) and the SAGES Framework for Post Residency Surgical Education (https://www.sages.org/pub lications/guidelines/framework-for-post-residency-surgical- education-training/).

Future perspectives/research opportunities

Future research opportunities include, but are not limited to, the following areas:

Randomized controlled trials

Randomized controlled trials remain the gold standard in medical research and produces high-level evidence. Many technological, organizational, and time-consuming obstacles exist however, to perform a high-quality ST randomized controlled trial. Nonetheless, an attempt should be made to design studies in this manner to demonstrate effectiveness of ST.

A quality registry of ST

A prospective quality registry of ST may produce high level of evidence. All ST cases performed (videos, educational and clinical outcome variables) may be registered in such a quality archive.

Qualitative research

We also need to have an in-depth understanding of what instructional activities are happening before, during, and after ST, and how these elements interface with each other, in order to design and develop educational guidance for ST.

ePortfolio or learning analytics

For lifelong learning, it is important to capture and display learner performance in ST longitudinally. It should not be a one-time episode but continuous improvement.

Developing standardized lexicon/taxonomy for ST

There is a need to develop a standardized lexicon and taxonomy for ST. Communication during ST is different compared to traditional mentoring and a standardized ST lexicon/taxonomy will facilitate research and standardize deployment in a variety of settings.

At present, there is limited evidence regarding the educational outcomes of ST. Several case–control or observational surveys assessing educational outcomes of ST are published, but they do not represent a high level of evidence. Outcome studies (focusing on both clinical and educational outcomes) are important to future development in this field, and more robust research is needed.

Acknowledgements Rolv-Ole Lindsetmo, MD, PhD; Michael Kim, MD; Melina Vassilou, MD; Erik Dutson, MD; Maria Marcela Bailez, MD; Rifat Latifi, MD. Industry participants: Chuck Kennedy (Med- tronic), Brian Witte (Ethicon), Kris Danforth (Stryker), Suzanne Jefferson (Karl Storz), Anthony Jarc (Intuitive).

Disclosures Dr. Paige discloses other support from Oxford University Press, as well as grants from LSU Board of Supervisors, Acell, Inc., Intuitive Inc., HRSA, and SGEA, during the conduct of the study; he also discloses that he serves as Chair of the Continuing Education Committee for SAGES. Dr. Ponsky discloses personal fees from Conmed, during the conduct of the study. Drs. Augestad, Han, Schlachta, Dunkin, and Mellinger have no conflicts of interest or financial ties to disclose.

References

1. 1. 1. Soper NJ (2001) SAGES and surgical education. Surg Endosc 15(8):775–780
      2. Cooke M, Irby DM, Sullivan W, Ludmerer K (2006) American medical education 100 years after the Flexner Report. N Engl J Med 14(355):1339–1344
      3. Kim S, Dunkin BJ, Paige JT, Eggerstedt JM, Nicholas C, Vassilliou MC et al (2014) What is the future of training in surgery? Needs assessment of national stkeholders. Surgery. 156(3):707–717
      4. Claydon E, McAlister VC (2012) The life of John Wishart (1850–1926): study of an academic surgical career prior to the Flexner Report. World J Surg 36(3):684–688
      5. Bilimoria K, Hoyt D, Lewis F (2015) Making the case for investigating flexibility in duty hour limits for surgical residents. JAMA Surg 150(6):503–504
      6. Sheldon GF (2007) Surgical workforce since the 1975 study of surgical services in the United States: an update. Ann Surg 246(4):541–545
      7. Fraher E, Knapton A, Sheldon G, Meyer A, Ricketts TC (2013) Projecting surgeon supply using a dynamic model. Ann Surg 257(5):867–872
      8. Williams TE Jr, Satiani B, Thomas A, Ellison EC (2009) The impending shortage and the estimated cost of training the future surgical workforce. Ann Surg 250(4):590–597
      9. Ponsky T, Schwachter M, Parry J, Rothenberg S, Augestad K (2014) Telementoring: the surgical tool of the future. Eur J Pediatr Surg 24(4):287–294
      10. Augestad KM, Bogen E, Patel H (2015) Telemedicine as a quality improvement facilitator in pelvic cancer surgery. In: Patel H, Mourls T, Joseph JV, Delaney CP (eds) Pelvic cancer surgery. Springer, London, pp 29–37
      11. Wood D (2011) No surgeon should operate alone: how telementoring could change operations. Telemed J e-Health 17(3):150–152
      12. Doarn CR (2003) Telemedicine in tomorrow’s operating room: a natural fit. Semin Laparosc Surg 10(3):121–126
      13. Augestad KM, Lindsetmo RO (2009) Overcoming distance: video-conferencing as a clinical and educational tool among surgeons. World J Surg 33(7):1356–1365
      14. Gambadauro P, Magos A (2008) NEST (network enhanced surgical training): a PC-based system for telementoring in gynaecological surgery. Eur J Obstet Gynecol Reprod Biol 139(2):222–225
      15. Schlachta C, Kent S, Lefebvre K, McCune M, Jayaraman S (2009) A model for longitudinal mentoring and telementoring of laparoscopic colon surgery. Surg Endosc 23(7):1634–1638
      16. Schlachta C, Lefebvre K, Sorsdahl A, Jayaraman S (2010) Mentoring and telementoring leads to effective incorporation of laparoscopic colon surgery. Surg Endosc 24(4):841–844
      17. Rothenberg S, Yoder S, Kay S, Ponsky T (2011) Initial experience with surgical telementoring in pediatric laparoscopic surgery using remote presence technology. J Laparoendosc Adv Surg Tech A 5:1–4
      18. Netto NR, Mitre AI, Lima SVC, Fugita OE, Lima ML, Stoianovici D, et al (2003) Telementoring between Brazil and the United States: initial experience.  J Endourol 17(4):217–220
      19. Baram-Clothier E, Augestad KM (2014) Surgical telementoring news. The American Medical Foundation [Internet]. 7(1):1–10. http://www.medicalfoundation.org/wp-content/ uploads/2012/04/Surgical-Telementoring-News-2014e.pdf
      20. Rosser JC, Gabriel N, Herman BAB, Murayama M (2001) Telementoring and teleproctoring. World J Surg 25(11):1438– 1448
      21. Society of American Gastrointestinal and Endoscopic Surgeons [Internet] (2004) Guidelines for the surgical practice of telemedicine.  Society of American Gastrointestinal and Endoscopic Surgeons [Internet]. https://www.sages.org/publications/guidelines/guidelines-for-the-surgical-practice-of-telemedicine/
      22. Schlachta CM, Nguyen NT, Ponsky T, Dunkin B (2016) Project 6 Summit: SAGES telementoring initiative. Surg Endosc 30(9):3665-3672
      23. SAGES (2013) Framework for post-residency surgical education and training. Society of American Gastrointestinal and Endoscopic Surgeons [Internet]. https://www.sages.org/publications/guidelines/framework-for-post-residency-surgical-education-training/
      24. Budrionis A, Hasvold P, Hartvigsen G, Bellika JG (2016) Assessing the impact of telestration on surgical telementoring: A randomized controlled trial. J Telemed Telecare 22(1):12–17
      25. Budronius A, Belika JG, Augestad K (2012) Telestration in mobile tlementoring. In: Van Gemert-Pijnen, et al (eds)  eTELEMED 2013: the fifth international conference on eHealth, telemedicine, and social medicine. IARIA, Wilmington, DE, pp 307-309
      26. Augestad KM, Bellika JG, Budrionis A, Chomutare T, Lindsetmo RO, Patel H, et al (2013) Surgical telementoring in knowledge translation—clinical outcomes and educational benefits: a comprehensive review. Surg Innov 20(3):276–284
      27. Byrne JP, Mughal MM (2000) Telementoring as an adjunct to training and competence-based assessment in laparoscopic cholecystectomy. Surg Endosc 14(12):1159–1161
      28. Ereso AQ, Garcia P, Tseng E, Gauger G, Kim H, Dua MM, et al (2010) Live transference of surgical subspecialty skills using telerobotic proctoring to remote general surgeons. J Am Coll Surg  211(3):400–411
      29. Valentino MD, Alerci M, Bogen M , Tutta P, Sartori F, Marty B, et al (2005) Telementoring during endovascular treatment of abdominal aortic aneurysms: a prospective study. J Endovasc Ther 12(2):200–205
      30. Snyder CW, Vandromme MJ, Tyra SL, Hawn MT (2009) Proficiency-based laparoscopic and endoscopic training with virtual reality simulators: a comparison of proctored and independent approaches. J Surg Educ 66(4):201–207
      31. Panait L, Rafiq A, Tomulescu V, Boanca C, Popescu I, Carbonell A et al (2006) Telementoring versus on-site mentoring in virtual reality-based surgical training. Surg Endosc 20(1):113–118
      32. Rafiq A, Moore JA, Zhao X, Doarn CR, Merrell RC (2004) Digital video capture and synchronous consultation in open surgery. Ann Surg 239(4):567–573
      33. Sereno S, Mutter D, Dallemagne B, Smith CD, Marescaux J (2007) Telementoring for minimally invasive surgical training by wireless robot. Surg Innov 14(3):184–191
      34. Greenberg CC, Dombrowski J, Dimick JB (2015) Video-based surgical coaching: an emerging approach to performance improvement. JAMA Surg 151(3):282-283
      35. Singh P, Aggarwal R, Tahir M, Pucher PH, Darzi A (2015) A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills. Ann Surg 261(5):862–869
      36. Cole SJ, Mackenzie H, Ha J, Hanna GB, Miskovic D (2014) Randomized controlled trial on the effect of coaching in simulated laparoscopic training. Surg Endosc 28(3):979–986
      37. Karam MD, Thomas GW, Koehler DM, Westerlind BO, Lafferty PM, Ohrt GT, et al (2015) Surgical coaching from head-mounted video in the training of fluoroscopically guided articular fracture surgery. J Bone Joint Surg Am 97(12):1031–1039
      38. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442
      39. Vassiliou M, Feldman L, Andrew C, Bergman S, Leffondre´ K, Stanbridge D, et al (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg  190(1):107-113
      40. Martin J, Regehr G, Reznick RK, Macrae H, Murnaghan J, Hutchinson C, et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278
      41. Larson L, Williams R, Ketchum J, Boehler M, Dunnington G (2005) Feasibility, reliability and validity of an operative performance rating system for evaluating surgery. Surgery 138(4):640–649
      42. Budrionis A, Augestad KM, Patel HR, Bellika JG (2013) An evaluation framework for defining the contributions of telestration in surgical telementoring. Interact J Med Res 2(2):e14
      43. Bandura A (1977) Social learning theory. Prentice Hall, Toronto
      44. Lave J, Wenger E (1991) Situated learning. Cambridge University Press, Cambridge
      45. Dube´ L, Bourhis A, Jacob R (2005) The impact of structuring characteristics on the launching of virtual communities of practice. J Org Change Mgmt 18(2):145–166
      46. More MG (1997) Theory of transactional distance. In: Keegan D (ed) Theoretical principles of distance education. Routledge, Abingdon, pp 22-38
      47. Branch RM (2009) Instructional design: the ADDIE approach. Springer Science & Business Media, Berlin
      48. Asad E, Hassan RB, Sherwani EF (2014) Instructional models for enhancing the performance of students and workforce during educational training. Academ Arena 6(3):27–31
      49. Singh P, Aggarwal R, Tahir M, Pucher PH, Darzi A (2015) A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills. Ann Surg 261(5):862–869
      50. Wyles S, Miskovic D, Ni Z, Darzi A, Valori R, Coleman M, et al (2016) Development and implementation of the Structured Training Trainer Assessment Report (STTAR) in the English National Training Programme for laparoscopic colorectal surgery. Surg Endosc 30(3):993–1003
      51. Vincent C, Moorthy K, Sarker S, Chang A, Darzi A (2004) Systems approaches to surgical quality and safety: from concept to measurement. Ann Surg 239(4):475–482
      52. Warth L, Bogen E, Lindsetmo RO, Patel H, Augestad KM (2015) Collaboration in surgical training: a qualitative study of mentoring laparoscopic surgeons by using videoconference in northern Norway. eTELEMED: the seventh international conference on eHealth, telemedicine, and social medicine. IARIA, Wilmington, DE, pp 6-9
      53. Warth L, Patel H, Bogen E, Gjessing P, Augestad KM, Ponsky T, et al (2015) Video-recorded observations of surgical telementoring. Int J Adv Life Sci 7(3/4):1–8.