Surgical Simulation: The Value of Individualization

Primary Authors

Greta V. Bernier, MD:  University of Washington Medical Center, 1959 NE Pacific St, Seattle WA, 98195.  (206) 598-3300, gbernier@health.usf.edu.

Jaime E. Sanchez, MD:  University of South Florida, 1 Tampa General Circle, Suite F-145, Tampa, FL, 33606.  (813) 844-4545, jasanche@health.usf.edu.

---

Introduction

Surgical simulation programs have the potential to broadly benefit numerous healthcare stakeholders. Patients, hospital, and surgeons themselves all stand to gain, either directly or indirectly, from skills that can be learned and refined through simulation. This is especially true in communities with residencies and fellowships where proficiency of surgical technique remains in development for novice surgeons. With public pressure for increased oversight of surgeons in training, prevention of medical errors, reductions in health care costs and decreased work hours, current residents have a much different training experience today than in years past^1-4. Limited work hours have led to reduced exposure of surgical trainees to operative procedures and the ability to practice those associated technical skills⁵. Bridging this educational gap will become more important than ever as proficiency based evaluation overtakes the traditional time based model^6-8.  Many now look to simulation as the possible solution for deficits emerging in current surgical training and as a way to improve patient safety and possibly overall costs.

Benefits of Simulation

Surgeon Benefits

Several small studies have shown improvement in resident performance after simulation training. Seymour et al randomized junior residents to virtual reality training compared to standard residency training and evaluated them, in a blinded fashion, on their performance of a laparoscopic cholecystectomy⁹. Those who underwent virtual reality training were 29% faster, nine times less likely to have a stall or lack of progress during the case and, most importantly, five times less likely to cause gallbladder injury or cauterize unintended structures.  In addition, all attending take-overs occurred in the traditional training group. This exemplifies that skills learned through simulation may lead to greater trainee autonomy in the operating room and in turn increased opportunity for more advanced practice.

While Seymour and others have used higher fidelity systems such as virtual reality, low fidelity simulators are also associated with improved resident operative performance. One such model, the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) was ultimately adopted for Fundamentals of Laparoscopic Surgery curricula (FLS Program, SAGES, Los Angeles, CA)⁷. Increased use of this simulator was associated with improved evaluation scores for each of five specific tasks. More importantly, MISTELS scores were highly correlative with technical skill in the operating room, validating this system both as a training model but also a means of assessment.

Simulation allows residents to move more quickly through the learning curve of basic skills and progress to greater depth of understanding of surgical technique and decision making. The primary benefit is obviously to the resident, however simulation may also counteract the increased operative time that comes with training residents. Babineau et al looked at operative times for surgeons at a community hospital before and after addition of a third year resident to their service¹⁰. Operative times were monitored for inguinal hernia repair, laparoscopic cholecystectomy, carotid endarterectomy and partial colectomy. There were statistically significant increases in operative times for all four procedures with the addition of a resident, ranging from 8 to 44 minutes increased time. With an estimated $53 million yearly operating room cost for surgical resident training in the US (based on 1,014 residents in 1997), it is likely that effective simulation training can help recoup some of this added time and expense¹¹. However, validated, standardized training curricula which can be used for proficiency training, as in other technical fields were simulation is successfully used, are lacking for most general surgery operations.

Hospital Benefits

Hospitals continue to be under pressure to decrease costs while maintaining excellent patient care standards. The cost of a typical OR minute is estimated at ~$15-20 for hospitals in the United States¹².  However, this is highly variable between institutions, countries, and even types of operation. In this regard, simulation training has been shown to decrease operative time which in turn reduce both OR and anesthesia costs, as well as increasing surgeon productivity^7,9,13. Decreased operative time also benefits patients by means of reduced charges; as each operative minute in the US has been estimated to cost patients $62 dollars on average (range: $22 to $133/min)^14,15.

A decrease in complication rate is also advantageous to hospitals in terms of both costs and community ratings. With the shift toward bundled payments for all operative and peri-operative care, hospitals will be absorbing the increased cost from patient complications. In a study focusing on catheter related blood stream infections (CRBSI), Cohen et al trained 69 of their second and third year residents over a 12-month period¹³. After completion of training these residents rotated in the medical ICU.  The authors compared the rate of CRBSI before and after training intervention, with an observed decreased rate of infection from 4.2/100 MICU CVC admissions (11 infection in 239 CVC patients) to 0.42/100 admissions (1 infection in 238 CVC patients), therefore preventing approximately 9.95 CRBSIs in the year after intervention. The authors calculated a cost of approximately $82,000 and 14 additional hospital days per infection. As the annual cost to train their residents was approximately $112,000, they reduced their overall annual expense by more than $700,000. Studies like these cannot be directly compared to surgical, laparoscopic or robotic trainers, however, they should be considered in the equation of cost value for simulation centers.

Patient Benefits

Patient health benefits are less well established as most studies focus on intraoperative measures and there are few linking surgical simulation training directly to subsequent complication rates. Therefore, much of the patient health benefit must be inferred based on decreased anesthetic time and the assumption that fewer surgeon errors on a simulated device would yield fewer errors in the operating room.

In a recent systematic review of the literature, Cox et al specifically looked at studies that showed patient benefit as a result of simulation training programs¹⁶. They identified only 12 articles out of an original search of over 200,000 articles that linked simulation with improved patient outcome. These studies focused primarily on non-operative simulation: CVC placement (n=3), newborn outcome after labor and delivery simulation (n=4), team skills building (n-2), and mock pediatric codes (n=1). Only two studies evaluated operative simulation and patient outcomes, with only one focusing on a general surgery operation:  laparoscopic totally extraperitoneal (TEP) inguinal hernia repair.  This highlights the paucity of current data on long term outcomes after initiation of operative simulation curricula.

In their evaluation of the TEP inguinal hernia repair, Zendejas et al randomized 50 residents to standard practice (SP) or mastery learning with addition of simulation (ML)¹⁷. After performing a median of 16 simulations, ML residents “achieved mastery,” as measured by decreased operative time, improved performance, decreased intra-operative and post-operative complications and decreased need for post-operative overnight stay.  Post-operative complications included urinary retention, seroma or hematoma formation and surgical site infection.  While this study did not include long term follow-up, it is unique in that it measured and demonstrated improvement in post-operative outcomes, not just intraoperative measures, and therefore a direct patient benefit from surgical simulation.

Costs of Simulation

Most institutions recognize the overall benefits of simulation, however concern exists regarding the capital costs of developing a simulation center. The reported average cost to start a well-equipped lab is estimated at $450,000 (ranging from $100,000 to several million dollars)¹⁸. Up-front costs include space development or remodeling if necessary and purchase of trainer systems which can range from very simple models and multipurpose laparoscopic box trainers to more sophisticated virtual reality endoscopic, laparoscopic or robotic platforms (Tables 1 & 2). In addition, a simulation center requires between $12,000 to $300,000 annually for consumable materials, such as surgical supplies, maintenance, upgrades as well as other administrative overhead costs¹⁸. However, it may cost as little as a few thousand dollars to obtain simple laparoscopic trainer boxes and supplies if already available laparoscopic equipment can be utilized for training sessions.  Overall, however, the investment in comprehensive simulation training can be daunting for many institution even when possible benefits are clear.

Optimizing Cost-Benefit Balance

Non-Hospital Sources of Funding/Cost sharing

While support from the primary hospital/department is necessary for financial sustainability, several strategies have been employed by simulation centers to help offset the cost of a simulation lab. One such strategy is sharing the simulation center among specialties. Simulation has benefits for many procedural specialties including anesthesia, gastroenterology and emergency medicine as well as non-procedural specialties.  Improvement in non-surgical skills such as auscultation of heart sounds and critical care management have also been demonstrated¹⁹.  Moreover, simulation centers are also excellent resources for team skills training, which are known to improve operating room teamwork, communication, efficiency, regulatory compliance, and overall patient outcomes²⁰. Interestingly, increased patient satisfaction and willingness to recommend the institution have also been shown to improve as well. These results again have the potential to increase hospital revenue while simultaneously decreasing costs by means of decreased operative time and complications.

Hospital partnership with a school of medicine or nursing is also a consideration for earlier implementation of hands-on training for students and a potential source of financial support derived from tuition. Currently 69% of simulation centers are being used by medical students²¹. This partnership benefits the school by increasing marketability to future students and providing a competitive edge in national rankings.

Many simulation centers are supported by the medical device or pharmaceutical industries, with 68% of centers receiving direct funding from these sources²¹. In fact, 42% of centers cite industry as their major source of funding²². Donations of instruments or other consumable materials can also decrease expense. Clear expectations must be set for this kind of partnership to succeed, especially when involving a public institution. Each center must evaluate and consider their institutional goals and ethics when considering a partnership with industry.

Other potential sources for outside funding include tuition from maintenance of certification courses and use of the simulation center by outside providers for a fee.

Choice of Simulation Equipment

When setting up a simulation lab institutions must challenge the assumption that ultra-realistic, and generally more expensive, simulators are better than lower fidelity options. There are, in fact, benefits to low fidelity models that may be lost with certain higher fidelity systems such as haptic feedback and use of actual operative (laparoscopic or open) instruments and equipment. Several small studies have compared the benefits of low fidelity versus high fidelity simulators without showing superiority in various skills^23-26. Particularly relevant are two direct comparisons between laparoscopic box trainers and virtual reality laparoscopic trainers.

Munz et al randomized 24 medical students to box training (Simulations Trainer; Germany), LapSim (Surgical Science, Gothenburg Sweden) training and no training (8 individuals per group)²⁵. Those assigned to LapSim or box training underwent three weekly instructor supervised training sessions lasting 30 minutes each. All participants underwent blinded pre- and post-test assessments as well as a quantitative assessment by the Imperial College Surgical Assessment Device (ICSAD)²⁷. Quantitative evaluation was focused on economy of movement as measured by total number of hand movements, total distance traveled per hand and total time to perform the required task. There were no differences identified in pre-test parameters between the three groups. Results revealed that both pre-trained groups had significant improvement in post test scores as compared to the non-trained group, however without significant difference between the two training modalities.

Madan et al also looked at differences in skill acquisition between medical students trained with a box trainer, LTS 2000 (Laparoscopic Training Simulator 2000; Realsim Systems, LLC Albuquerque, NM), versus those trained with a box trainer as well as a virtual reality trainer, MIST-VR (Minimally Invasive Surgery Trainer-Virtual Reality; Medical Education Technologies, Inc., Sarasota, FL)²⁶.  There was no significant difference in skill level of those trained with the both modalities as compared to those using the box trainer alone.

These studies should not be misinterpreted as demonstrating no benefit to higher fidelity simulations. Simply, more realistic systems may not be necessary to achieve some of the same skills improvement that can be learned with low fidelity options.  Simulator choice should be based on the desired outcome. For example, Sidhu et al compared a plastic tube model to cadaveric brachial artery for simulating vascular anastomoses²⁸. Participants were evaluated after training by creation of an anastomosis in a pig model. These authors found statistically better “final product” scores for those trained on the higher fidelity cadaveric artery model. For this outcome the higher fidelity model was superior.

In a similar finding, Gomez et al compared endoscopic training of PGY-1 residents with the GI Mentor virtual reality simulator (Simbionix Corp., Cleveland, OH), to the Kyoto Kagaku simulator (Kyoto Kagaku Co. Ltd., Kyoto, Japan), composed of a flexible rubber colon in a plastic abdomen²⁹.  Residents were either trained with the GI Mentor alone, the Kyoto simulator alone or both. They were then assessed during a live patient colonoscopy using the Gastrointestinal and Endoscopic Surgeons-Colonoscopy (GAGES-C) tool evaluating scope navigation, strategies for scope advancement, clear lumen, user of instrumentation and overall quality. The GI Mentor and combined GI Mentor/Kyoto trained groups had significant improvement in post-test scores as compared to pre-test. Specifically these groups had significant improvement in scope navigation and overall quality of examination. The group trained on the Kyoto model alone showed post-test improvement as well as increased time with clear endoscopic lumen, however neither of these findings were statistically significant. The authors concluded that the Kyoto simulator while useful overall, may be better suited for more advanced learners, as it was more difficult to navigate. Therefore, again, the higher fidelity model was superior for this desired outcome.  Subsequently, GI Mentor has been utilized in the validated Fundamentals of Endoscopic Surgery curriculum developed by SAGES⁸.

Reuse/Recycle

Consumable materials and supplies can be restocked from expired or unused operating room supply with permission from the institution. This requires both an understanding with operating room staff, as well as a dedication to save uncontaminated instruments or equipment that would otherwise be discarded.

Conclusions

The benefits of surgical simulation are unquestionable for many aspects of surgery, however the high cost of setting up and running a comprehensive center continues to be a formidable barrier to their establishment. Based on the compelling literature supporting simulation in surgical education and its resulting benefits for patients and health care institutions, we can no longer afford to train without simulation. With cost saving and cost sharing strategies, thoughtful choice of equipment, and in some cases partnership with industry, training institutions can minimize the expense of a simulation center while maintaining the undeniable value of its benefits when used appropriately.

---

Table 1: Costs of Common Simulation Platforms

Simulation Equipment	Estimated Cost(s)	Details
Fundamentals of Laparoscopic Surgery Trainer Box (Limbs & Things, Savannah, GA)30,31	$1,130-$6,200	BASE PRICE INCLUDES: Trainer Box, TV Camera, Light Strip, Power Cable, Simulated Skin, 2 x String & Alligator Clips, 2 x Trocar, 1 x Peg Board, 6 Triangles, 1 x Jumbo Clip Retainer, 25 x Single Circle Gauze, 25 x Double Circle Gauze, 3 x Foam Organs, 1 x Suture Block, 50 x Penrose Drains
MIST VR (Mentice, Gothenburg, Sweden)32,33	not sold in US
LapSim Basic (Surgical Science, Minneapolis, MS)34	$75,000	Provided modules: Basic Skills, Task Training, Camera Anatomy Training; Additional modules ($19,000 each): Appendectomy, bariatrics, cholecystectomy, gynecology, hysterectomy, nephrectomy, suturing & anastomosis, VATS lobectomy
Lap Mentor (Simbionix, Cleveland, OH)35,36	$65,000 – $200000	Prie depends on software purchased; Available Modules: basic lap skills acquisition, basic laparoscopic procedure training, advanced procedure training (ventral hernia, inguinal hernia, gastric bypass, sigmoidectomy, nephrectpmy, hysterectomy,vaginal cuff closure, lung lobectomy)
Lap Mentor Express (Simbionix, Cleveland, OH)	Up to $165,000	Desktop portable model, price depends on software purchased (see above)
Robotic surgical simulator (RoSS; Simulated Surgical Systems LLS, San Jose, CA)40,41	>$100,00042	Used in Fundamental Skills of Robotic Surgery Curriculum.43,44 Module Categories: Orientation, motor skills, basic surgical skills, intermediate surgical skills.
Mimic dV-Trainer (Mimic Technologies, Seattle, WA)45,46	$100,00042	Module Categories: Overview & basic skills, advanced surgical skills training, procedure-specific content, team training.
Da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA)47	$85,00042	Attaches directly to da Vinci Surgical System console. Module Categories: EndoWrist manipulation, camera & clutching, fourth arm integration, system settings, needle control & driving, energy & dissection.
GI-Mentor (Simbionix, Cleveland OH)37,38	Up to $160,000	Price depends on modules purchased; Modules available: Fundamental skills, cyberscopy, upper GI endoscopy, lower GI endoscopy, ERCP, emergency bleeding, flexible sigmoidoscopy, EUS; Additional GI-BRONCH modules available on same plateform: Essential bronchoscopy, diagnostic bronchoscopy, essential EBUS, emergency bronchoscopy.
GI-Mentor Express (Simbionix, Cleveland, OH)38,39		Desktop simulator platform, price depends on modules purchased (see above).
Trauma Man (Simulab Corporation, Seattle, WA)48,49	$24,000	TraumaMan with Surgical Ab Instrument Kit
CentraLine Man (Simulab Corporation, Seattle, WA)50	$1,501 -$3,262	CentralineMan (with/without Articulating Head)
Extended warranty	Varies by equipment	Depending on coverage selected

*References denote validation studies for above simulation platforms.

 

 

Table 2: Sample of additional equipment and consumable materials

Operating Equipment and Materials
	Training laboratory computer		$800.00
	Television monitor (19″ LED Television)		$200.00	This 19” Television will allow you to use the FLS Trainer Standard that is provided with a TV camera.
	Portable video recording devices		$200.00
	Stapler (Endo GIA 12 mm stapler)		$1,110.00
	Stapler reloads (60 mm stapler reloads)		$3,725.00
	Animal part resections (distal colon including anus and rectum  each)		$30.00
	FLS/task trainer disposables
		Triangles for Peg Board (pack of 6)	$11.00	6 colored Triangle replacements for FLS Task 1: Peg Transfer; 3 of each color, color may vary.
		Gauze Pads with Single Circle (pack of 100)	$60.00	FLS Task 2: Precision Cutting test.
		Gauze Pads with Double Circle (pack of 100)	$60.00	FLS Task 2: Precision Cutting.
		Foam Organs (pack of 15)	$62.00	Foam Organs with three appendages for practice and testing of FLS Task 3: Ligating Loop.
		Single Use Ligating Loop (pack of 3)	$95.00	Single-Use Ligating Loops for practicing and testing FLS Task 3: Ligating Loop.
		Penrose Drains (pack of 100)	$82.00	100 Penrose Drains for practicing and testing FLS Task 4: Suture with Extracorporeal Knot and FLS Task 5: Suture with Intracorporeal Knot.
		Suture (pack of 50)	$180.00	A 50 piece pack of 2/0 Silk 90cm Sutures with a 26mm ½ Circle Taperpoint Needle for practicing and testing of FLS Task 4: Suture with Extracorporeal Knot and FLS Task 5: Suture with Intracorporeal Knot.
	Suture Blocks (1 each)		$15.00
	Clips for FLS Trainer (1 each)		$12.00
	Laparoscopic Trocars (2)		$25.00
	Skin Frame for FLS Trainer (1 each)		$136.00
	Trauma Man Replaceable Skin Sets		$400.00	1 chest, 2 Abdomen, 2 neck, 2 pleural patch, 4 cric
	CentralLine Man Replaceable Tissue		$487.00	Neck pad

* Costs estimated from the Center for Advanced Medical Learning and Simulation (CAMLS), Tampa, Florida.

---

Disclosures

Dr. Jaime Sanchez has no conflicts of interest or financial ties to disclose.  Dr. Greta Bernier has no conflicts of interest or financial ties to disclose.

References

1. Institute of Medicine (IOM). To Err is Human: Building a safer health system. Kohn LT, Corrigan JM, Donaldson S, eds. Washington, D.C.: National Academy Press.
2. Dennis BM, Long EL, Zamperini KM, Nakayama DK. The effect of the 16-hour intern workday restriction on surgical residents’ in-hospital activities. J Surg Educ 2013;70(6):800-5.
3. Antiel RM, Reed DA, Van Arendonk KJ, et al. Effects of duty hour restrictions on core competencies, education, quality of life and burnout among general surgery interns. JAMA Surg 2013;148(5)448-55.
4. Barden CB, Specht MC, McCarter MD, et al. Effects of limited work hours on surgical training. JACS 2002;195(4):531-8.
5. Sadaba JR, Urso S. Does the introduction of duty-hour restriction in the United States negatively affect the operative volume of surgical trainees? Interact Cardiovasc Thorac Surg. 2011;13(3):316-9.
6. Willis RE, Richa J, Oppelz R, et al. Comparing three pedagogical approaches to psychomotor skills acquisition. Am J Surg 2012;203(1):8-13.
7. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg 2004;240:518-28.
8. Vassiliou MC, Dunkin BJ, Fried GM, et al. Fundamentals of endoscopic surgery:  creation and validation of the hands-on test. Surg Endosc 2014;28(3):704-11.
9. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-64.
10. Babineau TJ, Becker J, Gibbons G, et al. The “cost” of operative training for surgical residents. Arch Surg 2004;139:366-70.
11. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177(1):28-32.
12. Park KW, Dickerson C. Can efficient supply management in the operating room save millions? Curr Opin Anaesthesiol 2009;22:242-8.
13. Cohen ER, Feinglass J, Barsuk JH, et al. Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Sim in Healthcare 2010;5(2):98-102.
14. Macario A. What does one minute of operating room time cost? J Clin Anes 2010;22-233-6.
15. Available at: https://www.akrongeneral.org/portal/page?_pageid= 153,10351153&_dad=portal&_schema=PORTAL. Accessed September 23, 2015
16. Cox T, Seymour N, Stefanidis D. Moving the needle: Simulation’s impact on patient outcomes. Surg Clin N Am 2015;95:827-38.
17. Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair. Ann Surg 2011;254:502-11.
18. Henry B, Clark P, Sudan R. Cost and logistics of implementing a tissue-based American College of Surgeons/Association of Program Directors in Surgery surgical skills curriculum for general surgery residents of all clinical years. Am J Surg 2014;207(2):201-8.
19. Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Med Educ 2012;46:636-47.
20. Forse RA, Bramble JD, McQuillan R. Team training can improve operating room performance. Surgery 2011;150(4): 771-8.
21. Kapadia MR, Darosa DA, Macrae HM, et al. Current assessment and future directions of surgical skills laboratories. J Surg Educ 2007;64(5):260-5.
22. Gould JC. Building a laparoscopic surgical skills training laboratory: resources and support. JSLS 2006;10(3):293-6.
23. Anastakis DJ, Regehr G, Reznick RK, et al. Assessment of technical skills transfer from the bench training model to the human model. Am J Surg 1999;177(2):167-70.
24. Grober ED, Hamstra SJ, Wanzel KR, et al. The educational impact ofbench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg 2004;240(2):374-81.
25. Munz V, Kumar BD, Moorthy K, et al; Laparoscopic virtual reality and box trainers. Surg Endosc 2004;18:484-94.
26. Madan AK, Frantzides CT. Substituting virtual reality trainers for inanimate box trainers does not decrease laparoscopic skills acquisition. JSLS 2007;11(1)87-9.
27. Datta V, Chang A, Mackay S, et al. The relationship between motion analysis and surgical technical assessments. Am J Surg 2002;184:70-3.
28. Sidhu RS, Park J, Brydges R, et al. Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition. J Vasc Surg 2007;45(2):343-9.
29. Gomez PP, Willis RE, Van Sickle K. Evaluation of two flexible colonoscopy simulators and transfer of skills into clinical practice.
30. FLS Box Trainer cost: https://www.fls-products.com/products/fls-trainer-box-with-tv-camera-with-tasks; Accessed September 11, 2015.
31. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg 2004;240:518-28.
32. Gallagher AG, McClure N, McGuigan J, et al. Virtual reality training in laparoscopic surgery: A preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy 1999;31(4)310-3.
33. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann of Surg 2002;236(4):458-64.
34. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of virtual reality training on laparoscopic surgery: randomized controlled trial. BMJ 2009;338:b1802.
35. Mann T, Gillinder L, Szold A. The use of virtual reality simulation to determine potential for endoscopic surgery skill acquisition. Minim Invasive Ther Allied Technol 2014;23(4):190-7.
36. Shanmugan S, Leblanc F, Senagore A, et al. Virtual reality simulator training for laparoscopic colectomy: what metrics have construct validity? Dis Colon Rectum 2014;57(2):210-4.
37. Poulose BK, Vassiliou MC, Dunkin BJ, et al. Fundamentals of endoscopic surgery cognitive examination: development and validity evidence. Surg Endosc 2014;28(2):631–638.
38. Accessed at: https://simbionix.com/wp-content/pdf/Brochures/GI_Mentor_Brochure_06_2015-Web.pdf. Accessed on September 30, 2015.
39. Mueller CL, Kaneva P, Fried GM, et al. Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test. Surg Endosc 2015 Jul 3. [Epub ahead of print].
40. Seixas-mikelus SA, Kesavadas T, Srimathverravalli G, et al. Face validation of a novel robotic surgical simulator. Urology 76;357-60.
41. Seixas-mikelus S, Stegemann AP, Kesavadas T, et al. Content validation of a novel robotic surgical simulator. BJU Int 2011;107:1130-5.
42. Bric JD, Lumbard DC, Frelich MJ, et al. Current state of virtual reality simulation in robotic surgery training: a review. Surg Endosc 2015 Aug 25 [Epub ahead of print].
43. Raza SJ, Froghi S, Chowriappa A, et al. Construct validation of the key components of fundamental skills of robotic surgery (FSRS) curriculum – a multi-institution prospective study. J Surg Educ 2014;71:316-24.
44. Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology 2013;81:767-74.
45. Kenney P, Wszolek MF, Gould JJ, et al. Face, content and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 2009;73:1288-92.
46. Perrenot C, Perez M, Tran N, et al. The virtual reality simulator dV-Trainer is a valid assessment tool for robotic surgical skills. Surg Endosc 2012;26:2587-93.
47. Connolly M, Seligman J, Kastenmeier A, et al. Validation of a virtual reality-based robotic surgical skills curriculum. Surg ENdosc 2014;28:1691-4.
48. Marshall RL, Smith JS, Gorman PJ, et al. Use of a human patient simulator in the development of resident trauma management skills. J Trauma 2001;51(1):17-21.
49. Block EF, Lottenberg L, Flint L, et al. Use of a human patient simulator for the advanced trauma life support course. Am Surg 2002;68(7):648-51.
50. Cohen ER, Feinglass J, Barsuk JH, et al. Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Sim in Healthcare 2010;5(2):98-102.