Guidelines for the Introduction of New Technology and Techniques

This document was reviewed and approved by the Board of Governors of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Aug 2014.

Dimitrios Stefanidis, MD, PhD, FACS, FASMBS, Robert D Fanelli, MD, Ray Price, MD, William Richardson, MD
SAGES Guidelines Committee

Preamble

These guidelines for the surgical introduction of new technologies and techniques are systematically developed statements designed to assist surgeons when making decisions about the appropriate adoption of modified or new devices and procedures in their practice. The statements included in these guidelines are the product of a systematic review of published literature and expert opinion, and the recommendations are explicitly linked to the supporting evidence. The strengths and weaknesses of the available evidence are highlighted and expert opin­ion sought where the evidence is lacking.

Disclaimer

These guidelines are intended to indicate preferable approaches for adopting medical technologies and techniques based on existing evidence or a consensus of expert opinion when little or no data are available. Guidelines are applicable to all physicians who are learning to adopt new procedures and technology without regard to specialty training or interests and are intended to indicate the preferable, but not necessarily the only, acceptable approaches. In light of the complexity of the healthcare environment, these guidelines are intended to be flexible. Given the wide range of specifics in any new procedure or technology, the surgeon must always choose the course best suited to the individual patient, taking into account the context at the moment of decision. Guidelines are developed under the auspices of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and its various committees and approved by its Board of Governors. Each clinical practice guideline has been systematically researched, reviewed, and revised by the Guidelines Committee (Appendix 1) and appraised by external reviewers. The recommendations are therefore considered valid, based on the data available at the time of their production. Each guideline is scheduled for periodic review to allow incorporation of pertinent new developments in medical research, knowledge, and practice.

Introduction

New technology and techniques (NT&T) are constantly being introduced into surgical practice, and recent years have seen an unprecedented growth in innovation. Adoption of these advances has been at times uncoordinated and undisciplined, leading to an increased incidence of complications [1] and at times better controlled, with efficacious outcomes [2, 3]. While several organizations have attempted to develop criteria and processes for the safe introduction of new technology and procedures [2, 4-10], no uniformly accepted criteria and processes are currently in place in the US. The need for such a guideline is highlighted by the results of a recent Delphi survey of the SAGES membership that determined the top priorities in minimally invasive and endoscopic surgery research. Two of the top 10 research questions, “What is the best method for incorporating new techniques and technology for surgeons of variable levels of experience or training?” and “What are the costs associated with the introduction of new technologies and how can they be minimized?,” were directly related to the introduction of NT&T [11]. Given that SAGES has been in the forefront of new technology and procedure adoption since the introduction of minimally invasive techniques in surgery, its leadership determined that a national guideline on this subject would be valuable to its membership and surgeons in general and serve the US public well. The SAGES guidelines committee was, therefore, tasked to develop these guidelines based on the available evidence. It should be noted that over the years, SAGES has produced related position statements and guidelines [12, 13] that will be referenced in the present document.

Innovation is extremely important in surgery and has served our patients well. As stated by Riskin et al, “most surgeons innovate on a daily basis, tailoring therapies and operations to the intrinsic uniqueness of every patient and their disease” [14]. Without processes that promote innovation, significant progress in patient care, such as minimally invasive surgery, may never have been realized. From this perspective there is little disagreement that processes governing the introduction of new technology and procedures should not stifle the adoption of innovative technology and techniques. On the other hand, uncoordinated introduction of new technology and procedures in clinical practice has been demonstrated repeatedly to be associated with an increased incidence of patient complications and poor outcomes [1, 15] and, therefore, does not serve the US public well. Further, the implementation of NT&T into surgical practice may be unduly influenced by industry’s financial incentives, physician drive to remain competitive, the lure of the procedure or technology, hospital financial pressures, and the lay public’s impressions, instead of a concern for optimal clinical outcomes [8, 16]. These guidelines are, therefore, intended to both safeguard patient interests and promote the beneficial effects of innovation in patient care. While these guidelines address surgeons who practice in the US, they may also apply to surgeons outside US borders.

Two of the first issues we need to address in these guidelines are (1) the definition of the terms technology, technique, and new, and (2) what constitutes a significant change in technology or procedure. Even though the term technology has elsewhere been used to describe both devices and procedures [17], in these guidelines, a distinction is made between the term technology, which we consider synonymous to devices, and the term technique, which we consider synonymous to procedures.  Our definitions are in agreement with the succinct descriptions of what constitutes new interventional technology and procedures proposed in previous publications [10, 12]. In the context of these guidelines, the term new, however, can suggest several meanings and should be explicated here. For example, a new stapler that may have a different handle or firing mechanism cannot be placed in the same category as a new procedure that has not been previously performed in a region or country.  The issue is further complicated by the reality that a new procedure may also be one that is novel and under development versus one that is well established but new to surgeons who implement it in their practice. To address these issues and to narrow the wide spectrum of what constitutes new or a significant change in a device or procedure, the present guidelines will employ the following 4 definition categories, developed based on input received during interviews with SAGES board members:

  1. Modified Device: existing device the surgeon has experience with that has been altered to improve functionality or performance e.g., a modified stapler, a new mesh etc.
  2. New Device: product of disruptive innovation or device that has not been previously used by surgeon. Includes modified devices that surgeon has no prior experience with e.g., endoscopic hemoclips, when the surgeon has not used similar clips before.
  3. Modified Procedure: modification of known procedure or technique. Surgeon has experience with similar procedures/techniques e.g., a surgeon experienced with laparoscopic Nissen wants to perform a laparoscopic Toupet fundoplication or a surgeon who performs a laparoscopic bypass wants to adopt laparoscopic sleeve gastrectomy
  4. New Procedure: novel technique that differs dramatically from what surgeon is used to or technique not previously used by surgeon, e.g., POEM vs. laparoscopic myotomy or adaptation of a laparoscopic or robotic procedure by an open surgeon.

These guidelines will only address Food and Drug Administration (FDA)-approved technology in the US and procedures for which some preliminary evidence already exists and are not intended to guide innovators in developing new devices or procedures.

Methods

A systematic literature search focused on adoption of new technology and procedures in the surgical field was performed in MEDLINE in June 2013 by a licensed librarian.  The search strategy was limited to articles related to human subjects published in the last decade and is shown in Appendix 2.

Seventy relevant articles were identified by the literature search.  All retrieved abstracts were reviewed and their methodological quality assessed. Randomized controlled trials, meta-analyses, and system­atic reviews were considered the highest quality. Observational studies and nonsystematic reviews were considered when high level quality evidence was lacking. This initial review identified very few relevant studies (n=22) and indicated that a web search on the topic would be needed. A Google search was then performed that identified an additional 13 relevant articles, which were subsequently retrieved and reviewed by the working group. The references of the available literature were hand searched to identify additional evidence that might have been missed by the original literature search (n=7). Given that the quality of the available evidence on the topic was limited, even after this review, the working group decided to pursue input from the SAGES leadership (the Board); for this purpose the following key questions were developed based on the review of the existing literature and identification of knowledge gaps:

  1. What is the definition of new technology and procedure?
  2. What process should be followed during introduction of new technology and procedures?
  3. What should be the training requirements for surgeons incorporating new devices and procedures into their practice?
  4. How should surgeons be assessed for their readiness to safely implement new technology and procedures in their practice?
  5. What are the criteria that should be taken into account when evaluating the value of new technology and procedures (before introduction)?
  6. What outcomes should be assessed after the introduction of new technology and procedures to prove their safety and effectiveness?
  7. What should be included in the consent when patients undergo new technology and procedures?
  8. What should be the regulatory requirements for the introduction of new technology and procedures?

Face-to-face or phone interviews with eight SAGES board members were conducted using these questions, and Board members were asked to provide any additional comments or questions that the working group may not have considered.

The feedback from these initial interviews with Board members was used to create a survey for the full SAGES Board (Appendix 3). Participants were allowed to make more than one choice in the majority of questions. The results of this anonymous survey and of the review of the available literature are presented here.

The SAGES guidelines committee assesses both the quality of the evidence and the strength of the recommendation for each of its produced guidelines, according to the GRADE system described in Table 1, using a 4-tiered system for quality of evidence (very low (+), low (++), moderate (+++), or high (++++)) and a 2-tiered system for strength of recommendation (weak or strong). Further definitions are provided by SAGES in “The Definitions Document: A Reference for Use of SAGES Guidelines[18]. Given that the existing evidence on the topic of NT&T introduction was found to be limited and generally of low quality, we also considered the results of the SAGES Board survey when formulating the guidelines in this document. Responses to survey questions where >70% or <30% agreement existed were considered majority agreement or disagreement, respectively, and weighted more in the creation of the presented guidelines. On the other hand, response agreement between 30-70% was considered less definitive and received little consideration. While the recommendations provided in this manuscript may be considered weak according to GRADE, as they are mainly based on expert opinion, the SAGES guidelines committee feels that the presented guidelines represent the best available evidence today to guide surgeons and may spark the generation of additional, better quality evidence over time.

These guidelines have been reviewed and approved by the SAGES guidelines committee and by the SAGES full board of directors.

Survey participation

Out of the 35 SAGES Board members who received the survey, 21 responded (60% RR) and 20 (95%) completed all questions in the survey. The survey remained open for 2 weeks and one reminder was sent after the first week. The majority of SAGES Board members (86%) felt that guidelines for the introduction of NT&T were necessary. The literature also indicates that this need exists, and several organizations have attempted to establish their own criteria for this process [2, 4-10].

In the following section, we will pose several key questions related to the introduction of NT&T, and present guidelines based on both the available literature and the input of SAGES leadership (survey results).

Key Question #1: Which steps should be part of the NT&T introduction process?

Guideline 1:  Surgical societies should provide assessments of new technology and techniques in a timely fashion to practicing surgeons to aid their decision making when contemplating the introduction of new technology and techniques. (++, strong)

 

While for minor device or procedure modifications only familiarization of the surgeon may be necessary, for more substantial changes, a more elaborate introduction process should be followed that may include familiarization, cognitive training, hands-on practice, performance assessment, patient disclosure, proctoring, and local and national outcome monitoring with the new device or procedure. (++, strong)

The literature suggests that perhaps the most important initial step in the introduction of a NT&T is establishment of its efficacy and effectiveness. [17] Currently the best accepted method for this is the generation of health technology assessments (HTAs), which may offer several benefits including optimization of health outcomes, improved quality of care, reduced adverse reactions, elimination of unnecessary/outdated procedures, decreased care spending, and improved overall value of investment [17]. Comparative clinical effectiveness research, a component of HTA, enables the development of comparative information on new and old treatments and technologies that can be very valuable for the decision making of physicians and hospitals when they are considering the introduction of NT&T [17].  The incorporation of a structured process for the introduction of NT&T that includes HTAs has been previously demonstrated to lead to significant cost savings [19], and in some instances it has even facilitated innovation [20]. Realizing the importance of HTAs in surgery, SAGES recently created the technology and value assessment committee (TAVAC) that is tasked to generate such HTAs in minimally invasive surgery and has already produced its first assessments [21]. Recently, consensus criteria were generated for use during HTA development to guide the adoption of NT&T by institutions [22]. A relevant framework that describes the stages of innovation in surgery has been developed by the IDEAL Collaboration, an initiative to improve the quality of research in surgery. This framework categorizes innovation in surgery in 5 stages: Innovation, Development, Exploration, Assessment, and Long-term study and provides recommendations for specific study designs and reporting standards at each stage [23]. These suggestions are underpinned by a series of general principles for design and reporting, which are based on the different questions to be addressed and the challenges faced at each stage in the process [24].

The responses of the SAGES leadership to the question of whether specialty societies should provide their members with frequent and timely assessments of the new technology are shown in Figure 1and were dependent on the magnitude of change. All respondents indicated that this should be done for new devices and procedures, but only 35% agreed on its importance for modified devices; 55% recommended it for modified procedures.

Figure 1: Responses of SAGES leadership to the question: Should specialty societies provide their members with frequent and timely assessments of new technology?

For the purposes of these guidelines, it is assumed that the necessary information required for surgeons to make a decision of incorporation of NT&T in their clinical practice exists (whether that comes from an HTA or from a personal review of the available evidence). Surgeons should assess their practice needs and determine whether an NT&T should be adopted in their practice after review of the available evidence and a systematic gap analysis of their practice [16]. It is also assumed that surgeons incorporating NT&T in their practice have appropriate training and credentials to perform the requisite procedures. According to the literature, the decision-making process should take into consideration the existing need for the new device or procedure; the existing evidence on its safety, efficacy and clinical effectiveness; the associated cost and training requirements; the generation of patient information and consent; and monitoring and evaluation of the NT&T once used in practice [10]. The literature also suggests that surgeons should disclose conflicts of interest (financial involvement, etc.) relating to NT&T to both the hospital and patients [10]. The interviews with SAGES Board members identified the following steps for the introduction of NT&T:

  1. Familiarization with the device or procedure before introduction
  2. Cognitive training in new device or procedure (e.g., indications, patient selection, etc.)
  3. Hands-on practice on appropriate training models before use in patients
  4. Assessment of surgeon ability to perform safely prior to introduction
  5. Full disclosure to patient
  6. Proctoring/preceptorship of initial cases (can be partner who is familiar with device or procedure)
  7. Meticulous recording and monitoring of surgeon outcomes with device or procedure
  8. Regional/national monitoring of outcomes (e.g., with the use of a database)

Figure 2 shows the opinions of SAGES leadership about which of these steps would be necessary during the introduction of NT&T, based on the 4 predefined categories.

Figure 2 SAGES leadership recommendations for steps to take in the introduction of NT&T

The survey results indicated that fewer steps were deemed necessary for the introduction of modified devices or procedures in surgical practice than for the introduction of new devices or procedures.

For modified devices, none of the steps were felt to be necessary by the majority of respondents (<30% agreement) except for familiarization with the device prior to its use (felt to be important by 57% of respondents). An equal percentage of respondents deemed familiarization important for modified procedures, and 43% indicated that outcome monitoring was important.

For new devices, there was majority agreement that familiarization, cognitive training, hands-on practice, performance assessment, patient disclosure, and outcome monitoring were necessary steps during introduction (all >70% agreement). The presence of a proctor (48%) or national monitoring of outcomes (62%) was not deemed necessary by the majority. On the other hand, for new procedures, all steps were considered necessary by responders (>70% in all and >80% in all except performance assessment).

Key Question #2: What elements should be part of training in NT&T?

Guideline 2: Surgeons considering the introduction of new technology and techniques in their practice should have device- or procedure-specific training to decrease learning curve-related complications and thus improve patient safety. (+++, strong) The necessary training steps depend on the degree of novelty/ change and may include informal familiarization of surgeon with the device or procedure before its introduction; review of existing data/literature; the pursuit of expert input; video review of device use or procedure; practice on appropriate simulated, animate, or cadaveric training models; course participation at society meetings; proctoring or tele-proctoring of initial cases; and team training. (+, strong)

Several publications have emphasized the importance of training prior to NT&T introduction. Traditionally surgeons have attended short courses, often offered over weekends, to learn new skills. Such courses in isolation are not considered sufficient to promote safe patient care; however [16], as higher complication rates have been reported for surgeons who participated in an isolated laparoscopic surgery course without additional training [25]. To improve safety, training(which can involve several modalities) is commonly recommended prior to the application of the NT&T to patients [16]. The reasoning behind this approach is based on the evidence that surgeons experience unique, individual learning curves associated with new procedures and that these vary based on their skill and the complexity of the task and may also be impacted by disease process and anatomic variation [16]. The learning curve is generally steep during the early experience with a new modality and coincides with the highest risk for harm to the patient [16]. The issue of increased complications associated with both new devices and procedures as a result of the learning curve has been documented by several studies [15, 26-30]. While a randomized controlled trial that compares training to no training before introduction of NT&T does not exist, it is intuitive that absorbing some or the majority of the learning curve using training models before application to patients may dramatically improve patient safety in the introduction of NT&T. Depending on the type of NT&T, training may be necessary just for the surgeon or for the whole operating room team and should be approached in that manner [5, 8]. Further, collaborative training during early practice of some procedures may be beneficial to the learning curve and patients [2, 3]. The readers of this guideline are referred to another SAGES guideline published in 2010 that addresses post-residency surgical education & training and provides a framework for training in NT&T [12]. Guidelines for continuing medical education (CME) activities that are also applicable to training courses for NT&T have been proposed by the American College of Chest Physicians and emphasize the superiority of multimedia over single-medium interventions, the advantage of multiple instructional techniques versus a single instructional technique, and the importance of multiple exposures (sessions) over a single session. Further, the ACS has developed a five-level model for verifying and documenting surgeons’ participation in educational programs as well as their knowledge and skills [31].

It should also be noted that while proctoring could be very effective and valuable for surgeon training during NT&T introduction, its optimal application to practice is not known. While a defined number of proctored cases is frequently suggested or required [9] such recommendations are usually arbitrary and not based on solid evidence of effectiveness. Further, an important parameter surgeons need to consider is the anticipated number (and frequency) of procedures after introduction of NT&T and its relationship to the learning curve [7]; for procedures with a steep learning curve that are performed infrequently it may be impossible for the surgeon to achieve competence within a reasonable time frame.

The review of the existing literature and focused interviews with SAGES Board members identified the following elements that could be part of training in NT&T:

  1. Informal familiarization of surgeon with device or procedure before introduction
  2. Review of existing data/literature
  3. Pursuit of expert input
  4. Video review of device use or procedure
  5. Practice on appropriate simulated models (e.g., realistic or virtual reality)
  6. Practice on animate models
  7. Practice on cadavers or cadaveric tissues
  8. Participation at courses at society meetings (e.g.,. SAGES, ACS)
  9. Participation in online courses
  10. Completion of formal training (e.g., fellowship)
  11. Proctored initial cases
  12. Tele-proctoring of initial cases
  13. Team training (if applicable)

The elements that were considered important by SAGES board members depending on our pre-defined 4 categories of NT&T are presented in Figure 3.

Figure 3 SAGES leadership recommendations for NT&T training elements

Similar to the results of the previous question, survey results here indicated that fewer steps were deemed necessary for training on modified devices or procedures in surgical practice than for training on new devices or procedures.

For modified devices, only informal familiarization was felt to be necessary by the majority of respondents (76%), while all other steps were deemed unnecessary (all <30% of agreement). Similarly, for modified procedures, only informal familiarization, literature review, and expert input were considered necessary by 67%, 52%, and 48% of responders, respectively. No other step received >30% agreement.

For new devices, there was majority agreement that literature review, expert input, and video review were necessary components for surgeon training (all >70% agreement) while familiarization (62%), simulator practice (57%), training on animate models (48%), and exposure in meeting courses (43%) were considered important by some respondents. The other choices were not deemed necessary (<30% agreement).

For new procedures, literature review, expert input, video review, and practice on simulators achieved majority agreement (>70% agreement), while the other options were chosen less frequently (33-67%), except for online courses and formal training, which were not considered necessary (agreement <30%).

Key Question #3: Who should be responsible for training surgeons in the NT&T?

Guideline 3: While institutions, experienced centers, specialty societies, and industry all play a role in the training of surgeons in new technology and techniques, experienced centers and specialty societies should have the primary responsibility for training in new procedures and devices; industry’s role should be limited to new and modified devices, and all conflicts of interest should be disclosed and minimized. (+, weak)

No guidelines exist currently as to who should be responsible for the training of surgeons in NT&T. Education in NT&T has traditionally been offered by industry with an interest in the NT&T. This inherent conflict of interest has brought increased scrutiny and regulation of industry involvement in educational activities of surgeons and other physicians [32]. As a consequence, today no CME credits can be offered to surgeons when they participate in educational activities supported by industry. However, in the absence of a significant overhaul of the current educational system for practicing surgeons, the existing needs in NT&T education cannot be financed without industry support. Further, many argue that the interaction between surgeons and industry is a prerequisite for innovation that may in fact benefit patients. In addition, a vascular surgery study demonstrated that endovascular AAA repair training of early adopters was mainly accomplished by industry for those surgeons >15 years out of fellowship training [33]. Nevertheless, given current trends, the regulation of industry involvement in the training of physicians is expected to become more stringent over time, indicating that institutional or even state and federal funding sources may be needed to support training in NT&T.

The opinions of SAGES leadership about entities’ responsibility for the provision of NT&T training is shown in Figure 4.

Figure 4 SAGES leadership determination of responsibility for provision of NT&T training

The results indicate that institutions, specialty societies, industry, and experienced centers all play important roles in the education of surgeons in NT&T introduction; however, SAGES leadership overwhelmingly did not think that industry should be involved in procedural training (<30% agreement). The majority of respondents agreed (>70% agreement) that surgeon training in NT&T should be provided in experienced centers and that specialty societies (68%) and industry (58%) should also be involved in the training of new procedures and devices, respectively (but not the reverse). The survey results indicated a limited role for societies (21%) and institutions (24%) for training in modified and new devices, respectively.

Key Question #4: What are the prerequisites for the surgeon introducing the NT&T?

Guideline 4:  Surgeons who introduce a new device or procedure in their practice should have completed relevant surgical training, possess operating privileges in the affected organ system, and be able to address anticipated complications. (++, strong)

Several credentialing guidelines exist that are specialty and/or procedure specific [2, 8, 9, 13]. There is general agreement in the literature that the individuals introducing NT&T in their practice should be credentialed at their local institutions to perform related procedures and have appropriate training and experience  [2, 8, 9, 13].

Based on the literature and interviews, the following elements were identified as prerequisites for surgeons introducing NT&T and were included in our leadership survey:

  1. Credentialing by local institution to operate on the affected organ system
  2. Prior completion of relevant surgical residency
  3. Prior completion of relevant advanced sub-specialty training
  4. Certification in well-validated, relevant performance tests, if existing (eg, FLS for laparoscopy, FES for endoscopy)
  5. Surgeon ability to recognize and treat anticipated complications

Responses varied again based on the degree of change, but to a lesser extent than in previous questions, and are shown in Figure 5.

Figure 5 SAGES leadership suggested prerequisites for surgeons introducing NT&T

For all 4 categories, there was majority agreement (>70%) that residency completion, ability to deal with anticipated complications, and institutional privileges were necessary (with the exception of institutional privileges for modified devices where agreement was 65%). Procedure certification or subspecialty training was not considered as important.

Key Question #5: How should surgeon readiness for the NT&T be assessed?

Guideline 5: For minor modifications of devices and procedures, surgeons should monitor their practice based on self-assessment. The more substantial the change in surgeons’ practice and the higher the risk to the patient, the more important it is that surgeons complete a relevant didactic course, and have their performance objectively assessed and their outcomes monitored by an external entity. (+, strong)

Current practice is based on self-assessment and self-regulation, expecting surgeons to use the highest standards of professionalism when assessing their own education, experience, and confidence in performing the new procedure or using the new technology before requesting privileges and applying it to patients [31]. Given that surgeons often overestimate their performance [34] and self-regulation can fail to achieve the highest required standards for safe practice, several organizations have proposed more objective criteria for surgeon performance assessment. Such criteria typically include documenting appropriate knowledge, skills, and experience with the new procedure along with audits of outcomes after implementation [9, 13]. Along these lines, the American College of Surgeons (ACS) has defined a five-level model for verifying and documenting surgeons’ participation in educational programs as well as their knowledge and skills. These five levels include verification of attendance, verification of satisfactory completion of course objectives, verification of knowledge and skills, verification of preceptorial experience, and demonstration of satisfactory patient outcomes [31]. Several publications have assessed the learning curve for specific procedures [27-29] that could be used as guidelines for assessment and monitoring when surgeons introduce a new procedure into their practice. Given however that learning curves vary significantly from surgeon to surgeon [16], competency-based assessments may be more appropriate to evaluate the readiness of surgeons for the introduction of new procedures.  Few publications have addressed this issue using well-validated methods of standard setting and performance assessment tools and have proposed specific cut-off scores for competent performance of a few specific procedures [35, 36]. Given that such cut-off scores may not exist for new procedures at the time of introduction or may lack adequate level of psychometric evaluation, surgeons can use CUSUM curves or other real-time assessments of performance such as run charts to assess their learning curves and monitor their outcomes [37, 38].

The specific answer choices to this question in our survey included:

  1. Surgeon self-assessment only
  2. Objective assessment of performance by third party (eg, verification of readiness by expert)
  3. Demonstration that the surgeon has completed a relevant short course
  4. Completion of competency-based training and assessment
  5. Monitoring of safety/outcomes only after introduction

Figure 6 SAGES leadership suggestions for assessment of surgeon readiness to adopt NT&T

The responses are shown in Figure 6 and indicate that for modified devices and procedures, only surgeon self-assessment is necessary, with only a minority of responders choosing outcome monitoring after introduction of modified procedures (33%).

For new devices, the completion of a short didactic course (61%), outcome monitoring (61%), and completion of proficiency-based training (33%) were considered important with no choice reaching majority agreement. Self-assessment and objective assessment were not considered necessary (<30% agreement).

While for new procedures, self-assessment was not considered important (11%), outcome monitoring and completion of a course reached majority agreement (>70%). Also considered by the majority of responders were objective assessment of surgeon performance (67%) and completion of competency-based training (55%).

Key Question #6: Who should monitor the introduction of the NT&T?

Guideline 6:  To protect their patients, surgeons should demonstrate the highest level of professionalism and exercise self-assessment and self-regulation when introducing new technology and techniques in their practice. Besides the FDA, which regulates the production and sale of new devices, institutional credentialing and/or new technology committees and the IRB should monitor their introduction in clinical practice. The introduction of novel procedures should be overseen by the credentialing committee and/or the IRB, while the role of specialty societies and new technology committees needs further assessment. (+, strong)

As mentioned previously, self-assessment and self-regulation by surgeons, individually and collectively as a profession, using the highest levels of professionalism, remains key to ensuring patient safety. Given however that a surgeon’s decision to adopt a new procedure or technology may be influenced by numerous factors such as the desire to provide the best care to patients, the lure of the procedure or technology, the drive to remain competitive, or pressures from health care systems, industry, and even the patients themselves [16], several authors have suggested that entities other than surgeons also be involved in monitoring. The following possible answer choices to the question of who should monitor the introduction of NT&T were included in the survey:

  1. Surgeons themselves
  2. FDA
  3. Institutional Review Board (IRB)
  4. Institution credentialing committee
  5. Local new technology and technique committee (should be established if non-existing)
  6. Specialty society
  7. State medical board
  8. Specialty board

The survey responses are demonstrated in Figure 7.

Figure 7 SAGES leadership recommendations for who should monitor the introduction of NT&T

For modification of devices the majority agreed that only surgeons themselves should monitor their introduction while all other choices achieved <30% agreement. Results were similar for procedure modification except that institutional credentialing committees were also favored as a choice for monitoring by 40% of responders.

For new devices, surgeons themselves and the FDA were considered the best options (>70% agreement), while the credentialing committee, new technology committee, and the IRB were perceived as appropriate for monitoring by 60%, 50%, and 45% of responders, respectively. Other choices were not considered necessary.

For new procedures, the credentialing committee of the institution was the most frequent choice (85%), followed by surgeons themselves (55%), the IRB (50%), specialty society (40%), and new technology committee (30%). Other choices were not considered necessary.

Key Question #7: What parameters should be assessed before and after the introduction of NT&T?

Guideline 7:  The effectiveness compared to alternatives, the cost, patient outcomes, and the safety profile of new technology and techniques should always be assessed prior to and after their introduction.. Other parameters such as existing and required resources; benefits to patients, surgeons, and hospitals; existing or anticipated volume of use; barriers to adoption; and whether the anticipated benefits prove real after introduction should also be considered, especially for significant changes in devices and procedures, besides minor modifications. (+++, strong)

As mentioned above, before introduction of NT&T, well-done HTAs tend to be very helpful for the decision making of both surgeons and hospitals. In regards to specific information needed prior to the introduction of NT&T, a survey of hospital executives indicated that the most important factors were increased cost-effectiveness compared with another technology (listed by 58% of respondents), increased efficacy (55%), and a potential decrease in complication rates (45%). The most frequent considerations against adoption were increased complication rates (64%), increased side effects (59%), decreased efficacy compared with an existing technology, and decreased cost-effectiveness (57%) [39]. The literature uniformly suggests that both surgeons and hospitals should monitor the outcomes of NT&T introduction closely [10, 40]. The following parameters were included in the survey:

  1. Safety profile
  2. Cost
  3. Effectiveness compared to alternatives
  4. Patient outcomes (specific to technology and technique)
  5. Required resources (e.g., training of individual and/or team)
  6. Benefits to patient (e.g., patient satisfaction, quality of life)
  7. Benefits to surgeon
  8. Benefits to hospital
  9. Existing or anticipated volume of use
  10. Barriers to adoption (before introduction)
  11. Existing resources (before introduction)
  12. Whether the anticipated benefits prove real after introduction

Survey responses are presented in Figure 8.

For modified devices, the safety profile, cost, and comparative effectiveness were favored by most (>70% agreement), while barriers to adoption and anticipated use were not considered to be important (<30% agreement). Other choices ranged between 47-63% agreement. For modified procedures, majority agreement also existed for patient outcomes, safety profile, cost, and comparative effectiveness, while the other parameters were chosen by 47-63% of responders.

For new devices, majority agreement (>70%) was achieved for all parameters except anticipated use of the device, which was still chosen by most responders (68%). Similarly, for new procedures, majority agreement was achieved in all parameters except anticipated use (68%), benefit to the hospital (68%), and benefit to the surgeon (58%).

Figure 8 SAGES leadership recommendations for parameters to be assessed before and after the introduction of the NT&T

Discussion

In this paper, we present our recommendations for the introduction of NT&T in surgical practice. Our recommendations are based on the available evidence in the literature and on expert opinion of the SAGES Board. We were obliged to use expert opinion, as the quality of the available evidence is extremely limited; while there are numerous publications proposing criteria and processes for the introduction of NT&T, there is hardly any evidence demonstrating the effectiveness of these standards.

A few exceptions to this rule include the study by Spigelman, in which he reported his experience with the implementation of a local policy for the introduction of new interventional procedures. This study demonstrated that despite initial skepticism, the policy facilitated rather than inhibited innovation [20]. Further, McGregor and Brophy at the McGill University Health Centre in Canada have reported that the budget impact of their policies around introduction of new technology led to an estimated $3,000,000 of savings per year [19]. Still, whether one introduction process or approach is superior to another remains to be seen and needs further study. Perhaps these guidelines will encourage additional research and provide needed evidence to delineate the optimal processes during the introduction of NT&T.

The existing literature tends to lump together technology and technique when it proposes recommendations for the introduction of NT&T. These guidelines attempt, for the first time, to separately address the two and to more systematically approach the existing variability in what constitutes new. Our approach proved appropriate, as the results of the survey indicated that the responses varied considerably, depending on our 4 predefined categories (modified devices, new devices, modified procedures, and new procedures). While the spectrum of variability in novelty for devices and procedures is difficult to capture in defined categories, the categories we created for the purposes of these guidelines provide a framework that may help better define the processes needed during the introduction of NT&T. Nevertheless, given the existing variability in novelty and change which directly affect the provided recommendations, surgeons need to first decide to which of the 4 categories the NT&T they are considering incorporating into their practice belongs in order to benefit the most from the provided guidelines.

We realize that in the US, devices are regulated by the FDA, which has specific processes and procedures that must be followed before a new or modified device is introduced to patient care. We still elected to include devices in these guidelines, as many aspects of their introduction are not well defined or regulated; this approach further allows for a direct comparison of requirements between devices and procedures.

The degree of change is also important when considering the training needs of surgeons. While familiarization with a modified device may be good enough in the majority of cases (as indicated by our expert responders), a significantly more elaborate training process may be necessary when the implemented change is significant (such as for a new procedure) and may include participation in formal courses, hands-on practice on various training models, verification of knowledge and skill, proctoring, or tele-mentoring and monitoring of the learning curve. Fortunately, the accumulated evidence on the value of surgical simulators and their expanded use in surgery provides us with a very effective training tool that also enables objective assessment of performance that can enhance training and assessment of surgeons during the introduction of NT&T [41-43]. Further, the established ACS consortium of educational institutes may offer an ideal environment to offer such training [16, 44].

In these guidelines, we focused on the needs of the surgeon during the introduction of NT&T. Given that surgery involves a team, the needs of the other members of the OR team should also be considered by future guidelines. While some of our guidelines may also apply to other OR members, significant differences likely exist that should be addressed separately. The importance of OR team training has been highlighted by several publications, especially as it relates to the introduction of robotic surgery [5, 8].

Our work also highlights the existing biases during the adoption of NT&T that can come from many sources including the patient, surgeon, hospital, society, insurance programs and manufacturers. To address these biases, an objective evaluation of the existing evidence for the new technology or procedure in question is necessary to guide surgeons and institutions. While the best evidence (randomized controlled trials and systematic reviews) should always be sought when available, it may not be feasible or realistic to obtain it in a timely fashion. Further, even the best available evidence may not appropriately address all important aspects of NT&T, as highlighted by some authors [45, 46]. Thus, it is important to realize that the process of NT&T evaluation is and should be ongoing and does not stop at the time of completion of an HTA. It may therefore be prudent for hospitals to develop dedicated committees that provide oversight of the NT&T introduction process, as has been successfully done by some institutions [10]. These committees should likely be different from the IRB, which may not be ideal for the monitoring of these processes, as they are not always related to research.

The provision of updated and timely information on NT&T is highly desirable, as in its absence surgeon decision making is impaired.  Our expert consensus was that the development of technology assessments should be done by medical societies with the best interest of the patient in mind. Such a process would provide appropriate information to practicing surgeons when they are considering the adoption of a new technique or device.  SAGES has created a new committee to address this need, and initial assessments have already been produced [21] It is important to note that the lack of timely information is considered one of the most important barriers to an optimal decision-making process during NT&T adoption by hospital administrators [39] and probably also applies to surgeons.

Careful monitoring of outcomes with a new device or procedure and meticulous reporting in a database that allows comparisons are also perceived as very important.  Such monitoring should be performed by the surgeon and the institution but also by societies who can provide guidance to their membership when needed.  Databases such as NSQIP can be very useful for surgeons. It should be noted, however, that while this process may be ideal, its application to daily practice may not be straightforward. An example can be found in the efforts of NOSCAR to create a registry of all NOTES human cases. The development of such a registry proved difficult due to incomplete voluntary reporting of cases, lack of financial resources to support the registry, and performance of a significant number of cases outside the US that were not captured by the registry. [3] This example also highlights the importance, in the era of globalization, of ensuring that processes be applicable internationally.

The guidelines provided throughout this document aim to address the existing needs of surgeons for the safe introduction of NT&T. Expert opinion supports the value of the proposed guidelines, but empirical data is also needed to confirm their appropriateness.

Summary of Recommendations

Guideline 1:

Surgical societies should provide assessments of new technology and techniques in a timely fashion to practicing surgeons to aid their decision making when contemplating the introduction of new technology and techniques. (++, strong)

While for minor device or procedure modifications only familiarization of the surgeon may be necessary, for more substantial changes, a more elaborate introduction process should be followed that may include familiarization, cognitive training, hands-on practice, performance assessment, patient disclosure, proctoring, and local and national outcome monitoring with the new device or procedure. (++, strong)

Guideline 2:

Surgeons considering the introduction of new technology and techniques in their practice should have device- or procedure-specific training to decrease learning curve-related complications and thus improve patient safety. (+++, strong) The necessary training steps depend on the degree of novelty/ change and may include informal familiarization of surgeon with the device or procedure before its introduction; review of existing data/literature; the pursuit of expert input; video review of device use or procedure; practice on appropriate simulated, animate, or cadaveric training models; course participation at society meetings; proctoring or tele-proctoring of initial cases; and team training. (+, strong)

Guideline 3:

While institutions, experienced centers, specialty societies, and industry all play a role in the training of surgeons in new technology and techniques, experienced centers and specialty societies should have the primary responsibility for training others in new procedures and devices; industry’s role should be limited to new and modified devices, and all conflicts of interest should be disclosed and minimized. (+, weak)

Guideline 4:

Surgeons who introduce a new device or procedure in their practice should have completed relevant surgical training, possess operating privileges in the affected organ system, and be able to address anticipated complications. (++, strong)

Guideline 5:

For minor modifications of devices and procedures, surgeons should monitor their practice based on self-assessment. The more substantial the change in surgeons’ practice and the higher the risk to the patient, the more important it is that surgeons complete a relevant didactic course, and have their performance objectively assessed and their outcomes monitored by an external entity. (+, strong)

Guideline 6:

To protect their patients, surgeons should demonstrate the highest level of professionalism and exercise self-assessment and self-regulation when introducing new technology and techniquesin their practice. Besides the FDA, which regulates the production and sale of new devices, institutional credentialing and/or new technology committees and the IRB should monitor their introduction in clinical practice. The introduction of novel procedures should be overseen by the credentialing committee and/or the IRB, while the role of specialty societies and new technology committees needs further assessment. (+, strong)

Guideline 7:

The effectiveness compared to alternatives, the cost, patient outcomes, and the safety profile of new technology and techniques should always be assessed prior to and after their introduction. Other parameters such as existing and required resources; benefits to patients, surgeons, and hospitals; existing or anticipated volume of use; barriers to adoption; and whether the anticipated benefits prove real after introduction should also be considered, especially for significant changes in devices and procedures, besides minor modifications. (+++, strong)

References

  1.    Club TSS (1991) A prospective analysis of 1518 laparoscopic cholecystectomies. N Engl J Med 324:1073-1078
  2.    ASGE/SAGES (2006) Working Group on Natural Orifice Translumenal Endoscopic Surgery White Paper October 2005. Gastrointest Endosc 63:199-203
  3.    Rattner DW, Hawes R, Schwaitzberg S, Kochman M, Swanstrom L (2011) The Second SAGES/ASGE White Paper on natural orifice transluminal endoscopic surgery: 5 years of progress. Surg Endosc 25:2441-2448
  4.    Surgeons ACo (1994) Statements on Emerging Surgical Technologies and the Evaluation of Credentials. Bull Am Coll Surg 79:40-41
  5.    Zorn KC, Gautam G, Shalhav AL, Clayman RV, Ahlering TE, Albala DM, Lee DI, Sundaram CP, Matin SF, Castle EP, Winfield HN, Gettman MT, Lee BR, Thomas R, Patel VR, Leveillee RJ, Wong C, Badlani GH, Rha KH, Eggener SE, Wiklund P, Mottrie A, Atug F, Kural AR, Joseph JV (2009) Training, credentialing, proctoring and medicolegal risks of robotic urological surgery: recommendations of the society of urologic robotic surgeons. J Urol 182:1126-1132
  6.    Surgery ASfMaB (2009) Position Statement on emerging endosurgical interventions for treatment of obesity. Surg Obes Relat Dis 5:297-298
  7.    Biffl WL, Spain DA, Reitsma AM, Minter RM, Upperman J, Wilson M, Adams R, Goldman EB, Angelos P, Krummel T, Greenfield LJ (2008) Responsible development and application of surgical innovations: a position statement of the Society of University Surgeons. J Am Coll Surg 206:1204-1209
  8.    Perrier ND, Randolph GW, Inabnet WB, Marple BF, VanHeerden J, Kuppersmith RB (2010) Robotic thyroidectomy: a framework for new technology assessment and safe implementation. Thyroid 20:1327-1332
  9.    Society AU (2012) Guidelines for providing privileges and credentials to physicians for transvaginal placement of surgical mesh for pelvic organ prolapse. Female Pelvic Med Reconstr Surg 18:194-197
  10. RACS. General Guidelines for Assessing, Approving & Introducing New Surgical Procedures into a Hospital or Health Service. ASERNIP-S; 2007. Available at: http://www.surgeons.org/media/297973/REA_ASE_3103_P_General_Guidelines_for_Assessing_Approving_Introducing_New_Surgical_Procedures_into_a_Hospital_or_Health_Service.pdf. Accessed 1 January 2014.
  11. Stefanidis D, Montero P, Urbach D, Qureshi A, Petersen R, Bachman S, Madan A, Perry K, Pryonr A (2014) SAGES Research Agenda in Gastroinstestinal and Endoscopic Surgery: Updated Results of a Delphi Study. Surg Endosc [Epub ahead of print]
  12. SAGES. Framework for Post-Residency Surgical Education & Training. Los Angeles: Society of American Gastrointestinal and Endoscopic Surgeons; 2010 updated 2010. Available at: https://www.sages.org/publications/guidelines/framework-for-post-residency-surgical-education-training/. Accessed 1 January 2014.
  13. SAGES. Guidelines for Institutions Granting Privileges Utilizing Laparoscopic and/or Thoracoscopic Techniques. Los Angeles: Society of Gastrointestinal and Endoscopic Surgeons; 2010. Available at: https://www.sages.org/publications/guidelines/guidelines-for-institutions-granting-privileges-utilizing-laparoscopic-andor-thoracoscopic-techniques/. Accessed 1 January 2014.
  14. Riskin DJ, Longaker MT, Gertner M, Krummel TM (2006) Innovation in surgery: a historical perspective. Ann Surg 244:686-693
  15. Peltola M, Malmivaara A, Paavola M (2012) Introducing a knee endoprosthesis model increases risk of early revision surgery. Clin Orthop Relat Res 470:1711-1717
  16. Sachdeva AK, Russell TR (2007) Safe introduction of new procedures and emerging technologies in surgery: education, credentialing, and privileging. Surg Clin North Am 87:853-866, vi-vii
  17. Wulsin J, L, Dougherty A. A Briefing on Health Technology Assessment. Sacramento: California State Library; 2008. Available at: http://library.ca.gov/crb/08/08-019.pdf. Accessed 1 January 2014.
  18. SAGES. The Definitions Document: A Reference for Use of SAGES Guidelines. Los Angeles: Society of American Gastroinstestinal Endoscopic Surgeons; 2009. Available at: https://www.sages.org/publications/guidelines/definitions-document-reference-for-use-of-sages-guidelines/. Accessed 1 January 2014.
  19. McGregor M, Brophy JM (2005) End-user involvement in health technology assessment (HTA) development: a way to increase impact. Int J Technol Assess Health Care 21:263-267
  20. Spigelman AD (2006) Governance and innovation: experience with a policy on the introduction of new interventional procedures. ANZ J Surg 76:9-13
  21. SAGES. TAVAC Safety and Effectiveness Analysis: LINX® Reflux Management System. Los Angeles: Society of Gastrointestinal and Endoscopic Surgeons; 2013. Available at: https://www.sages.org/publications/guidelines/tavac-safety-and-effectiveness-analysis-linx-reflux-management-system/. Accessed 1 January 2014.
  22. Poulin P, Austen L, Scott CM, Waddell CD, Dixon E, Poulin M, Lafreniere R (2013) Multi-criteria development and incorporation into decision tools for health technology adoption. J Health Organ Manag 27:246-265
  23. McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J, Aronson JK, Barkun JS, Blazeby JM, Boutron IC, Campbell WB, Clavien PA, Cook JA, Ergina PL, Feldman LS, Flum DR, Maddern GJ, Nicholl J, Reeves BC, Seiler CM, Strasberg SM, Meakins JL, Ashby D, Black N, Bunker J, Burton M, Campbell M, Chalkidou K, Chalmers I, de Leval M, Deeks J, Ergina PL, Grant A, Gray M, Greenhalgh R, Jenicek M, Kehoe S, Lilford R, Littlejohns P, Loke Y, Madhock R, McPherson K, Meakins J, Rothwell P, Summerskill B, Taggart D, Tekkis P, Thompson M, Treasure T, Trohler U, Vandenbroucke J (2009) No surgical innovation without evaluation: the IDEAL recommendations. Lancet 374:1105-1112
  24. The IDEAL Collaboration.
  25. See WA, Cooper CS, Fisher RJ (1993) Predictors of laparoscopic complications after formal training in laparoscopic surgery. JAMA 270:2689-2692
  26. Lyman S, Sedrakyan A, Do H, Razzano R, Mushlin AI (2011) Infrequent physician use of implantable cardioverter-defibrillators risks patient safety. Heart 97:1655-1660
  27. Renaud M, Reibel N, Zarnegar R, Germain A, Quilliot D, Ayav A, Bresler L, Brunaud L (2013) Multifactorial analysis of the learning curve for totally robotic Roux-en-Y gastric bypass for morbid obesity. Obes Surg 23:1753-1760
  28. Holzhey DM, Seeburger J, Misfeld M, Borger MA, Mohr FW (2013) Learning minimally invasive mitral valve surgery: a cumulative sum sequential probability analysis of 3895 operations from a single high-volume center. Circulation 128:483-491
  29. Miskovic D, Ni M, Wyles SM, Tekkis P, Hanna GB (2012) Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum 55:1300-1310
  30. Pucher PH, Sodergren MH, Singh P, Darzi A, Parakseva P (2013) Have we learned from lessons of the past? A systematic review of training for single incision laparoscopic surgery. Surg Endosc 27:1478-1484
  31. Sachdeva AK (2005) Acquiring skills in new procedures and technology: the challenge and the opportunity. Arch Surg 140:387-389
  32. AdvaMed. Code of Ethics on Interactions with Health Care Professionals. Washington, DC: Advanced Medical Technology Association; 2009. Available at: http://advamed.org/res.download/112. Accessed 1 January 2014.
  33. Chang DC, Easterlin MC, Montesa C, Kaushal K, Wilson SE (2012) Adoption of endovascular repair of abdominal aortic aneurysm in California: lessons for future dissemination of surgical technology. Ann Vasc Surg 26:468-475
  34. Sidhu RS, Vikis E, Cheifetz R, Phang T (2006) Self-assessment during a 2-day laparoscopic colectomy course: can surgeons judge how well they are learning new skills? Am J Surg 191:677-681
  35. Jelovsek JE, Walters MD, Korn A, Klingele C, Zite N, Ridgeway B, Barber MD (2010) Establishing cutoff scores on assessments of surgical skills to determine surgical competence. Am J Obstet Gynecol 203:81.e81-86
  36. Sedlack RE (2011) Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 74:355-366.e351-352
  37. Grigg OA, Farewell VT, Spiegelhalter DJ (2003) Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Stat Methods Med Res 12:147-170
  38. Chang WR, McLean IP (2006) CUSUM: a tool for early feedback about performance? BMC Med Res Methodol 6:8
  39. Greenberg D, Peterburg Y, Vekstein D, Pliskin JS (2005) Decisions to adopt new technologies at the hospital level: insights from Israeli medical centers. Int J Technol Assess Health Care 21:219-227
  40. Committee on Technological Innovation in Medicine IoM. Adopting New Medical Technology. Annetine C. Gelijns HVD, editor. Washington, DC: National Academy Press; 1994.
  41. Grunwald T, Krummel T, Sherman R (2004) Advanced technologies in plastic surgery: how new innovations can improve our training and practice. Plast Reconstr Surg 114:1556-1567
  42. Patel A, Patel M, Lytle N, Toro JP, Medbery RL, Bluestein S, Perez SD, Sweeney JF, Davis SS, Lin E (2013) Can we become better robot surgeons through simulator practice? Surg Endosc [Epub ahead of print]
  43. Stefanidis D (2010) Optimal acquisition and assessment of proficiency on simulators in surgery. Surg Clin North Am 90:475-489
  44. Ellison EC, Carey LC (2008) Lessons learned from the evolution of the laparoscopic revolution. Surg Clin North Am 88:927-941
  45. Bergqvist D (2009) Introduction of new technology: the surgical point of view. Scand J Surg 98:3-7
  46. Zietman A, Ibbott G (2012) A clinical approach to technology assessment: how do we and how should we choose the right treatment? Semin Radiat Oncol 22:11-17

This document was prepared and revised by the SAGES Guidelines Committee

This document was reviewed and approved by the Board of Governors of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in Aug 2014.

SAGES publication #NEWTECH01

For more information please contact:

SOCIETY OF AMERICAN GASTROINTESTINAL ENDOSCOPIC SURGEONS (SAGES)
11300 West Olympic Blvd., Suite 600
Los Angeles, CA 90064
Tel:
(310) 437-0544
Fax:
(310) 437-0585
Email:
publications@sages.org

Guidelines for clinical practice are intended to indicate preferable approaches to medical problems as established by experts in the field. These recommendations will be based on existing data or a consensus of expert opinion when little or no data are available. Guidelines are applicable to all physicians who address the clinical problem(s) without regard to specialty training or interests, and are intended to indicate the preferable, but not necessarily the only acceptable approaches due to the complexity of the healthcare environment. Guidelines are intended to be flexible. Given the wide range of specifics in any health care problem, the surgeon must always choose the course best suited to the individual patient and the variables in existence at the moment of decision.

Guidelines are developed under the auspices of the Society of American Gastrointestinal and Endoscopic Surgeons and its various committees, and approved by the Board of Governors. Each clinical practice guideline has been systematically researched, reviewed and revised by the guidelines committee, and reviewed by an appropriate multidisciplinary team. The recommendations are therefore considered valid at the time of its production based on the data available. Each guideline is scheduled for periodic review to allow incorporation of pertinent new developments in medical research knowledge, and practice.