Effect of Self-Training Using Virtual Reality Head-Mounted Display Simulator on the Acquisition of Holmium Laser Enucleation of the Prostate Surgical Skills

Article information

Int Neurourol J. 2024;28(2):138-146

Publication date (electronic) : 2024 June 30

doi : https://doi.org/10.5213/inj.2448042.021

Taesoo Jang ¹^,^*

, Hyoun-Joong Kong ²^,^*

, Changhoon Baek ²

, Junki Kim ²

, Min Soo Choo ³

, Seung-June Oh^,⁴

¹Department of Biomedical Engineering, College of Medicine, Chungnam National University, Daejeon, Korea

²Department of Transdisciplinary Medicine, Seoul National University Hospital, Seoul, Korea

³Department of Urology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, Korea

⁴Department of Urology, Seoul National University Hospital, Seoul, Korea

Corresponding author: Seung-June Oh Department of Urology, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea Email: sjo@snu.ac.kr

*Taesoo Jang and Hyoun-Joong Kong contributed equally to this study as co-first authors.

Received 2024 January 8; Accepted 2024 March 20.

Abstract

Purpose

We aimed to evaluate the effect of self-training using a virtual reality head-mounted display simulator on the acquisition of surgical skills for holmium laser enucleation surgery.

Methods

Thirteen medical students without surgical skills for holmium laser enucleation of the prostate were trained using multimedia to learn the technique via simulator manipulation. Thereafter, participants performed the technique on a virtual benign prostatic hyperplasia model A (test A). After a 1-week wash-out period, they underwent self-training using a simulator and performed the technique on model B (test B). Subsequently, participants were asked to respond to Training Satisfaction Questions. Video footage of hand movements and endoscope view were recorded during tests A and B for later review by 2 expert surgeons. A 20-step Assessment Checklist, 6-domain Global Rating Scale, and a Pass Rating were used to compare performance on tests A and B.

Results

Thirteen participants completed both tests A and B. The 20-step Assessment Checklist and 6-domain Global Rating Scale evaluation results showed significantly improved scores in test B than in test A (P<0.05). No evaluator rated participants as passed after test A, but 11 participants (84.6%) passed after test B. Ten participants (76.9%) indicated that the simulator was helpful in acquiring surgical skills for holmium laser enucleation of the prostate.

Conclusions

The virtual reality head-mounted display holmium laser enucleation of the prostate simulator was effective for surgical skill training. This simulator may help to shorten the learning curve of this technique in real clinical practice in the future.

Keywords: Prostatic hyperplasia; Computer simulation; Education; Prostate; Transurethral resection of prostate; Virtual reality

INTRODUCTION

Benign prostatic hyperplasia (BPH) is one of the most common urological conditions in men aged >50 years [1], with incidence increasing as the patients age [2]. When medical treatment for BPH is ineffective, surgical treatment is commonly applied. The standard treatment for endoscopic surgery is transurethral resection of the prostate (TURP). However, holmium laser enucleation of the prostate (HoLEP) has gained significant popularity [3], which is a fundamentally different procedure from laser vaporization or TURP. Theoretically, HoLEP can completely remove the prostatic adenoma by enucleating the prostatic adenoma from the prostatic capsule and has the advantage of less bleeding and shorter hospital stays [4]. HoLEP enables surgeons to enucleate adenoma by applying important modes of operation, including incision, hemostasis, and vibration, depending on the distance between the tip of the holmium laser fiber and the target tissue of the prostate [5-7].

Despite these advantages, HoLEP is not widely used due to its steep learning curve [3], owing to not only the surgical method differing from conventional TURP or vaporization but also the steep learning curve of surgical techniques for beginners [3, 8]. The need for a systematic learning program to solve these shortcomings has been raised, and surgical training simulators have been developed [9]. These simulators include a synthetic bench model [9, 10], an animal and human cadaver model [11], as well as a computer-assisted virtual reality (VR) simulator [12]. Existing biological tissue-based simulators have limitations in practical application due to cost issues and the limited number of uses [13]. Meanwhile, the VR head-mounted display (HMD) simulator has the great advantage of being reusable [14], cost-effective and usable at any time [15]. Therefore, in this study, we aimed to determine whether the surgical training method using a VR HMD simulator is effective in acquiring HoLEP skills.

MATERIALS AND METHODS

Study Participants

Overall, 13 second to fourth-year medical students (average age, 22.0±1.0 years) were recruited during the university vacation period in January 2022 in this study. Two researchers (TJ and JK) arranged each participant’s schedule to ensure that the study proceeded sequentially.

Each participant was individually brought to the Medical XR Studio at Seoul National University Hospital (SNUH) Innovative Medical Technology Research Institute. All participants received preliminary introductory training in the same manner (Fig. 1). The educational material included a 72-minute video, categorized into 4 modules. The first module contained general knowledge about BPH, various surgical methods, the step-by-step surgical procedure of HoLEP [8, 16, 17], and basic manipulation of the endoscope during surgery. An explanation of the outline of this study, evaluation method, and manipulation of the HMD simulator was also included (40 minutes). The second module illustrated core knowledge of the surgical anatomy for HoLEP, a step-by-step introduction to HoLEP using the 3-dimensional (3D) conceptual BPH model based on anatomical viewpoints. Therefore, participants were expected to understand how adenoma is actually enucleated endoscopically in 20 consecutive surgical steps based on the classic three-lobe technique in medium-sized BPH [8, 16, 17], as shown in Table 1 (20 minutes). The third training module was a recorded video of an actual surgery that simultaneously showed the surgical endoscope screen and the surgeon’s hand movements (10 minutes). The fourth module was an explanation of the basic manipulation of the simulator at the Medical XR Studio site (2 minutes).

Fig. 1.

Flow diagram. 20-SC, 20-step Assessment Checklist; 6-GRS, 6-domain Global Rating Scale; TSQ, Training Satisfaction Questions; VR HMD, virtual reality head-mounted display.

Table 1.

Evaluation results of the 20-step Assessment Checklist for the 13 participants

After completing the above audio-visual education, participants performed the technique on a virtual BPH model A (test A). Participants wore the VR HMD simulator in a sitting position and performed bench surgery on BPH model A with a total prostate volume (TPV) of 90 mL using the simulator handle synchronized with it. After a 1-week wash-out period, the participants were asked to visit the Medical XR Studio again at a convenient time for self-training within the next 1 week. In the same environment as test A, the simulator equipped with BPH model A was used to enable participants to self-train. The basic training time comprised 2 sessions, each lasting 60 minutes, with a break in the middle (total training time, 2 hours). However, participants were allowed to train less or more if they desired. Training included detailed information on surgical procedures and appropriate hand movements at each stage of surgery. After self-training, the participants visited the studio again the following week. They performed the technique on BPH model B (TPV, 56 mL) (test B) in the same environment as test A.

Software Part of the SNUH VR HMD HoLEP Simulator

The software part of the SNUH VR HMD HoLEP simulator comprises the virtual operating room, virtual organ, and surgical training interaction parts. The virtual operational room was implemented based on the environment of the SNUH HoLEP operating room to maximize participants’ sense of immersion during training and tests (Fig. 2D). The virtual organ was implemented through the following process. Using a 3D segmentation software (OSIRIX MD 12.0.2, Pixmeo, Bernex, Switzerland), 3D models including the urinary sphincter, prostate, and bladder were reconstructed from magnetic resonance images of 22 patients with BPH with TPVs of 56 and 90 mL, respectively. Subsequently, postprocessing of the reconstruction model was optimized using a 3D model editing software (Maya 2022, Autodesk Inc., San Rafael, CA, USA) as well as a 3D digital painting software (Mudbox 2022, Autodesk Inc., San Rafael, CA, USA) and applied to the 3D models (Fig. 2C).

Fig. 2.

Operating handle and simulator software interface: (A) Testing and recording the test with simulator. (B) Head-mounted display (HMD) controller that acts as a handle for the endoscope; an aluminum pipe that acts as a unipolar resectoscope; 2 adapters to connect the pipe and the HMD controller; and a trocar porthole panel. The pipe and adapter were constructed using a 3-dimensional (3D) printer. (C) Benign prostatic hyperplasia (BPH) model A (total prostate volume, 90 mL) and BPH model B (total prostate volume, 56 mL) produced based on an actual imaging study. (D) Screen capture of Seoul National University Hospital virtual reality HMD holmium laser enucleation of the prostate simulator.

To develop a method for the operator to manipulate virtual organs in the virtual operational room, a 3D Game Engine (Unity 2020.3.26, Unity Technologies, San Francisco, CA, USA) and a soft-body physics simulation plug-in (Obi Softbody 6.1, Virtual Method Studio, Madrid, Spain) were used. To complete the interaction part of the surgical training, the characteristic profile of the holmium laser during laser application to the prostatic tissue, including incision, hemostasis, and vibration, was determined according to the distance between the laser fiber tip and the prostate tissue [8]. Realistic soft-body physics was implemented so that the degree of direct contact of the tip of the endoscope to the prostatic tissue can deform the prostate tissue to a varying degree. The sound generated during lasing was reflected as VR sound output. The shaking of the endoscope handle generated by shock waves during lasing was also reflected as a vibration of the VR controller. White noise in the operating room was recorded and reflected in the VR audio. To develop VR HMD simulator software, one 3D graphics programmer (JK) and one biomedical engineer (TJ) majoring in VR initiated the development. They observed actual HoLEP procedures more than 20 times and had detailed discussions with a clinician (SJO).

Hardware Part of the SNUH VR HMD HoLEP Simulator

The hardware part of the SNUH VR HMD simulator was the endoscope-manipulating system (EMS) designed to simulate the Storz endoscope set [8] used for HoLEP. Oculus Rift S (Meta platform, Inc., Menlo Park, CA, USA) was used as a VR HMD. The hardware consists of the following 4 parts: an HMD controller that acts as a handle for the endoscope; an aluminum pipe that acts as a unipolar resectoscope; 2 adapters to connect the pipe and the HMD controller; and a trocar porthole panel (ArtiSential Training Kit, Livsmed, Seongnam, Korea) that can hold the endoscope (Fig. 2B). The pipe and adapter were constructed using a 3D printer (Cubicon Style NEO-A31C, Cubicon, Seongnam, Korea). The EMS was designed to ensure that participants performed any movement with the resectoscope, including forward-backward, up-down, left-right, linear, curved, and rotational movements.

Assessments

The entire surgery performed by each participant was videotaped (LG smartphone V10, LG Electronics, Seoul, Korea). The surgical endoscope screen and the participant’s upper body were included together so that the evaluator could later review the surgical endoscope screen and participants’ hand movements during surgery simultaneously. The left-back side of the participant’s body was recorded to avoid direct identification of their face (Fig. 2A). The recorded video files were anonymized and separately delivered to 2 experienced HoLEP surgeons (SJO and MSC) for evaluation.

Two evaluators determined the effectiveness of self-learning using the 20-step Assessment Checklist (20-AC) and 6-domain Global Rating Scale (6-GRS). The 20-AC was designed to evaluate the test surgery by classifying HoLEP procedures into 20 steps [16]. For each step, ‘‘not performed” or ‘‘incorrectly performed” was rated as 0 points, and ‘‘performed correctly’’ was rated as 1 point. The 6-GRS was used as a tool to evaluate key elements of surgical proficiency by setting scoring standards for each domain for essential skills and summing the scores. After evaluating 20-AC and 6-GRS of each participant, each evaluator was required to assign a Pass Rating to each participant. The question was: ‘‘Are you confident that you would allow this participant to perform this surgery in a real operating room?’’ Each evaluator was asked to select either ‘‘yes’’ or ‘‘no.’’ The 2 evaluators were only involved in obtaining consent forms but not in the process of patient recruitment, self-training, or performing the test surgeries.

Immediately after completing all test schedules, each participant was asked to fill out the Training Satisfaction Question (TSQ) questionnaire. The TSQ content was designed to self-evaluate the degree of dedication to self-training, understanding of HoLEP, and satisfaction with training based on a 5-point Likert scale. The questionnaires were validated through a pilot study. After all tests were completed, 2 researchers in charge (TJ and JK) listened to the participants’ feedback about the simulator in an open-ended manner.

Statistical Analysis

The 20-AC and 6-GRS results were analyzed by summing the results for each evaluator. The TSQ administered after test B was also analyzed. All data analyses were performed using the SAS 9.4 (SAS Institute, Cary, NC, USA). Statistical significance was set at P<0.05.

RESULTS

20-Step Assessment Checklist

At least 6 participants (46.2%) demonstrated proficiency in 3 specific steps (9, 13, and 20) of test A, while they did not perform the remaining 17 steps well. Of the 20 steps, the 2 evaluators rated the participants’ skills in 9 steps (2, 3, 4, 5, 6, 7, 14, 17, and 18) as significantly improved during test B compared to that during test A (P<0.05). After self-training, for 3 steps (12, 15, and 19), the participants’ skills were assessed as significantly improved by 1 evaluator. For 7 steps (1, 8, 9, 10, 13, 16, and 20), both evaluators rated the participants’ skills as not significantly improved after self-training (Table 1).

6-Domain Global Rating Scale

Both evaluators rated that the scores of all 6 domains of 6-GRS as improved after self-training. Test A scores ranged from 1.23 to 1.85 in all domains. After self-training, the scores ranged from 3.38 to 4.15, indicating a significant improvement in the participants’ skills (P <0.05). Among these, GRS 6 (bleeding control) reached the highest level of skill after self-training. The total score significantly increased after self-training (Table 2).

Table 2.

Test scores of the 6-domain Global Rating Scale for the 13 participants

Pass Rating Question

Immediately after evaluating test A, neither of the evaluators were not confident that they would allow even 1 participant to perform HoLEP in an actual operating room. However, after test B, evaluator 1 responded that he would allow 11 participants (84.6%) to perform surgery.

Training Satisfaction Question

Overall, 69.3% of the participants reported having trained themselves for more than 2–3 hours using the simulator. During the self-training period, 76.9% of participants responded that they devoted sufficient time to self-training. Overall, 12 individuals (92.3%) reported an improvement in their understanding of HoLEP. Additionally, 76.9% of participants responded that this surgical technique training using VR was helpful in acquiring surgical skills. Furthermore, 69.3% of participants responded that their surgical skill training module using VR was useful (Table 3).

Table 3.

Responses to the Training Satisfaction Questions provided by the 13 participants

When all procedures were completed, the participants’ opinions on this study were given in the form of open-ended feedback. Most of the participants stated that it was their first time using a VR HMD. They reported experiencing dizziness at first when training or testing using the HMD. However, over time, the dizziness disappeared as they became accustomed to the HMD.

DISCUSSION

To the best of our knowledge, this is the first study to investigate the usefulness of a HoLEP skill training module using a VR HMD HoLEP simulator. The steep learning curve of the surgery is the primary reason for the delay in the widespread adoption of HoLEP [18]. It is known that a surgeon becomes familiar with the technique after an initial 30 [19] to 50 cases of surgery [20]. Close mentorship is emphasized to overcome the learning curve [21].

Kansai Medical University developed a Bench Model HoLEP simulator as an educational simulator to improve HoLEP skills [22], which was physically composed of 2 layers of adenoma rather than implemented on a computer. The UroSim HoLEP simulator (VirtaMED, Zurich, Switzerland) uses a computer system and VR [9]. However, this simulator did not reflect the physical characteristics of the laser profile on prostate tissue. The simulator in our study implemented the characteristics of the laser’s physical action, including incision, coagulation, and vibration, among others, on prostatic soft tissue.

Generally, system configuration that reflects the surgeon’s gesture significantly impacts performance. The wavelength of the holmium laser enables incision or enucleation owing to its physical properties, specifically its high absorption by water [23]. Among the range of distances between the laser tip and biological tissue, the distance at which the incision is prominent is known to be 0.5–1.0 mm [24]. Additionally, the holmium laser can transmit heat over a wide range owing to its ability to be easily absorbed by water. It is known to produce tissue heating or hemostatic effects at a distance of 4–6 mm.

After all tests were completed, opinions about the SNUH VR HMD HoLEP simulator were freely discussed with the developers and the participants. Most of the participants stated that it was their first time using a VR HMD. Most participants reported that they experienced dizziness at the beginning of wearing the HMD. However, as they gradually became used to it, the dizziness disappeared. There was also an opinion that no one had used VR HMD devices for a long time. Participants who had previously experienced First-Person Shooter games on a computer appeared to have less dizziness and no difficulty in adapting. However, as the experience time increased through self-learning, everyone stated that they had no problem adapting to the HMD device.

There are a few limitations of this study. First, participants included students who had never been exposed to actual HoLEP before. Generally, medical students are the target audience who need to be educated from very basic anatomy to surgical techniques. However, generally, HoLEP is performed by urology specialists who have a considerable amount of knowledge in these areas and have undergone a certain period of training. Therefore, it may be different for medical students to learn basic knowledge or surgical techniques of HoLEP tailored to them and for specialists to acquire the learning curve of techniques needed at their level. Second, using these simulators can improve simulation skills with time and repetition. However, whether this actually enhances surgical skills and efficiently trains individuals is a separate issue. In this respect, additional research regarding whether training effect using a VR simulator actually leads to a shortening of the learning curve of HoLEP for urology residents is needed in the future. Since the training effect using a VR HMD simulator for medical students was clearly demonstrated in this study, the effect of self-training using a VR simulator for participants majoring in actual urology is expected to be considerably more positive than the results of this study.

In conclusion, the SNUH VR HMD HoLEP simulator was effective for HoLEP surgical skill training. We expect that this simulator will help shorten the learning curve associated with HoLEP in real clinical practice in the future.

Notes

Grant/Fund Support

This study was supported by grant No. 03-2020-2120 from the SNUH Research Fund, as well as the MSIT (Ministry of Science and ICT), South Korea, under the ITRC (Information Technology Research Center) support program (IITP 2023-2018-0-01833) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).

Research Ethics

This study was conducted in accordance with the declaration of Helsinki and received ethical approval from the Seoul National University Hospital (IRB No. 2008-075-1147 and 2008-079-1148). All participants provided written informed consent.

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTION STATEMENT

· Conceptualization: SJO, HJK

· Data curation: SJO, TJ

· Formal analysis: TJ, SJO, MSC, CB

· Funding acquisition: SJO, HJK

· Methodology: SJO, HJK

· Project administration: SJO, TJ, JK

· Visualization: TJ, JK, SJO

· Writing - original draft: SJO, TJ

· Writing - review & editing: SJO, TJ, HJK, CB, MSC

References

1. Fourcade RO, Lacoin F, Rouprêt M, Slama A, Le Fur C, Michel E, et al. Outcomes and general health-related quality of life among patients medically treated in general daily practice for lower urinary tract symptoms due to benign prostatic hyperplasia. World J Urol 2012;30:419–26.

2. Lim KB. Epidemiology of clinical benign prostatic hyperplasia. Asian J Urol 2017;4:148–51.

3. Jeon BJ, Chung H, Bae JH, Jung H, Lee JG, Choi H. Analysis of present status for surgery of benign prostatic hyperplasia in Korea using nationwide healthcare system data. Int Neurourol J 2019;23:22–9.

4. Zhong J, Feng Z, Peng Y, Liang H. A systematic review and metaanalysis of efficacy and safety following holmium laser enucleation of prostate and transurethral resection of prostate for benign prostatic hyperplasia. Urology 2019;131:14–20.

5. Vassar GJ, Chan KF, Teichman JM, Glickman RD, Weintraub ST, Pfefer TJ, et al. Holmium: YAG lithotripsy: photothermal mechanism. J Endourol 1999;13:181–90.

6. Yoon HS, Kim MH, Park JS, Choo MS, Jeong SJ, Oh SJ. Risk factors for transurethral coagulation for hemostasis during holmium laser enucleation of the prostate. Int Neurourol J 2022;26:153–60.

7. Lü T, Xiao Q, Xia D, Ruan K, Li Z. Cavitation effect of holmium laser pulse applied to ablation of hard tissue underwater. J Biomed Opt 2010;15:048002.

8. Kim M, Lee HE, Oh SJ. Technical aspects of holmium laser enucleation of the prostate for benign prostatic hyperplasia. Korean J Urol 2013;54:570–9.

9. Kuronen-Stewart C, Ahmed K, Aydin A, Cynk M, Miller P, Challacombe B, et al. Holmium laser enucleation of the prostate: simulation-based training curriculum and validation. Urology 2015;86:639–46.

10. Aydin A, Ahmed K, Brewin J, Khan MS, Dasgupta P, Aho T. Face and content validation of the prostatic hyperplasia model and holmium laser surgery simulator. J Surg Educ 2014;71:339–44.

11. Tunc L, Bozzini G, Scoffone CM, Guven S, Hermann T, Porreca A, et al. Determination of face and content validity of cadaveric model for holmium anatomic endoscopic enucleation of the prostate training: an ESUT AEEP Group Study. Eur Urol Open Sci 2021;32:28–34.

12. Khan R, Aydin A, Khan MS, Dasgupta P, Ahmed K. Simulationbased training for prostate surgery. BJU Int 2015;116:665–74.

13. Wake N, Nussbaum JE, Elias MI, Nikas CV, Bjurlin MA. 3D printing, augmented reality, and virtual reality for the assessment and management of kidney and prostate cancer: a systematic review. Urology 2020;143:20–32.

14. Childs BS, Manganiello MD, Korets R. Novel education and simulation tools in urologic training. Curr Urol Rep 2019;20:81.

15. Sweet RM. Review of trainers for transurethral resection of the prostate skills. J Endourol 2007;21:280–4.

16. Oh SJ. Current surgical techniques of enucleation in holmium laser enucleation of the prostate. Investig Clin Urol 2019;60:333–42.

17. Oh SJ, Shitara T. Enucleation of the prostate: an anatomical perspective. Andrologia 2020;52:e13744.

18. Gravas S, Bachmann A, Reich O, Roehrborn CG, Gilling PJ, De La Rosette J. Critical review of lasers in benign prostatic hyperplasia (BPH). BJU Int 2011;107:1030–43.

19. Bae J, Oh SJ, Paick JS. The learning curve for holmium laser enucleation of the prostate: a single-center experience. Korean J Urol 2010;51:688–93.

20. Shah HN, Mahajan AP, Sodha HS, Hegde S, Mohile PD, Bansal MB. Prospective evaluation of the learning curve for holmium laser enucleation of the prostate. J Urol 2007;177:1468–74.

21. Wright HC, Fedrigon D, De S. Learning from those who learned: a survey of fellowship trained HoLEP surgeons and their current practice patterns. Urology 2021;149:193–8.

22. Knoll T, Omar MI, Maclennan S, Hernández V, Canfield S, Yuan Y, et al. Key steps in conducting systematic reviews for underpinning clinical practice guidelines: methodology of the european association of urology. Eur Urol 2018;73:290–300.

23. Wollin TA, Denstedt JD. The holmium laser in urology. J Clin Laser Med Surg 1998;16:13–20.

24. Johnson DE, Cromeens DM, Price RE. Use of the holmium:YAG laser in urology. Lasers Surg Med 1992;12:353–63.

Article information Continued

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Table 1.

Evaluation results of the 20-step Assessment Checklist for the 13 participants

Step	Evaluator 1			Evaluator 2
Step	Test A (No. of participants who performed the step correctly)	Test B (No. of participants who performed the step correctly)	P-value	Test A (No. of participants who performed the step correctly)	Test B (No. of participants who performed the step correctly)	P-value
Step 1. Observation of overall anatomy before the start of surgery	0 (0)	1 (7.7)	> 0.99	0 (0)	2 (15.4)	0.5
Step 2. The initial longitudinal incision along each lateral side of the verumontanum to determine the surgical capsule	4 (30.8)	12 (92.3)	0.007	5 (38.5)	13 (100.0)	0.008
Step 3. Longitudinal incisions from the 5, 7 o’clock direction of BN to the previous incisions	3 (23.1)	12 (92.3)	0.003	2 (15.4)	12 (92.3)	0.002
Step 4. A transverse incision immediately above the verumontanum for median lobe enucleation	3 (23.1)	12 (92.3)	0.003	5 (38.5)	13 (100.0)	0.008
Step 5. Upward enucleation of both lateral lobes at the prostatic apex to form bilateral mucosal wing	4 (30.8)	12 (92.3)	0.007	6 (46.2)	13 (100.0)	0.016
Step 6. Longitudinal incision of the anterior lobe at 12 o’clock position and bilateral downward enucleation	3 (23.1)	10 (76.9)	0.03	2 (15.4)	12 (92.3)	0.002
Step 7. Inverted V-shape mucosal incision at the prostatic apex	4 (30.8)	11 (84.6)	0.01	3 (23.1)	12 (92.3)	0.004
Step 8. Conjoining incision along the 12 o’clock direction between the two points at the BN and the prostatic apex	1 (7.7)	4 (30.8)	0.25	2 (15.4)	5 (38.5)	0.45
Step 9. Upward enucleation of the left lobe at the prostatic apex	6 (46.2)	11 (84.6)	0.12	8 (61.5)	12 (92.3)	0.21
Step 10. Downward enucleation of the left lobe from the 12 o’clock position	0 (0)	0 (0)	> 0.99	2 (15.4)	2 (15.4)	> 0.99
Step 11. Conjoining incision at the left anterior apex	0 (0)	0 (0)	> 0.99	4 (30.8)	12 (92.3)	0.008
Step 12. Enucleating left lobe adenoma from the apex to toward the BN	6 (46.2)	12 (92.3)	0.03	7 (53.6)	12 (92.3)	0.06
Step 13. Pushing the enucleated left lobe into the bladder after complete separation of the adenoma from the capsule	10 (76.9)	11 (84.6)	> 0.99	9 (69.2)	12 (92.3)	0.37
Step 14. Upward enucleation of the right lobe	1 (7.7)	8 (61.5)	0.01	2 (15.4)	9 (69.2)	0.01
Step 15. Enucleation of the right lobe posterolaterally and anteriorly toward the BN	1 (7.7)	10 (76.9)	0.003	4 (30.8)	9 (69.2)	0.12
Step 16. Mucosal incision over the anterior apex of the right lobe	0 (0)	0 (0)	> 0.99	0 (0)	4 (30.8)	0.12
Step 17. Enucleation of right lobe proximally (cross-over technique if necessary)	3 (23.1)	11 (84.6)	0.007	4 (30.8)	12 (92.3)	0.008
Step 18. Circumferential incision at the BN to separate the adenoma from the bladder mucosa	4 (30.8)	11 (84.6)	0.03	5 (38.5)	12 (92.3)	0.03
Step 19. Pushing the enucleated left lobe into the bladder after complete separation of the adenoma from the capsule	6 (46.2)	12 (92.3)	0.03	8 (61.5)	13 (100.0)	0.06
Step 20. Meticulous hemostasis	8 (61.5)	12 (92.3)	0.21	10 (76.9)	12 (92.3)	0.62

Values are presented as number (%).

BN, bladder n.

Domain	Evaluator 1		P-value	Evaluator 2		P-value
Domain	Test A	Test B	P-value	Test A	Test B	P-value
GRS 1. Respect for the tissue	1.38 ± 0.51	3.38 ± 0.65	< 0.001	1.38 ± 0.51	3.46 ± 0.66	< 0.001
GRS 2. Time and motion	1.85 ± 1.14	3.77 ± 1.24	< 0.001	1.77 ± 1.09	3.62 ± 1.12	< 0.001
GRS 3. Handling of the endoscope	1.46 ± 0.66	3.92 ± 1.19	< 0.001	1.54 ± 0.78	3.92 ± 1.19	< 0.001
GRS 4. Flow of the procedure and forward planning	1.46 ± 0.66	3.92 ± 0.86	< 0.001	1.54 ± 0.66	4.00 ± 0.91	< 0.001
GRS 5. Knowledge of the procedure	1.31 ± 0.48	3.62 ± 0.77	< 0.001	1.23 ± 0.44	3.46 ± 0.97	< 0.001
GRS 6. Bleeding control	1.77 ± 0.93	4.15 ± 1.28	< 0.001	1.69 ± 0.95	4.15 ± 1.28	< 0.001
Grand Sum (GRS 1–6)	9.23 ± 3.24	22.77 ± 5.02	< 0.001	9.15 ± 3.31	22.62 ± 5.14	< 0.001

Questions	Response	No. (%)
Q1. Please select one option below that is closest to the amount of time you devoted to training during your self-study period.	0–1 hr	1 (7.7)
	1–2 hr	3 (23.1)
	2–3 hr	6 (46.2)
	3–4 hr	3 (23.1)
	Over 4 hr	0 (0)
Q2. Did you devote sufficient time to training during your self-training period?	Strongly disagree	0 (0)
	Disagree	1 (7.7)
	Neither agree nor disagree	2 (15.4)
	Agree	10 (76.9)
	Strongly agree	0 (0)
Q3. Has your understanding of HoLEP improved after experiencing this training program using VR?	Strongly disagree	0 (0)
	Disagree	0 (0)
	Neither agree nor disagree	1 (7.7)
	Agree	8 (61.5)
	Strongly agree	4 (30.8)
Q4. Do you think that your experience with this training program using VR was helpful for you in acquiring surgical skills for HoLEP?	Strongly disagree	0 (0)
	Disagree	2 (15.4)
	Neither agree nor disagree	1 (7.7)
	Agree	7 (53.8)
	Strongly agree	3 (23.1)
Q5. Do you think this surgical skills training module using VR is useful in acquiring surgical skills for HoLEP?	Strongly disagree	0 (0)
	Disagree	2 (15.4)
	Neither agree nor disagree	2 (15.4)
	Agree	6 (46.2)
	Strongly agree	3 (23.1)