I was a Technical Panel Speaker of RoPat20: IROS 2020 Workshop. IEEE IROS 2020 took place from Oct 25, 2020 to Jan 24, 2021, and was an On-Demand Conference.
In this first RoPat workshop titled “Robot-assisted Training for Primary Care: How can robots help train doctors in medical examinations?”, I briefly introduced my research on medical simulation and robot-assisted training for hand-eye coordination at the beginning of the panel discussion.
I began working on medical simulation in the 1990s through a research collaboration between a Singaporean publicly funded research institution and Johns Hopkins University in the US. Our aim was to develop a training simulator for interventional radiology, similar to a flight simulator for pilot training. Training a doctor to become a qualified interventional radiologist is a time-consuming process. Our focus was on providing realistic hand-eye coordination training for the trainees. We re-constructed the human vascular system from the Virtual Human project, and then modelled the interaction between the vessel wall, catheter and guidewire using finite element methods. Our simulator functions like a flight simulator or a computer game – it does not provide direct instruction.
About 10 years ago, together with my collaborators in NUH and A*Star research institutions in Singapore, we developed a robot trainer for laparoscopic surgery training. The robot would learn the motions from the master surgeon and guide the trainee to replicate them. The trainee could also freely perform the motions and compared them to the master’s. We tested the VR-based training system with medical students and have since transitioned to an AR sytsem.
The slide below shows the FLS peg transfer setup. Currently, we are focusing on robot motion learning using deep reinforcement learning. We also do scene segmentation and workflow recognition.
The IROS 2020 RoPat workshop was a success. The 2nd RoPat workshop (RoPat 21) on Robot-Assisted Systems for Medical Training was organized in IEEE ICRA 2021 conference, held from May 30 to June 5, 2021, in Xi’an, China. It was a hybrid event.
This time I participated in the workshop as an invited technical speaker. I delivered a presentation on physical tissue modelling and augmented reality training in medical training online. I apologize for not being able to answer questions immediately after my oral presentation. If you have any questions regarding my talk, please feel free to email me.
Following is the title and abstract of my talk in IEEE ICRA 2021 RoPat.
Title: Medical Simulation and Deep Reinforcement Learning
Abstract: Medical simulation provides clinicians with real-time interactive simulations of surgical procedures to enhance training, pre-treatment planning, and the design and customization of medical devices. Robots are increasingly becoming integrated elements of surgical training systems. We propose a robot-assisted laparoscopy training system that extensively utilizes deep reinforcement learning (DRL). By combining exercises, demonstrations from human experts, and RL criteria, our training system aims to improve the trainee’s surgical tool manipulation skills. DRL plays a crucial role in modelling the interaction between biological tissue and surgical tools. Additionally, we explore the application of DRL in surgical gesture recognition. As a pathway to artificial general intelligence, DRL has the potential to transform traditional medical simulation into intelligent simulation.
