NA-MIC Project Weeks

Back to Projects List

Surgical planning system for colorectal surgery based on Mixed Reality technology

Key Investigators

• Juan A. Sánchez-Margallo (Jesús Usón minimally invasion surgical centre, Spain)
• Daniel Caballero (Jesús Usón minimally invasion surgical centre, Spain)
• Lucia Salazar-Carrasco (Jesús Usón minimally invasion surgical centre, Spain)

Presenter location: Online

Project Description

The objective of this project is to implement the required tools for the development of a surgical planning system for colorectal interventions combining segmentation techniques and Mixed Reality (MR) technology. These tools are based on CT studies using 3D slicer as a graphical support and python as a programming language, as well as Unity as a MR engine.

Objective

1. Objective A. The general idea is to implement a semi-automatic method of the colon anatomy using a CT scan under Slicer 3D platform. This 3D model of the colon will be used for marking interest points such as lesions (polyps and tumors) and resection planes for the subsequent surgical intervention.
2. Objective B. To mplement the result of the project with the model that we obtain in 3D Slicer using Mixed Reality (Hololens). This wil offer surgeons an interactive aplication based on MR for the surgical planning of the subsequent colorectal intervention, including the previously defined points of interest (lesions) and resection lines.

Approach and Plan

1. Analyze the existing literature about the tools to be used in this project (3D Slicer, Python, and Unity).
2. Optimize the segmetation process of the 3D model.
3. Improve the method of annotation of the points of interest.
4. Improve the method of annotation of the resection lines.
5. Import the 3D model that we obtain in 3DSlicer to Unity.
6. Do the scripts we need in Unity with the model to be read.
7. Show the points of interest and the resection lines in the model with the information that we obtain in 3DSlicer.
8. Upload the final 3D model to the MR device (Hololens) with all the information for the surgeon interation.
9. Use the aplication in the surgical site to improve the planning of colorectal surgical activities.

Progress and Next Steps

1. We have done the semi-automatic method of the colon anatomy using a CT scan under Slicer 3D platform based on region growing algorithms.
2. We are working in the scripts in Unity to develop the MR application, so that we can read the 3D model and points of interest and resection lines of the surgical planning.

Illustrations

Background and References

• C Lobato-Gómez, L Salazar-Carrasco, JA Sánchez-Margallo, FM Sánchez-Margallo. Diseño e implementación de un planificador quirúrgico para cirugía colorrectal junto con un visualizador basado en realidad mixta. CASEIB 2023 Cartagena, Spain.
• C Lobato, E Pedregosa, JA Sánchez-Margallo, I Sánchez-Varo, B Durán, FM Sánchez-Margallo. Design of a colon model for training in transanal surgery using semi-automatic segmentation tools and 3D printing techniques. 34th Annual International Society for Medical Innovation and Technology Conference (iSMIT 2023). 19/10/23-21/10/23, Lukang (Taiwan).
• Pedregosa E, Sánchez-Margallo JA, Lobato C, Sánchez-Varo I, Plaza C, Durán B, Sánchez-Margallo. Preliminary validation of training tools for colorectal surgery based on 3D printing and mixed reality. 31st International Congress of the European Association for Endoscopic Surgery (EAES). 20/6/23-23/6/23, Roma (Italia).
• Juan A. Sánchez Margallo, Carlos Lobato Gómez, Francisco M. Sánchez Margallo. Colon segmentation using 3DSlicer. Certamen Internacional de Cine Médico y Salud (VIDEOMED 2022). 28/11/22-1/12/22, Badajoz (España).
• JA Sánchez-Margallo. The role of mixed reality in MIS. 30th International Congress of the European Association for Endoscopic Surgery (EAES). 5/7/22-8/7/22, Kraków (Poland).