Fig 1.
Augmented reality surgical navigation system for endoscopy.
Fig 2.
Experimental setup for the study on the skull phantom.
Fig 3.
Hand-eye calibration with a moving calibration plate.
Fig 4.
Relationship of the frame transformations.
Fig 5.
The workflow in a surgical scenario.
Overall performances of the image fusion system were evaluated on a plastic skull phantom with a realistic representation of the nasal cavity and adjacent skull base anatomy, including vessels, nerves and the pituitary gland. 1. The skull phantom with optical markers on its surface was positioned on the surgical table. The 3D position of the optical markers was detected by the OTS of the navigation system, to create a VRG for tracking of the phantom’s motion. 2. A CBCT image, co-registered with the 3D position of the optical markers (VRG) was acquired. 3. Anatomical structures of interest were manually segmented from the CBCT image. 4. The endoscope, automatically recognized and tracked by the OTS, was placed in the nasal cavity of the phantom. 5. Segmented structures at the base of the skull were augmented onto the live endoscopic image. The augmented endoscopic view, together with anatomical views to guide the surgeon inside the nasal cavity, were displayed.
Fig 6.
Example of image fusion on the endoscopic view.
Fig 7.
a) Custom-made grid designed for studying the accuracy of the image overlay on the endoscopic view. b) The endoscope was held in a perpendicular position with respect to the grid by means of a surgical arm.
Fig 8.
Boxplots of the errors in the image overlay as a function of the distance of the endoscope from the grid.
Fig 9.
a) Error distribution, expressed in mm, of the image overlay on the endoscopic view at several distances between the endoscope and the grid. Blue represents areas with lower TRE, and red indicates areas with larger TRE. b) Steel spheres segmented from the CBCT and overlaid on the endoscopic view at several distances between the endoscope and the grid.