The ArthroNav Project

Computer Assisted Navigation in Orthopedic Surgery using Endoscopic Images

Image Features

Keypoint detection and matching is of fundamental importance for many applications in computer and robot vision. The association of points across different views is problematic because image features can undergo significant changes in appearance due to camera motion. State-of-the-art methods, such as Scale Invariant Feature Transform (SIFT), allow efficient matching under common image transformations such as scale, affine changes in viewpoint and illumination changes. Unfortunately, these approaches are not resilient to the radial distortion that arises in images acquired by endoscopes due to lens miniturization.

We propose a model-based modification of the SIFT method, dubbed sRD-SIFT, that often duplicates the number of correct point correspondences, while keeping a high localization accuracy. sRD-SIFT requires a coarse estimation of the amount of image distortion and introduces a small computational overhead when compared with standard SIFT. Nevertheless, it is significantly faster than explicit distortion correction via image warping, and provides much better results. This is illustrated by a simple experiment that considers two endoscopic images of a knee model

Left view

Right view


The figures below show respectively the point matches obtained using standard SIFT (left), SIFT after correcting the distortion by image warping (right), and sRD-SIFT(bottom). Correct (green) and incorrect (red) matches are labeled by computing the epipolar geometry. It can been seen that SIFT barely provides correct correspondences, rectSIFT obtains some matches in textured regions, and sRD-SIFT provides correspondences all over the image.

Sift result

rectSift result

sRD-SIFT result



The table summarizes the results above, with the first column referring to the total number of matches, the second and third column showing the number and percentage of inliers, and the last column the re-projection error after Structure-from-motion (SfM). The superiority of sRD-SIFT is very clear.



sRD-SIFT can be advantageous in medical applications that rely in endoscopic imagery and also in more general computer vision tasks, ranging from SfM to visual recognition. A matlab implementation of the algorithm was made public here.

Other results