SIFT Algorithm for Image Comparison

SIFT stands for Scale-Invariant Feature Transform. This is used for comparison of images to check for similarities.

We will check this out step by step with codes and open CV.

Let's first check which images are we going to inspect using SIFT.

Images for Test

These are the two images and it can clearly be seen there are high similarities and I have simply moved my hands' position.

Let's get started. I will first read both the images in grayscale.

import cv2
img1 = cv2.imread("Path to image 1",0)
img2 = cv2.imread("Path to image 2",0)

The SIFT algorithm is based on Feature Detection and Feature Matching. In simple terms, if you want to understand this, we know an image is stored as a matrix of pixel values. The SIFT algorithm takes small regions of these matrices and performs some mathematical transformations and generates feature vectors which are then compared. For more details, you can check official OpenCV notes here.

For the SIFT algorithm, we need to detect the Keypoints and descriptions for comparison. Let us try to detect those.

# check for similarities
sift = cv2.xfeatures2d.SIFT_create()
# check keypoints and descriptions of images
kp_1,desc_1 = sift.detectAndCompute(img1,None)
kp_2,desc_2 = sift.detectAndCompute(img2,None)

Next, we need to compare the Keypoints and Descriptions of the two images. I use the code from Pysource from here for this.

index_params = dict(algorithm=0, trees=5)
search_params = dict()
flann = cv2.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(desc_1, desc_2, k=2)
result = cv2.drawMatchesKnn(img1,kp1,img2,kp2,matches,None)

Let's display and check the Mapping between the images.

cv2.imshow("Correlation", result)
cv2.imshow("Image 1", img1)
cv2.imshow("Image 2", img2)

Correlated Images

The Mapping of both images is shown above. It's not clear. You would need to run the codes to understand it better.

The Advantage of SIFT for comparison :

It is not affected by :

1. Rotation
2. Scaling
3. Image Brightness