Smile detection using haar cascade in OpenCV using Python

We will use the Haar cascade classifier for smile detection in an image. A haar cascade classifier is an effective object detection method. It is a machine learning based approach. To train a haar cascade classifier for smile detection, the algorithm initially needs a lot of positive images (images with smile) and negative images (images without smile). Then the classifier is trained from these positive and negative images. It is then used to detect smiles in other images.

We can use already trained haar cascades for smile detection. For smile detection in the input image, we need two haar cascades one for face detection and other for smile detection. We will use the following two haar cascades −

• haarcascade_frontalface_default.xml

• haarcascade_smile.xml

How to Download Haarcascades?

You can find different haarcascades following the GitHub website address −

To download the smile haarcascade, click on the haarcascade_smile.xml file. Open it in raw format, right click and save.

Steps

To detect smile in an image, you can follow the steps given below −

• Import the required library. In all the following examples, the required Python library is OpenCV. Make sure you have already installed it.

• Read the input image using cv2.imread(). Specify the full image path. Convert the image to grayscale image.

• Initiate the Haar cascade classifier objects face_cascade = cv2.CascadeClassifier() for face detection and smile_cascade = cv2.CascadeClassifier for smile detection. Pass the full path of the haar cascade xml files. You can use the haar cascade file haarcascade_frontalface_default.xml to detect faces in the image and haarcascade_smile.xml to detect smiles.

• Detect faces in the input image using face_cascade.detectMultiScale(). It returns the coordinates of detected faces in (x,y,w,h) format.

• Define face roi as image[y:y+h, x:x+w] for the detected face. Now detect the smile within the detected face area (roi). Use smile_cascade.detectMultiScale(). It also returns the coordinate of the bounding rectangle of eyes in (sx,sy,sw,sh) format.

• Draw the bounding rectangles around the detected smile (mouth) in the original image using cv2.rectangle().

• Display the image with the drawn bounding rectangles around the mouth.

Let's have a look at some examples for more clear understanding.

Example

In this Python program, we perform smile detection in the input image using haar cascade.

# import required libraries
import cv2

# read input image
img = cv2.imread('smile1.jpg')

# convert the image to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# read haar cascade for face detection
face_cascade = cv2.CascadeClassifier('haarcascades\haarcascade_frontalface_default.xml')

# read haar cascade for smile detection
smile_cascade = cv2.CascadeClassifier('haarcascades\haarcascade_smile.xml')

# Detects faces in the input image
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
print('Number of detected faces:', len(faces))

# loop over all the faces detected
for (x,y,w,h) in faces:
   
   # draw a rectangle in a face
   cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,255),2)
   cv2.putText(img, "Face", (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2)
   roi_gray = gray[y:y+h, x:x+w]
   roi_color = img[y:y+h, x:x+w]
 
   # detecting smile within the face roi
   smiles = smile_cascade.detectMultiScale(roi_gray, 1.8, 20)
   if len(smiles) > 0:
      print("smile detected")
      for (sx, sy, sw, sh) in smiles:
         cv2.rectangle(roi_color, (sx, sy), ((sx + sw), (sy + sh)), (0, 0, 255), 2)
         cv2.putText(roi_color, "smile", (sx, sy),
         cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
   else:
      print("smile not detected")

# Display an image in a window
cv2.imshow('Smile Image',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

We will use this image as the Input File for this program −

Output

On execution, it will produce the following output −

Number of detected faces: 1
smile detected

And we get the following output window −

Shahid Akhtar Khan

Updated on: 05-Dec-2022

2K+ Views

Kickstart Your Career

Get certified by completing the course

Get Started