Smile detection using haar cascade in OpenCV using Python

We will use the Haar cascade classifier for smile detection in an image. A haar cascade classifier is an effective object detection method. It is a machine learning based approach. To train a haar cascade classifier for smile detection, the algorithm initially needs a lot of positive images (images with smile) and negative images (images without smile). Then the classifier is trained from these positive and negative images. It is then used to detect smiles in other images.

We can use already trained haar cascades for smile detection. For smile detection in the input image, we need two haar cascades one for face detection and other for smile detection. We will use the following two haar cascades −

  • haarcascade_frontalface_default.xml

  • haarcascade_smile.xml

How to Download Haarcascades?

You can find different haarcascades following the GitHub website address −

To download the smile haarcascade, click on the haarcascade_smile.xml file. Open it in raw format, right click and save.


To detect smile in an image, you can follow the steps given below −

  • Import the required library. In all the following examples, the required Python library is OpenCV. Make sure you have already installed it.

  • Read the input image using cv2.imread(). Specify the full image path. Convert the image to grayscale image.

  • Initiate the Haar cascade classifier objects face_cascade = cv2.CascadeClassifier() for face detection and smile_cascade = cv2.CascadeClassifier for smile detection. Pass the full path of the haar cascade xml files. You can use the haar cascade file haarcascade_frontalface_default.xml to detect faces in the image and haarcascade_smile.xml to detect smiles.

  • Detect faces in the input image using face_cascade.detectMultiScale(). It returns the coordinates of detected faces in (x,y,w,h) format.

  • Define face roi as image[y:y+h, x:x+w] for the detected face. Now detect the smile within the detected face area (roi). Use smile_cascade.detectMultiScale(). It also returns the coordinate of the bounding rectangle of eyes in (sx,sy,sw,sh) format.

  • Draw the bounding rectangles around the detected smile (mouth) in the original image using cv2.rectangle().

  • Display the image with the drawn bounding rectangles around the mouth.

Let's have a look at some examples for more clear understanding.


In this Python program, we perform smile detection in the input image using haar cascade.

# import required libraries import cv2 # read input image img = cv2.imread('smile1.jpg') # convert the image to grayscale gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # read haar cascade for face detection face_cascade = cv2.CascadeClassifier('haarcascades\haarcascade_frontalface_default.xml') # read haar cascade for smile detection smile_cascade = cv2.CascadeClassifier('haarcascades\haarcascade_smile.xml') # Detects faces in the input image faces = face_cascade.detectMultiScale(gray, 1.3, 5) print('Number of detected faces:', len(faces)) # loop over all the faces detected for (x,y,w,h) in faces: # draw a rectangle in a face cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,255),2) cv2.putText(img, "Face", (x, y), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2) roi_gray = gray[y:y+h, x:x+w] roi_color = img[y:y+h, x:x+w] # detecting smile within the face roi smiles = smile_cascade.detectMultiScale(roi_gray, 1.8, 20) if len(smiles) > 0: print("smile detected") for (sx, sy, sw, sh) in smiles: cv2.rectangle(roi_color, (sx, sy), ((sx + sw), (sy + sh)), (0, 0, 255), 2) cv2.putText(roi_color, "smile", (sx, sy), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2) else: print("smile not detected") # Display an image in a window cv2.imshow('Smile Image',img) cv2.waitKey(0) cv2.destroyAllWindows()

We will use this image as the Input File for this program −


On execution, it will produce the following output −

Number of detected faces: 1
smile detected

And we get the following output window −

Updated on: 05-Dec-2022

2K+ Views

Kickstart Your Career

Get certified by completing the course

Get Started