Python – Eye blink detection project

In this tutorial you will learn about detecting a blink of human eye with the feature mappers knows as haar cascades. Here in the project, we will use the python language along with the OpenCV library for the algorithm execution and image processing respectively. The haar cascades we are going to use in the project are pretrained and stored along with the OpenCV library as haarcascade_frontalface_default.xml and haarcascade_eye_tree_eyeglasses.xml files. The project develops a basic understanding of the systems such as driver drowsiness detection, eye blink locks, eye detection, face detection and also the haar cascades usage with the OpenCV library.
About Haar Cascades: 
Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, “Rapid Object Detection using a Boosted Cascade of Simple Features” in 2001. It is a machine learning-based approach where a cascade function is trained from a lot of positive and negative images. Here positive images are the samples which contain the target object and negative are the ones which don’t. It takes a lot of positive and negative samples to train the classifier.
 

Haar features , source – OpenCV docs

Now, we extract the features from the given input image with the haar features shown in the above image. They are just like convolutional kernels. Each feature is a single value obtained by subtracting the sum of pixels under the white rectangle from the sum of pixels under the black rectangle. 
 

Application of haarcascades, source – OpenCV Docs

The Excessive calculation: 
With all the possible sizes of the classifiers the features are calculated, but the amount of computation it takes to calculate the features, a 24×24 window results over 160000 features. Also for each feature calculation, the sum of the pixels is also needed. To make it computationally less expensive the creators of haar cascades introduced the integral image, which means however large your image, it reduces the calculations for a given pixel to an operation involving just four pixels.
The false features 
Now among the features that are calculated, most of the features are false and irrelevant. Now the window which is applied to a region of the image may see a different region which seems with the same features to the window but is not in reality. So, there is a need to remove the false features which were done by the AdaBoost which helped select the best features out of 160000+ features. Adaboost short form of Adaptive Boosting is a Machine learning algorithm which was used for this sole task.
Algorithm : 
 

The frame is captured and converted to grayscale.
Bilateral Filtering is applied to remove impurities.
Face is detected with the haarcascade.
The ROI (Region Of Image) of Face is fed to eye detection part of algorithm.
Eyes are detected and resulting list is passed to if-else construct.
If the length of list is more than two, means that the eyes are there.
Else the program is marked to be eye blinked and restarted.

Code: 
The haarcascade_frontalface_default.xml and haarcascade_eye_tree_eyeglasses.xml are the xml files stored in the same directory as the python script. 
 

Sample Output: 
 

Sample run of above code

The Source code and the cascade classifiers can be found here.