Face Recognition Punch Clock

General Details

Workshop Instructor: Dr. Lior Wolf

Workshop Assistant: Assaf Zaritsky

Group Members: Tal Rice, Merav Meir

Project website: http://frpc.doodlekit.com

Introduction

The goal of our program is to replace a punch clock mechanism. The program follows the entrance and departure of employees to and from the work place and logs the time. In order to do that, we use two video cameras. The program will detect the faces of passers-by from the video, and then perform face recognition relying on a database of the employees' pictures.

The Algorithm

There are 3 basic stages in the face recognition algorithm: first, collecting face image data from the images in the database. We use Viola Jones algorithm for locating and detecting faces in the obtained images, and as a face descriptor we use Local Binary Patterns (LBP). Secondly, train the face recognition model, meaning to “teach” a model the facial features of each person's face (the data obtained in the first stage); in this project we use a Support Vector Machine (SVM) as a model. Finally, given a sample video (which we want to recognize a face from), the algorithm tries to detect a face in the video. When a face is detected its LBP vector is computed, and it is recognized based on the SVM model which was produced in the training stage. Once a worker's face is recognized while entering or exiting the work place, his/her information is updated accordingly. Basically, we assume that there is one camera positioned in the entrance, and another camera in the exit. Therefore the recognition algorithm is run in parallel in 2 threads respectively.

1. Face detection

For this stage we used OpenCV's face detector. We used the Haar classifier cascade which implements the Viola Jones face detection algorithm.

2. Face Description using Local Binary Patterns (LBP)

We used Local Binary Patterns for face representation. LBP has found to be a powerful feature for texture classification and face recognition. The main concept: we should look locally at each pixel of the image and threshold every neighbor of this specific pixel, by the specific pixel itself. The bits which represent the comparison results will be then translated into a binary number. The histogram of this numbers will be used as a signature which describes the image texture.

For example: consider the following central pixel and its 8 neighbors:

We will perform the threshold, and will get:

This will be then encoded to the binary number: 01101001.

3. Images Training using Support Vector Machine (SVM)

SVM is supposed to give the best results for LBP face representation. In our implementation we used libSVM. We used a multiclass SVM that performs classification of K individuals.

4. Algorithm Acceleration

At first we tried to parallelize the algorithm by using Intel Parallel studio tools, as Cilk and TBB. Due to insufficient results, we tried to use another technique: instead of computing the LBP feature for each pixel in the detected face image, we calculate it once for the whole image. The concept is to create 8 new images, while each image will be a shift of the original image by a different direction, i.e.: we will convert each pixel (x, y) in the photo to each of the following pixels: (x+1, y), (x-1, y), (x, y+1), (x, y-1), (x+1, y+1), (x+1, y-1), (x-1, y+1), (x-1, y-1) this way we will get 8 different images. The next stage is to subtract the original image from each one of the new images. By doing that we get another 8 new images. For computing LBP for a specific pixel (x, y), we will look at each one of the (x, y) pixels, in the different 8 new images. For example, let's name the new image we created by shifting the original image by (x+1, y) Img1. Considering we want to compute the (x, y) pixel's neighbor, which is located on top of this pixel, we should look at the pixel (x, y) at the image which is the subtraction of Img1 from the original image. This way we do all image needed calculations at once, instead of calculating it for each pixel separately. Combining this technique with performing detection operation on every other frame, we get a good improvement in performance.

The Application

The application allows an employer to add and remove employees from the database, edit their information (such as weekly schedule, rank etc.) and display the current information; including viewing reports of each employees' tardiness in the last week they worked as well as salary in the last 12 months they worked.

Project Related Links

http://www.cs.tau.ac.il/~wolf/papers/jpatchlbp.pdf:

"Face Recognition by Combining Multiple Descriptors and Learned Background Statistics”

http://upcommons.upc.edu/pfc/bitstream/2099.1/10772/1/PFC_LauraSanchez_(LBP_applied_to_FaceDetection%26Recognition).pdf: "Local Binary Patterns applied to face detection and recognition"

http://www.csie.ntu.edu.tw/~cjlin/libsvm/ : LibSVM
http://opencv.willowgarage.com/wiki/ : OpenCV Wiki-pages