Personal tools
You are here: Home Dataset Head Pose Database ICME 05 Set Up

ICME 05 Set Up

Head Pose Tracking Algorithms Evaluation

In our paper

Evaluation of Multiple Cues Head Pose Estimation Algorithms in Natural Environments
Sileye Ba and Jean-Marc Odobez [pdf file]
in Proceedings of the International Conference on Multi-media & Expo (ICME), Amsterdam, 2005

We ran experiments to compare two classes of head pose tracking algorithm: a first class where the head tracking and the pose estimation are performed one after the other, and a second class where the head tracking and pose estimation are optimized jointly. The evaluation data and the errors measures are given in the following.

Evaluation Data:

The tracking evaluation protocol is the following. For our experiments we use half of the persons of the meeting database as train set to train pose dynamic model and the half remaining persons as test set to evaluate the tracking algorithms. In each one of the recording of the 8 persons of the test set, we selected 1 minute of recording (1500 video frames) for evaluation data. We decided to use only one minute to save machine computation time, as we use a quite slow matlab implementation of our algorithms. Video frames corresponding to training and test data are given in the following table:

Traning Data

Test Data

 Recording  Video Frames  Recording  Video Frames
 Meeting 3 L  6001-7500  Meeting 1 R  1-1500
 Meeting 4 L  13501-15000  Meeting 1 L  4051-6000
 Meeting 5 L  13501-15000  Meeting 2 R  1501-3000
 Meeting 6 R  12001-13500  Meeting 2 L   6001-7500
 Meeting 6 L  7501-9000  Meeting 3 R   9001-10500
 Meeting 7 R  6001-7500  Meeting 4 R   9001-10500
 Meeting 7 L  18001-18500  Meeting 5 R   9001-10500
 Meeting 8 R  7501-9000  Meeting 8 L   16501-18000


Errors Measures:

In this paragraph, we define the head pose estimation error measures used to evaluate tracking performances. A head pose defines a vector in the 3D space, the vector indicating where the head is pointing at. It can be thought of as a vector based on the center of the head and passing through the nose. It is worth noticing that in the Pointing representation, this vector depends only on the head pan and tilt angles. The angle between the 3D pointing vector defined by the head pose ground truth (GT) and the head pose estimated by the tracker can be used as the first pose estimation error measure. In order to have more details about the origins of the errors we will also measure the individual errors made separately on the pan, tilt and roll angles measured in the Pointing representation. For each one of the four error measures, we compute the mean, standard deviation, and median value of the absolute value of the errors. These set of errors measures can be computed using functions available in the folder ROUTINES of the database (HeadPosetrackingErr.m).

Document Actions