----------------------------
| SCFACE DATABASE PROTOCOL |
----------------------------
| Authors: 
|   Mitchell McLaren, m.mclaren@let.ru.nl
|   Roy Wallace, roy.wallace@idiap.ch

This protocol for the SCface database defines a face authentication/verification scenario, to allow for testing of face verification algorithms in challenging real-world conditions, with substantial lighting, pose and image quality variation. In this scenario, a single high-quality mugshot of each person is assumed to be known. These mugshot images are then used to verify the identity of individuals in low-resolution images captured by a surveillance camera. 

------------
| OVERVIEW |
------------

The protocol is based based on the DayTime tests scenario [1], and utilises images from all 130 subjects in the database. Face verification scores are found by comparing facial images taken by five surveillance cameras at three specified distances (close, medium, far) to a single high-resolution mugshot image. Images are divided into three disjoint sets on a per-subject basis: world, dev and test sets. The filename convention used in the protocol matches that provided with the SCface database such that the subject id is the first field in the image filename:

Visible light mug shot:  {subjectID}_frontal.jpg
Surveillance image:      {subjectID}_{camNum}_{distanceLabel}.jpg  (e.g. 001_cam1_1.jpg) where,
- subjectID = {001-130}
- camNum = {cam1, cam2, cam3, cam4, cam5}
- distanceLabel = {1,2,3} and corresponds to 4.20, 2.60 and 1.00 meters, respectively.


-------------
| WORLD SET |
-------------
The world set consists of 16 images from each of 43 subjects. The list of images is provided in the file 'train_world.lst'. Only the images in this list should be used to train system components such as world/background models, PCA/LDA subspaces, etc., or to otherwise use as background data, for example for score normalisation, etc.

If any other data is used in addition to the files in train_world.lst for these purposes, this should be made absolutely clear in any resulting publication.

-----------
| DEV SET |
-----------
The development (dev) set is defined by ./dev/for_models.lst and ./dev/for_scores.lst. The dev set consists of 16 images from each of 44 subjects. Each line of file ./dev/for_models.lst lists the image to be used to train a subject model in the following format: 

{image1} {modelid1}
{image2} {modelid2}
...      ...
{imageN} {modelidN}

For example, the line "044_frontal 044" refers to use of image 044_frontal.jpg to train model 044. 

The file ./dev/for_scores.lst is a list of images to be compared to each model in the same form as above "{image1} {modelid1}". For instance, 048_cam1_1 044 refers to the comparison of image 048_cam1_1.jpg to model 044. Each such comparison should result in a verification score. It should be clear that same-subject comparisons occur when the first field of the image filename matches the model name. 

The development set only (and NOT the test set) should be used to train system hyperparameters such as the decision threshold, number of dimensions in a subspace, feature extraction and preprocessing hyper-parameters, coefficients for linear fusion, etc., to minimise the chosen error rate metric. We suggest the use of the equal error rate (EER) for this purpose, though other variants are possible.

------------
| TEST SET |
------------
The test set is defined by ./test/for_models.lst and ./test/for_scores.lst. The test set consists of 16 images from each of 43 subjects.  These lists follow the same format as those in the ./dev/ folder. 

The test set should be used to evaluate face verification accuracy. The decision threshold must be determined by tuning on the development set (for example, determining the threshold that results in the equal error rate, EER), and then blindly applied to face verification scores produced on the test set. We suggest that accuracy should be measured by the half total error rate (HTER), which is the average of false acceptance and false rejection rates after applying the threshold.


[1] M. Grgic, K. Delac, and S. Grgic, “SCface-surveillance cameras face database,” Multimedia tools and applications, vol. 51, pp. 863–879, 2011.

