Leaderboard¶
Next sections presents the leaderboard for each face database and its correspondent evaluation protocols.
Mobio¶
Testing only the mobio-male protocol.
System
ERR (dev)
HTER (eval)
VGG16
2.58%
3.09%
Facenet
0.56%
0.22%
DrGAN
0.8%
2.6%
CasiaNET
16.2%
9.9%
CNN8
14.8%
14.9%
Casia WebFace - Resnetv1 center loss gray
2.46%
1.34%
Casia WebFace - Resnetv1 center loss RGB
1.7%
0.95%
Casia WebFace - Resnetv2 center loss gray
2.77%
1.80%
Casia WebFace - Resnetv2 center loss RGB
1.23%
0.89%
MSCeleb - Resnetv1 center loss gray
1.51%
0.49%
MSCeleb - Resnetv1 center loss RGB
2.07%
0.73%
MSCeleb - Resnetv2 center loss gray
1.63%
0.88%
MSCeleb - Resnetv2 center loss RGB
0.33%
0.29%
ISV
3.2%
7.5%
To run each one of these baselines do:
$ bob bio baseline vgg16 mobio-male
$ bob bio baseline facenet mobio-male
$ bob bio baseline casianet mobio-male
$ bob bio baseline cnn8 mobio-male
$ bob bio baseline idiap_casia_inception_v1_centerloss_gray mobio-male
$ bob bio baseline idiap_casia_inception_v1_centerloss_rgb mobio-male
$ bob bio baseline idiap_casia_inception_v2_centerloss_gray mobio-male
$ bob bio baseline idiap_casia_inception_v2_centerloss_rgb mobio-male
$ bob bio baseline idiap_msceleb_inception_v1_centerloss_gray mobio-male
$ bob bio baseline idiap_msceleb_inception_v1_centerloss_rgb mobio-male
$ bob bio baseline idiap_msceleb_inception_v2_centerloss_gray mobio-male
$ bob bio baseline idiap_msceleb_inception_v2_centerloss_rgb mobio-male
$ bob bio baseline isv mobio-male
Follow below the DET curves for the development and dev sets, and the EPC for the best systems
LFW¶
LFW presents a 10 fold evaluation protocol for open-set identification. Follow below the average True Positive Identification Rate measures under different False Alarm Rates (10 folds averaging).
System
TPIR% (FAR=0.1)
TPIR% (FAR=0.01)
TPIR% (FAR=0.001)
VGG16
Facenet
99.6 (0.66)
98.37 (0.82)
93.13 (3.71)
Dr GAN
97.45 (0.96)
88.41 (1.81)
75.27 (10.12)
CasiaNET
96.81 (0.91)
52.0 (8.87)
13.13 (6.76)
CNN8
96.93 (0.83)
45.55 (11.85)
15.63 (10.98)
Casia WebFace - Resnetv1 cross loss gray
98.29 (0.91)
93.18 (1.57)
79.16 (9.6)
Casia WebFace - Resnetv1 cross loss rgb
98.36 (0.56)
92.53 (1.54)
77.53 (11.13)
Casia WebFace - Resnetv2 center loss gray
98.51 (0.64)
91.68 (2.13)
79.91 (4.97)
Casia WebFace - Resnetv2 center loss rgb
98.58 (0.65)
92.18 (1.68)
78.9 (10.35)
MSCeleb - Resnetv2 center loss rgb
99.77 (0.19 )
99.18 (0.43 )
77.75 (30.82)
Since these protocols are open-set, another way analyse this dataset is via an Open Set Identification evaluation. Follow below Detection Identification Rate for the first fold.
IJB-C¶
Verification protocol 1:1¶
This section presents the results for verification (1:1) protocol. Check here for more details.
System |
TPIR% (FAR=0.1) |
TPIR% (FAR=0.01) |
TPIR% (FAR=0.001) |
---|---|---|---|
VGG16 |
|||
Facenet |
97.137 |
85.944 |
64.979 |
Dr GAN |
90.397 |
62.152 |
31.912 |
CasiaNET |
17.799 |
4.341 |
0.92 |
CNN8 |
17.871 |
4.709 |
0.997 |
Casia WebFace - Resnetv1 center loss gray |
90.597 |
67.945 |
41.402 |
Casia WebFace - Resnetv1 center loss rgb |
90.985 |
68.4 |
42.041 |
Casia WebFace - Resnetv2 center loss gray |
90.806 |
66.754 |
39.577 |
Casia WebFace - Resnetv2 center loss rgb |
90.633 |
67.388 |
41.837 |
MSCeleba - Resnetv2 center loss rgb |
99.0 |
91.55 |
62.53 |
Follow below the ROC curve of the three best evaluated systems.