Mobio Dataset, female subjects

Dataset Description

Mobio is a collection of English voice recordings. The set of female subjects contains:

		Identities	Sample count
train		13	2406
dev	references	18	90
	probes		1890
eval	references	20	100
	probes		2100

GMM

Table 7 [Min. criterion: EER] Threshold on Development set: 7.550647e-01

	Development	Evaluation
Failure to Acquire	0.0%	0.0%
False Match Rate	20.6% (6632/32130)	7.8% (3093/39900)
False Non Match Rate	20.6% (390/1890)	26.8% (562/2100)
False Accept Rate	20.6%	7.8%
False Reject Rate	20.6%	26.8%
Half Total Error Rate	20.6%	17.3%

Command used to generate scores:

$ bob bio pipeline -d mobio-audio-female gmm-mobio -g dev -g eval -l sge -o results/gmm_mobio_female

On 1281 CPU nodes on the SGE Grid: Ran in 15 minutes (11 minutes of training).

ISV

Table 8 [Min. criterion: EER] Threshold on Development set: 3.483318e-01

	Development	Evaluation
Failure to Acquire	0.0%	0.0%
False Match Rate	14.7% (4710/32130)	20.3% (8103/39900)
False Non Match Rate	14.7% (277/1890)	17.4% (366/2100)
False Accept Rate	14.7%	20.3%
False Reject Rate	14.7%	17.4%
Half Total Error Rate	14.7%	18.9%

Command used to generate scores:

$ bob bio pipeline -d mobio-audio-female -p isv-voxforge -g dev -g eval -l sge -o results/isv_mobio_female

On 1281 CPU nodes on the SGE Grid: Ran in 8 minutes (3 minutes of training).

Speechbrain ECAPA-TDNN

Table 9 [Min. criterion: EER] Threshold on Development set: -5.091601e-01

	Development	Evaluation
Failure to Acquire	0.0%	0.0%
False Match Rate	1.9% (616/32130)	10.8% (4307/39900)
False Non Match Rate	1.9% (36/1890)	2.5% (52/2100)
False Accept Rate	1.9%	10.8%
False Reject Rate	1.9%	2.5%
Half Total Error Rate	1.9%	6.6%

Command used to generate scores:

$ bob bio pipeline -d mobio-audio-female -p speechbrain-ecapa-voxceleb -g dev -g eval -l sge -o results/speechbrain_mobio_female

On 1281 CPU nodes on the SGE Grid: 12 minutes (no training).

Footnotes

1(1,2,3)

The number of nodes is a requested maximum amount and can vary depending on the number of jobs currently running on the grid as well as the scheduler’s load estimation. The execution time can then also vary.