AugGen: Synthetic Augmentation using Diffusion Models Can Improve Recognition

Jan 1, 2025·
Parsa Rahimi
,
Damien Teney
Prof. Sébastien Marcel
Prof. Sébastien Marcel
· 0 min read
Abstract
The increasing reliance on large-scale datasets in machine learning poses significant privacy and ethical challenges, particularly in sensitive domains such as face recognition. Synthetic data generation offers a promising alternative; however, most existing methods depend heavily on external datasets or pre-trained models, increasing complexity and resource demands. In this paper, we introduce AugGen, a self-contained synthetic augmentation technique. AugGen strategically samples from a class-conditional generative model trained exclusively on the target FR dataset, eliminating the need for external resources. Evaluated across 8 FR bench- marks, including IJB-C and IJB-B, our method achieves 1-12% performance improvements, outperforming models trained solely on real data and surpass- ing state-of-the-art synthetic data generation approaches, while using less real data. Notably, these gains often exceed those from architectural enhancements, underscoring the value of synthetic augmentation in data-limited scenarios. Our findings demonstrate that carefully integrated synthetic data can both mitigate pri- vacy constraints and substantially enhance recognition performance.
Type
Publication
Conference on Neural Information Processing Systems
publications
Face Recognition Synthetic Data
Prof. Sébastien Marcel
Authors
Prof. Sébastien Marcel (he)
Senior Research Scientist
Prof Sébastien Marcel (IEEE Fellow and IAPR Fellow) is a senior research scientist at the Idiap Research Institute (Switzerland), he heads the Biometrics Security and Privacy group and conducts research on face recognition, speaker recognition, vein recognition, attack detection (presentation attacks, morphing attacks, deepfakes) and template protection. He is also Professor at the University de Lausanne (UNIL) at the School of Criminal Justice. He is also the Director of the Swiss Center for Biometrics at Idiap, which conducts certifications of biometric products. He was Associate Editor and Guest Editor of IEEE journals (TBIOM, SPL, TIFS and SPM). He is also the lead Editor of the Springer Handbook of Biometrics Anti-Spoofing (Editions 1, 2 and 3). Since June 2025 he is a member of the Idiap Direction ad interim.