Generative Range and Altitude Identity Learning

IDIAP proposes an integrated fusion approach to Deep Neural Network (DNN)-based whole-body biometrics combining gait and face recognition. We will explore both traditional Deep Convolutional Neural Networks (DCCNs) and Vision Transformers (VisTrans) The proposed system consists of 5 stages: - 3D pose estimation – we will train or adapt DCNNs for 3D pose estimation from public synthetic datasets and novel datasets (including partial occlusions and extreme viewpoints) mixing real scenes with synthetic body poses created by a first-order motion model; - Gait recognition – we will train or adapt DCNNs and VisTrans for gait recognition by processing both spatial and temporal information from image sequences of tracked body poses; - Face recognition – we will train or adapt DCNNs for face recognition and adopt a metric learning approach to create an optimal face representation for open set and covert scenarios (low-quality, low-resolution, …); - Compact whole-body representations – we will explore Neural Architecture Search (NAS) techniques to find compact face and gait representations with low-memory footprint and high-speed calculation for fast inference on edge devices; - Fusion – we will develop a fusion strategy to combine face and whole-body representations integrating quality measures from the individual modalities to cope with missing channels and adopt contrastive training (eg. cross-modal focal loss) to factor the confidence of other modalities. IDIAP has long standing experience in multi-modal biometric recognition -- we investigate and develop new biometrics based recognition algorithms, notably for face, voice, and vein biometric modalities; and Presentation attack detection (PAD) -- we look for new and better ways of detecting presentation attacks on face, voice, and vein biometric recognition systems. Moreover IDIAP has a significant experience in multi-modal biometric data collection with well-established methodologies to design acquisition protocols and implement experiment protocols.
University of Southern California
Idiap Research Institute
Intelligence Advanced Research Project Activity
Nov 12, 2021
Nov 11, 2025