Heterogeneous Face Recognition (HFR) aims at matching face images captured across different sensing modalities, such as thermal-to-visible or near-infrared-to-visible, enhancing the usability of face recognition systems in challenging real-world conditions. Although recent HFR methods have achieved significant improvements in performance, many rely on computationally expensive models, making them impractical for deployment on resource-limited edge devices. In this work, we introduce a lightweight yet effective HFR framework by adapting a hybrid CNN-Transformer model originally developed for RGB homogeneous face recognition. Our approach enables efficient end-to-end training with only a small amount of paired heterogeneous data, while still maintaining strong performance on standard RGB face recognition benchmarks. This makes it suitable for both homogeneous and heterogeneous settings. Comprehensive experiments on several challenging HFR and face recognition benchmarks show that our method achieves state-of-the-art or competitive performance while keeping computational requirements low.
xEdgeFace tackles the long-standing challenge of heterogeneous, or cross-spectral, face recognition, where visible-light mugshots must be matched to faces captured in other spectra such as thermal or near-infrared. This capability is essential for surveillance and low-light authentication, but it remains challenging for conventional RGB-trained models and difficult to deploy on resource-constrained devices.
The core insight of this work is that most modality discrepancies can be mitigated by adapting only a minimal set of low-level convolutional layers and the LayerNorm statistics of a lightweight CNN-Transformer backbone, EdgeFace. The remaining network components are kept frozen, while a contrastive self-distillation loss preserves the original RGB discriminative power.
This yields a single compact network, as small as 0.09 GFLOPs and 1.24M parameters, that:
We show that selective LayerNorm tuning plus self-distillation is sufficient to extend an off-the-shelf tiny face-recognition network to robust cross-spectral recognition, enabling real-time edge deployment without additional synthesis pipelines or modality-specific branches.
We evaluate the computational efficiency of xEdgeFace by reporting two key metrics: the number of floating-point operations, measured in GFLOPs, and the total number of parameters, measured in millions of parameters. The proposed xEdgeFace variants operate with significantly reduced computational overhead and parameter count, highlighting their suitability for deployment in resource-constrained environments.
Comparison of the size, in million parameters, and compute, in GFLOPs, of state-of-the-art HFR models against the xEdgeFace variants.
xEdgeFace improves performance across new sensing modalities without degrading accuracy on the original RGB benchmarks. The self-distillation regularization prevents catastrophic forgetting, resulting in a compact model that excels in both homogeneous and cross-spectral face recognition.
xEdgeFace achieves high accuracy across RGB and cross-spectral benchmarks using a single compact model.
The figure below shows the performance improvement of xEdgeFace on the VIS-Thermal protocol in the MCXFace dataset. The proposed xEdgeFace approach outperforms all compared methods, achieving the highest average Rank-1 accuracy of 91.68%.
Performance on the MCXFace dataset using the VIS-Thermal protocol.
xEdgeFace achieves state-of-the-art or competitive performance across several challenging heterogeneous face-recognition benchmarks while maintaining a compact computational footprint.
The MCXFace dataset used in this paper is publicly available from Idiap Research Institute.
@article{george2026lightweight,
title={Lightweight Cross-Spectral Face Recognition via Contrastive Alignment and Distillation},
author={George, Anjith and Marcel, S{\'e}bastien},
journal={IEEE Transactions on Biometrics, Behavior, and Identity Science},
year={2026},
publisher={IEEE}
}