Evaluating the Effectiveness of Attack-Agnostic Features for Morphing Attack Detection

Laurent Colbois^1,2 and Sébastien Marcel^1,2

¹Idiap Research Institute, ²UNIL

IEEE 2024 International Joint Conference on Biometrics
Special Session on Face Morphing Attack and Detection Techniques (FMADT-2024)

Paper DOI arXiv Reproducibility code Dataset Morph generation

Examples of generated morphs using as source dataset respectively FRGC (first row), FRLL (second row) and FFHQ (third row). The first and last column show the two real sources for which a morph must be created, and other columns show the results using each considered morphing algorithm.

Summary

Morphing attacks have diversified significantly over the past years, with new methods based on generative adversarial networks (GANs) and diffusion models posing substantial threats to face recognition systems. Recent research has demonstrated the effectiveness of features extracted from large vision models pretrained on bonafide data only (attack-agnostic features) for detecting deep generative images. Building on this, we investigate the potential of these image representations for morphing attack detection (MAD). We develop supervised detectors by training a simple binary linear SVM on the extracted features and one-class detectors by modeling the distribution of bonafide features with a Gaussian Mixture Model (GMM). Our method is evaluated across a comprehensive set of attacks and various scenarios, including generalization to unseen attacks, different source datasets, and print-scan data. Our results indicate that attack-agnostic features can effectively detect morphing attacks, outperforming traditional supervised and one-class detectors from the literature in most scenarios. Additionally, we provide insights into the strengths and limitations of each considered representation and discuss potential future research directions to further enhance the robustness and generalizability of our approach.

🔍 Key Highlights

🚀 Novel approach: Uses attack-agnostic features for morphing attack detection.
🏆 Outperforms: Traditional CNN-based detection methods, in many generalization scenarios.
🔄 Generalization: Works across different attack types, source datasets, and print-scan images.
📂 Reproducibility: Open-source code dataset available.

Proposed Methodology

Recent advancements in deepfake detection have demonstrated the unexpected effectiveness of using internal features from large vision models trained exclusively on real data. These features, which are then attack-agnostic, can be used in conjunction with simple downstream classifiers to perform detection. Notably, features extracted using pre-trained CLIP models, originally trained for image-caption alignment, have shown promise in previous studies.

This study focuses on evaluating the applicability of attack-agnostic features for MAD. Specifically:

Attack-Agnostic Supervised MAD: We develop and evaluate MAD systems using simple probe classifiers trained on attack-agnostic feature representations.
One-Class MAD: We develop and evaluate MAD systems based on one-class modeling of the bona fide class, and detecting morphs as out-of-distribution samples, an approach which is enabled by the use of attack-agnostic representations.
Comparison to Supervised CNNs: We compare our methodology against traditional supervised convolutional neural network (CNN) training, through extensive experiments involving three different datasets and five types of morphing attacks spanning three categories: landmark-based, GAN-based, and diffusion-based.
Systematic Generalization Study: Our evaluation includes a variety of scenarios, focusing on the generalization capabilities across different families of attacks, across source datasets, and across domains (digital to print-scan).

We tackle the problem of morphing attack detection (MAD) using pretrained attack-agnostic extractors. Morph generation: we generate morphs using a variety of algorithms (landmark-based, GAN-based, and diffusion-based). MAD stage 1: the attack-agnostic extractor is a large vision model trained on real images for a pretext task. We reuse it to summarize any image by extracting an internal representation as the feature vector. MAD stage 2: features are extracted for bonafide images and face morphs. We train a supervised morphing attack detector as a linear SVM on top of this features space. We train a one-class detector by modeling the distribution of bonafide features with a GMM, then using the likelihood of incoming samples as the discriminative score.

📈 DET Curve Analysis

The following DET curves illustrate the performance of various models under different evaluation settings. The scenarios include:

Baseline: Models trained and evaluated on attacks from the same source dataset, in the digital domain, with all attacks seen at training.
Source Generalization: Same as baseline, but the source dataset differs between training and evaluation.
Unseen Attacks Generalization: Same as baseline, but only a single family of attacks is seen at training, either LB (Landmark-based), GAN, or DIFF (Diffusion).
Print-Scan Generalization: Same as baseline, but the evaluation is done using print-scan versions of the images.
One-Class Detection: Same as baseline, but no attacks are seen during training (only bona fide samples), and attacks are detected as anomalies (out-of-distribution samples).

Baseline
Source Generalization
Unseen Attacks
Print-Scan Generalization
One-Class

Baseline: FRGC dataset, all attacks seen at training, evaluated in digital domain

Baseline: FFHQ dataset, all attacks seen at training, evaluated in digital domain

Source Generalization: FRGC -> FRLL, all attacks seen at training

Source Generalization: FFHQ -> FRLL, all attacks seen at training

Unseen Attacks: FRGC dataset, only Landmark-based attacks seen at training

Unseen Attacks: FRGC dataset, only GAN-based attacks seen at training

Unseen Attacks: FRGC dataset, only Diffusion-based attacks seen at training

Unseen Attacks: FFHQ dataset, only Landmark-based attacks seen at training

Unseen Attacks: FFHQ dataset, only GAN-based attacks seen at training

Unseen Attacks: FFHQ dataset, only Diffusion-based attacks seen at training

One-Class Model: FRGC dataset, trained only on bona fide samples

One-Class Model: FFHQ dataset, trained only on bona fide samples

📌 Key Takeaways

Baseline Performance: Models trained and tested on the same dataset with all attack types achieve near-optimal detection rates, especially in constrained datasets like FRGC.
Source Generalization: Generalizing to a different dataset is challenging, with noticeable performance degradation, particularly when transitioning from FRGC to FRLL. FFHQ enables better generalization due to its diversity.
Unseen Attacks Generalization: Our approach typically outperforms CNN training when testing on attack types unseen during training. AIM-based and CLIP-based features exhibit better robustness across different attack families.
Print-Scan Generalization: The print-scan transformation significantly degrades performance, indicating the need for training on real-world print-scan images to ensure robustness. However, DINOv2 shows surprisingly strong generalization in this setting, outperforming many other feature extractors.
One-Class Detection: When trained only on bona fide samples, models such as the one based on DNADet features maintain strong detection capabilities, for a constrained source dataset (FRGC).

Dataset Code Availability

New 🚀: The FFHQ-Morphs is now available at the following link: https://www.idiap.ch/dataset/ffhq-morphs.

Morph generation: The code for regenerating morphs is now available at the following link: https://gitlab.idiap.ch/biometric/morphgen.

Reproducibility: The code for reproducing the MAD experiments is available at the following link: https://gitlab.idiap.ch/bob/bob.paper.ijcb2024_agnostic_features_mad.

Try the Code Explore the Dataset Generate Morphs

BibTeX

 @INPROCEEDINGS{colbois_agnostic_features_mad,
        author={Colbois, Laurent and Marcel, Sébastien},
        booktitle={2024 IEEE International Joint Conference on Biometrics (IJCB)},
        title={Evaluating the Effectiveness of Attack-Agnostic Features for Morphing Attack Detection},
        year={2024},
        volume={},
        number={},
        pages={1-9},
        keywords={Training;Support vector machines;Systematics;Detectors;Feature extraction;Solids;Robustness;Data models;Data mining;Faces},
        doi={10.1109/IJCB62174.2024.10744532}}
      }