FantasyID

Abstract

Advancements in image generation led to the availability of easy-to-use tools for malicious actors to create forged images. These tools pose a serious threat to the widespread Know Your Customer (KYC) applications, requiring robust systems for detection of the forged Identity Documents (IDs). To facilitate the development of the detection algorithms, in this paper, we propose a novel publicly available (including commercial use) dataset, FantasyID, which mimics real-world IDs but without tampering with legal documents and, compared to previous public datasets, it does not contain generated faces or specimen watermarks. FantasyID contains ID cards with diverse design styles, languages, and faces of real people. To simulate a realistic KYC scenario, the cards from FantasyID were printed and captured with three different devices, constituting the bonafide class. We have emulated digital forgery/injection attacks that could be performed by a malicious actor to tamper the IDs using the existing generative tools. The current state-of-the-art forgery detection algorithms, such as TruFor, MMFusion, UniFD, and FatFormer, are challenged by FantasyID dataset. It especially evident, in the evaluation conditions close to practical, with the operational threshold set on validation set so that false positive rate is at 10%, leading to false negative rates close to 50% across the board on the test set. The evaluation experiments demonstrate that FantasyID dataset is complex enough to be used as an evaluation benchmark for detection algorithms

Dataset

Our dataset is split into train-val and test set. Train-val consists of 786 bonafide images and 1572 manipulates images. Test comprises of 300 bonafide and 1085 manipulated image. The bonafide is different between Train-val and Test along with the kind of manipulations. Manipulations are done on faces and text fields. Below is the finer details of our dataset.

Category	Sub-Category	Count	Devices/Sources	Description and Purpose
Bonafide Cards	Generation	362	AMFD, Face London, WMCA, and Flickr datasets	Thirteen unique design styles in ten languages. Random but realistic personal info.
	Print	362	Evolis Primacy 2 printer	Printed on physical plastic cards.
	Capture	1086	iPhone 15 Pro, Huawei Mate 30, office scanner	Plastic cards were captured using three devices.
Forged Cards	Digital manipulation	786	InSwapper	Each face in captured IDs from train-val set is swapped with another face.
	Digital manipulation	786	Facedancer	Each face in captured IDs from train-val set is swapped with another face.
	Digital manipulation (Attack-3)	150	Facedancer	Faces in captured IDs from subset of test set are swapped.
	Digital manipulation	786	DiffSTE	Parts of personal info in captured IDs from train-val set were replaced by another text.
	Digital manipulation	786	Textdiffuser2	Parts of personal info in captured IDs from train-val set were replaced by another text.
	Digital manipulation (Attack-1)	786	Finetuned-Textdiffuser2	Parts of personal info in captured IDs from train-val set were replaced by another text.
	Digital manipulation (Attack-3)	149	Finetuned-Textdiffuser2	Parts of personal info in captured IDs from test set were replaced by another text.

Benchmarking

We benchmark our dataset with following image manipulation detection algorithms: TruFor, MMFusion, UniFD, and FatFormer. With the operational threshold set on validation set so that false positive rate is at 10%, leading to false negative rates close to 50% across the board on the test set for all the baselines, showing the difficulty of the FantasyID dataset. The results are shown in the table below.

Model	Protocol	ACC	AUC	F1	FPR	FNR	HTER
TruFor	all	65.9	93.5	80.7	4.9	62.0	33.4
TruFor	Attack-1	67.8	99.5	79.0	0.1	62.0	31.1
TruFor	Attack-2	53.5	55.8	47.6	34.7	62.0	48.3
TruFor	Attack-3	67.8	99.9	55.3	0.0	62.0	31.0
MMFusion	all	55.1	94.4	73.7	4.0	47.7	25.8
MMFusion	Attack-1	55.2	99.8	67.0	0.1	47.7	23.9
MMFusion	Attack-2	54.5	61.5	29.8	28.0	47.7	37.8
MMFusion	Attack-3	55.2	99.1	30.0	0.0	47.7	23.8
UniFD	all	50.0	52.0	7.7	8.3	92.7	50.5
UniFD	Attack-1	50.0	54.3	12,0	7.4	92.7	50.0
UniFD	Attack-2	50.0	48.4	53.3	7.3	92.7	50.0
UniFD	Attack-3	50.0	43.5	53.5	14.1	92.7	53.4
FatFormer	all	48.8	53.5	15.6	6.5	92.3	49.4
FatFormer	Attack-1	49.3	57.6	20.4	1.1	92.3	46.7
FatFormer	Attack-2	48.3	43.6	53.8	24.0	92.3	58.2
FatFormer	Attack-3	46.7	42.3	51.8	16.8	92.3	54.6

BibTeX


      @misc{korshunov2025fantasyiddatasetdetectingdigital,
      title={FantasyID: A dataset for detecting digital manipulations of ID-documents}, 
      author={Pavel Korshunov and Amir Mohammadi and Vidit Vidit and Christophe Ecabert and Sébastien Marcel},
      year={2025},
      eprint={2507.20808},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2507.20808}, 
      }

FantasyID: A dataset for detecting digital manipulations of ID-documents

News

Abstract

Dataset

Digital Manipulations

Example of digital manipulations in FantasyID. Faces are swapped and textual details are digtially altered.

Benchmarking

BibTeX