AI Insight
This study introduces DecepGPT, a new approach to multimodal deception detection that analyzes audiovisual cues and provides structured reasoning chains to explain its decisions. The researchers created T4-Deception, a multicultural dataset with 1695 samples from the "To Tell the Truth" television format across four countries, and developed two technical modules (SICS and DMC) to improve performance when training data is limited. The method achieves state-of-the-art results on multiple benchmarks and demonstrates better generalization across different cultural contexts compared to existing approaches.
Why it matters
This research addresses critical needs in forensic investigation and security applications where transparent, verifiable decision-making is essential. The multicultural dataset and improved cross-cultural transferability could enable more reliable deception detection systems that work across diverse populations, while the explainable reasoning chains provide the auditable evidence that investigators require in high-stakes scenarios.
Understand the Science
arXiv:2603.23916v4 Announce Type: replace-cross
Abstract: Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security. In these high-stakes settings, investigators need verifiable evidence connecting audiovisual cues to final decisions, along with reliable generalization across domains and cultural contexts. However, existing benchmarks provide only binary labels without intermediate reasoning cues. Datasets are also small with limited scenario coverage, leading to shortcut learning. We address these issues through three contributions. First, we construct reasoning datasets by augmenting existing benchmarks with structured cue-level descriptions and reasoning chains, enabling models to output auditable reports. Second, we release T4-Deception, a multicultural dataset based on the unified “To Tell the Truth” television format implemented across four countries. With 1695 samples, it is the largest non-laboratory deception detection dataset. Third, we propose two modules for robust learning under small-data conditions. Stabilized Individuality-Commonality Synergy (SICS) refines multimodal representations by combining learnable global priors with sample-adaptive residuals and applying polarity-aware recalibration. Distilled Modality Consistency (DMC) aligns modality-specific predictions with the fused multimodal predictions via knowledge distillation to prevent unimodal shortcut learning. Experiments on three established benchmarks and our novel dataset demonstrate that our method achieves state-of-the-art performance in both in-domain and cross-domain scenarios, while exhibiting superior transferability across diverse cultural contexts. The datasets and code are available at this link.