Spotting Audio-Visual Inconsistencies

Spotting Audio-Visual Inconsistencies (SAVI) research project is a part of DARPA's Media Forensics (MediFor) program and focuses on techniques to identify audiovisual discrepancies present in a video that has been tampered with, including lip sync analysis, speaker inconsistency detection, scene inconsistency detection (room size and acoustics) and identifying frame drops or content insertions. Videos with audio are becoming a dominant means of documenting events and communicating messages around the world. Modifying or replacing the audio, or replacing the video, is often quite easy to do. These manipulations can change the message drastically, while being, at least in the case of audio, difficult for people to detect. These modifications, however, often leave discrepancies between the visual and audio channels that can be exposed by physical and semantic level analysis. The aim of SAVI project is to detect and characterize multiple types of inconsistencies involving different aspects of a video, and fuse these detections into a combined media integrity score. Within SAVI project, Idiap develops techniques to detect inconsistencies between lip motion and audio due to a manipulation of the audio or video tracks.
Biometric Person Recognition
Idiap Research Institute
Defense Advanced Research Projects Agency
May 19, 2016
May 18, 2020