Fighting Fake News: Image Splice Detection via Learned Self-Consistency

Minyoung Huh, Andrew Liu, Andrew Owens, Alexei A. Efros; The European Conference on Computer Vision (ECCV), 2018, pp. 101-117

Advances in photo editing and manipulation tools have made it significantly easier to create fake imagery, highlighting the need for better visual forensics algorithms. However, learning to detect manipulations from labelled training data is difficult due to the lack of good datasets of manipulated visual content. In this paper, we introduce a self-supervised method for learning to detect a visual manipulations using only unlabeled data. Given a large collection of real photographs with automatically recorded EXIF meta-data, we train a model to determine whether an image is self-consistent -- that is, whether its content could have been produced by a single imaging pipeline. We apply this self-supervised learning method to the task of localizing spliced image content. Our forensics model achieves state of the art results on many benchmarks, despite being trained without examples of actual manipulations, and without modeling specific detection cues. Beyond handcrafted benchmarks, we also show promising results spotting fakes on Reddit and The Onion, as well as detecting computer-generated splices.