Hafsa Ouajdi, Oussama Hadder, Mathieu Lagrange, Laurie M. Heller
Contact: mathieu dot lagrange at ls2n dot fr
Abstract
With the ever-rising quality of deep generative
models, it is increasingly important to be able to discern whether
the audio data at hand has been recorded or synthesized. While
the detection of fake speech signals has been studied extensively,
this is not the case for the detection of fake environmental audio.
We propose a simple and efficient pipeline for detecting fake
environmental sounds based on the CLAP audio embedding. We
evaluate this detector by using audio data from the 2023 DCASE
challenge task on Foley sound synthesis.
Our experiments show that fake sounds generated by 44 state-
of-the-art synthesizers can be detected with 98% accuracy on
average. We show that using an audio embedding learned on
environmental audio is beneficial over a standard VGGish one as
it provides a 10 % increase in detection performance. Informal
listening to Incorrect Negative examples demonstrates audible
features of fake sounds missed by the detector such as distortion
and implausible background noise.
Detection of Deepfake Environmental Audio
Our proposed system for audio fake detection is based on a multilayer perceptron (MLP) model. Instead of feeding
the audios directly to our detector, we use the embeddings generated by different systems. The figure below shows an
overview of the pipeline used in the experiments for the Deepfake detection, with a representation of the MLP’s
network architecture. The value of dim depends on the embedding method used.
