Multimodal Cognitive Learning for Media Forgery Detection: A Comprehensive Framework Combining Random Forest and Deep Ensemble Architectures (Xception, ResNeXt) across Image, Video, and Audio Modalities - S. Bhuvaneswari, Krithika K, Nithyasree I, Prashithaa Abhirami Balaji, Aadhithya R, Deexith P, Devesh R, A. Abirami (International Journal of Intelligent Systems and Applications in Engineering, 2024)
Deepfake content has become more prevalent in the age of quickly evolving technology, which has significantly undermined the reliability and integrity of digital media. An integrated multi-modal deepfake detection system is presented in this study as a response to the ubiquitous threat posed by altered photos, videos, and audio recordings. Our method, which makes use of advanced deep learning algorithms, provides a strong barrier against the spread of false information. The picture deepfake detection module examines visual data for telltale signs of manipulation using Convolutional Neural Networks (CNNs), Xception, and ResNeXT. This module successfully distinguishes between real and fake photos by carefully examining pixel-level attributes and contextual data. This capacity is expanded to include the world of movies by the video deepfake detection module. With the use of spatiotemporal CNNs (Xception & ResNeXT), it parses video frames to find minute discrepancies, making it possible to accurately identify deepfake films. Our multi-modal system is finished with the addition of deepfake audio detection. This module excels in differentiating between authentic and faked audio recordings using Mel spectrograms and Convolutional Neural Networks, adding to a thorough protection against audio deepfakes. Additionally, we provide a unifying framework that effectively unifies these three detection modules, boosting the system’s effectiveness and performance as a whole. We thoroughly assess our solution utilizing metrics such as AUC, ROC curve, F1 score, and accuracy, and we depict our model structures for in-depth comprehension. Our multi-modal deepfake detection technology acts as a crucial precaution in a time when false information is widely disseminated, enabling consumers to distinguish fact from fiction across numerous media types. This study highlights the importance of our integrated solution in maintaining the legitimacy of digital content in today’s information-driven world while also showcasing its technological capability.