An Intelligent Ensemble Techniques for Enhancing Deepfake Video Detection Performance
Keywords:
Deepfakes, Video Deepfake Detection, Convolutional Neural Networks(CNN), XceptionNet, EfficientNetb0, Ensemble.Abstract
In this research paper exploits the Ensembling approach for the detection of Deepfake videos. Fewer experiments employing Ensmebling techniques were conducted in the Deepfake domain. By training our model on many datasets and then ensembling their feature layers, we aimed to tackle the problem of over-fitting on a single dataset. Our study yielded superior outcomes compared to current techniques, leveraging the Face Forensics++ (FF++) dataset. This dataset comprises 1000 original video sequences subject to various manipulations. We have also employed effective pre-processing approaches to increase the accuracy of our present work. This includes Katna framework for key-frame extraction and MTCNN library for extracting facial features from the key-frames. MTCNN’s accuracy in face detection is one of its key features. It features a cascaded architecture composed of three neural networks that enhance the face identification results gradually. This method aids in reducing false positives and improving the precision of face detection. After training these weak learners we will use them along with a CNN model for final prediction. So the final model will contain these weak learners and a Convolutional Neural Network (CNN) model.
Downloads
References
“Video personalization using deep fake technology,” Maverick, 2023.
M. M. Antonio Bruno, Davide Moroni, “Efficient adaptive ensem- bling for image classification,” ResearchGate, 2022.
A. H. S. Md. Shohel Rana, Beddhu Murali, “Deepfake detection using machine learning algorithms,” in 10th International Congress on Advanced Applied Informatics (IIAI-AAI 2021), IEEE, 2021.
F. L. Irene Amerini, Gianmarco Baldini, “Image and video forensics,” ResearchGate, 2021.
J. C. D. K. N. A. V. A. B. S. C. G. C. R. Kartheek, “Detecting Deepfakes using deep learning,” in 2021 International Conference on Recent Trends on Electronics, Information, Communication Technology (RTEICT), IEEE, 2021.
D. G. E. J. Delp, “Deepfake video detection using recurrent neural networks,” in 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), IEEE, 2018.
A. A. L. Artem A Maksutov, Viacheslav O Morozov, “Methods of Deepfake detection based on machine learning,” in 2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), IEEE, 2020.
W. M. Wubet, “The Deepfake challenges and Deepfake video de- tection,” in International Journal of Innovative Technology and Exploring Engineering (IJITEE), IEEE, 2021.
A. A. Ahmed Khalifa, Nawal Zaher, “Convolutional neural network based on diverse gabor filters for Deepfake recognition,” IEEE, 2022.
P. C. Alakananda Mitra, Saraju, “A novel machine learning based method for Deepfake video detection in social media,” in 2020 IEEE International Symposium on Smart Electronic Systems (iSES) (Formerly iNiS), Research Gate, 2020.
Y. W. Siddharth Solaiyappana, “Machine learning based medical image Deepfake detection: A comparative study,” Elsevier, 2022.
B. K. S. A. F. W. H. Z. A. A. Hadid, “Hcit: Deepfake video detection using a hybrid model of cnn features and vision transformer,” in 2021 International Conference on Visual Communications and Image Processing (VCIP), IEEE, 2021.
R. W. Deng Pan, Lixian Sun, “Deepfake detection through deep learning,” in 2020 IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT), IEEE, 2020.
A. D. Sowmen Das, Selim Seferbekov, “Towards solving the deep- fake problem: An analysis on improving Deepfake detection using dynamic face augmentation,” in IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3776-3785, IEEE, 2021.
J. Z. K. C. G. S. X. Lin, “A heterogeneous feature ensemble learning based Deepfake detection method,” in ICC 2022 - IEEE International Conference on Communications, IEEE, 2022.
M. K. Serhat AtaS¸, ˙Ismail ˙Ilhan, “An efficient Deepfake video de-
tection approach with combination of efficientnet and xception models using deep learning,” in 2022 26th International Conference on Information Technology (IT), IEEE, 2022.
T. C. Joanna Baciak, Magdalena Zurawska, “Deepfake video detec- tion using the ensemble of neural networks,” ResearchGate, 2020.
A. H. S. Md. Shohel Rana, “Deepfakestack: A deep ensemble- based learning technique for Deepfake detection,” in 2020 7th IEEE International Conference, 2020.
A. M. V. Samuel Henrique Silva, Mazal Bethany, “Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models,” Elsevier, 2022.
P. C. . E. K. Alakananda Mitra, Saraju P. Mohanty, “A machine learning based approach for Deepfake detection in social media through key video frame extraction,” Springer, 2021.
S. M. Nicolò Bonettini, Edoardo Daniele Cannas, “Video face manipulation detection through ensemble of cnns,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2020.
J. John(B) and B. V. Sherif, “Multi-model Deepfake detection using deep and temporal features,” in ICIPCN 2022: Third International Conference on Image Processing and Capsule Networks, 2022.
J. F. Ruben Tolosana, Ruben Vera-Rodriguez, “Deepfakes and be- yond: A survey of face manipulation and fake detection,” in Infor- mation Fusion, 2020, arXiv, 2020.
Z. S. Zhengbo Luo, Sei-ichiro Kamata, “Transformer and node- compressed dnn based dual-path system for manipulated face de- tection,” in 2021 IEEE International Conference on Image Processing (ICIP), IEEE, 2021.
J. K. S.-K. J. S. L. D. J. J.-U. Hou, “Detection enhancement for various Deepfake types based on residual noise and manipulation traces,” IEEE, 2022.
K. K. TackHyun Jung, Sangwon Kim, “Deepvision: Deepfakes detec- tion using human eye blinking pattern,” ResearchGate, 2020.
A. K. A. P. A. N. M. G. S. Baudha, “Exploiting spatiotemporal inconsistencies to detect Deepfake videos in the wild,” in 2022 10th International Conference on Emerging Trends in Engineering and Technology - Signal and Information Processing (ICETET-SIP-22), IEEE, 2022.
D. D. H. C. G. C. H. Shi, “Multi branch Deepfake detection based on double attention mechanism,” in 2021 International Conference on Electronic Information Engineering and Computer Science (EIECS), IEEE, 2021.
J. F. Ruben Tolosana, Ruben Vera-Rodriguez, “Video face manipu- lation detection through ensemble of cnns,” ResearchGate, 2021.
Y. H. Z. L. M. Z. W. L. S. Li, “Df-vlad: Deepfake video detection based on feature aggregation,” in 2021 11th International Conference on Information Technology in Medicine and Education (ITME), IEEE, 2021.
J. D. Cai Yu, Peng Chen, “Focus by prior: Deepfake detection based on prior-attention,” in 2022 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2022.
P. Korshunov and S. Marcel, “Improving generalization of Deepfake detection with data farming and few-shot learning,” in IEEE Trans- actions on Biometrics, Behavior, and Identity Science, IEEE, 2022.
Alok, “Video key frame extraction with katna,” Medium, 2019.
H. H. Nguyen, F. Fang, J. Yamagishi, and I. Echizen, “Multi-task learning for detecting and segmenting manipulated facial images and videos,” in 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS). IEEE, 2019, pp. 1–8.
D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “Mesonet: a compact facial video forgery detection network,” in 2018 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 2018, pp. 1–7
J. Straub, “Using subject face brightness assessment to detect ‘deep fakes’(conference presentation),” in Real-Time Image Processing and Deep Learning 2019, vol. 10996. SPIE, 2019, p. 109960H
Y. Nirkin, L. Wolf, Y. Keller, and T. Hassner, “Deepfake detection based on discrepancies between faces and their context,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6111–6121, 2021
Y. Wang and A. Dantcheva, “A video is worth more than 1000 lies. comparing 3dcnn approaches for detecting deepfakes,” in 2020 15Th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). IEEE, 2020, pp. 515–519
H. H. Nguyen, J. Yamagishi, and I. Echizen, “Use of a capsule network to detect fake images and videos,” arXiv preprint arXiv:1910.12467, 2019
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
