Comparative IndoBERT, Logistic Regression, SVM, Random Forest, and Naïve Bayes for TikTok Deepfake Misinformation
-
Tri Pidianto
*
Universitas Ngudi Waluyo, Semarang,
Indonesia
- Iwan Setiawan Wibisono Universitas Ngudi Waluyo, Semarang, Indonesia (*) Corresponding Author
Abstract
Generative AI has made deepfake content, especially face-swap videos and cloned voices, easier to produce and spread, and short-video platforms such as TikTok have become a major channel for this material. Most existing work still focuses on detecting the manipulated audio or video itself, leaving audience reactions in the comment section relatively unexplored. The problem this study addresses is that, without reading these comments, moderation systems have no way to know whether audiences are being misled by AI-generated content; the objective is to determine whether a fine-tuned IndoBERT model can flag such misinformation-leaning comments more reliably than conventional machine-learning classifiers. This paper takes a comparative approach to that gap, testing whether a fine-tuned IndoBERT model can flag potential misinformation in Indonesian-language TikTok comments more reliably than four TF-IDF-based classifiers: Logistic Regression, Random Forest, Multinomial Naive Bayes, and Linear SVM. Starting from 1,345 comments scraped across 15 TikTok videos, a multi-stage cleaning process left 1,071 usable comments, manually sorted into three labels: Misinformation, Skeptical, and Other. IndoBERT was fine-tuned with a weighted cross-entropy loss to offset class imbalance and evaluated through a train-validation-test split and Stratified 5-Fold Cross Validation. The fine-tuned model reached 82.41% accuracy and 81.58% F1-Macro, beating every machine-learning baseline by at least 13.84 points in F1-Macro, with five-fold results holding steady at an average F1-Macro of 80.17%. These results suggest IndoBERT is a stronger option than conventional machine learning for flagging misinformation-leaning comments on AI-generated content, offering a practical foundation for automated moderation on social platforms.
Downloads
References
F. N. Fa'izah and R. J. Kurniasari, "Deep fake crime on social media: A juridical analysis of the spread of hoaxes and the manipulation of public opinion in state security," Yurijaya: Jurnal Ilmiah Hukum, vol. 7, no. 3, pp. 522–550, 2025, doi: 10.51213/yurijaya.v7i3.238.
S. Alanazi and S. Asif, “Exploring deepfake technology: creation, consequences and countermeasures,” Human-Intelligent Systems Integration, vol. 6, no. 1, pp. 49–60, Dec. 2024, doi: 10.1007/s42454-024-00054-8.
Á. F. Gambín, A. Yazidi, A. Vasilakos, H. Haugerud, and Y. Djenouri, “Deepfakes: current and future trends,” Artif. Intell. Rev., vol. 57, no. 3, Art. no. 64, Mar. 2024, doi: 10.1007/s10462-023-10679-x.
A. H. Soudy et al., “Deepfake detection using convolutional vision transformers and convolutional neural networks,” Neural Comput. Appl., vol. 36, no. 31, pp. 19759–19775, Nov. 2024, doi: 10.1007/s00521-024-10181-7.
H. Liu, Y. Chen, A. Narayanan, A. Balachandran, P. J. Moreno, and L. Wang, “Can DeepFake Speech be Reliably Detected?,” arxiv, Oct. 2024, [Online]. Available: http://arxiv.org/abs/2410.06572
L. Li, T. Lu, X. Ma, M. Yuan, and D. Wan, “Voice Deepfake Detection Using the Self-Supervised Pre-Training Model HuBERT,” Applied Sciences, vol. 13, no. 14, p. 8488, 2023, doi: 10.3390/app13148488.
L. Luceri, T. V. Salkar, A. Balasubramanian, G. Pinto, C. Sun, and E. Ferrara, “Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem,” Oct. 2025, [Online]. Available: http://arxiv.org/abs/2505.10867
R. H. Nugroho, I. R. Kusumasari, V. Febrianto, M. A. Farhan N. H, and M. R. Mahardika, “Strategi Teknologi Artificial Intelligence (AI) dalam Pengambilan Keputusan Bisnis di Era Digital,” Jurnal Bisnis dan Komunikasi Digital, vol. 2, no. 2, p. 7, Dec. 2024, doi: 10.47134/jbkd.v2i2.3476.
R. Anggrainingsih, G. M. Hassan, and A. Datta, “Evaluating BERT-based language models for detecting misinformation,” Neural Comput. Appl., vol. 37, no. 16, pp. 9937–9968, Jun. 2025, doi: 10.1007/s00521-025-11101-z.
M. L. Jamil, S. Pais, J. Cordeiro, and G. Dias, “Detection of extreme sentiments on social networks with BERT,” Soc. Netw. Anal. Min., vol. 12, no. 1, Dec. 2022, doi: 10.1007/s13278-022-00882-z.
D. R. Faisal and R. Mahendra, “Two-Stage Classifier for COVID-19 Misinformation Detection Using BERT: a Study on Indonesian Tweets,” arxiv, Jun. 2022, [Online]. Available: http://arxiv.org/abs/2206.15359
F. Abbas and A. Taeihagh, “Unmasking deepfakes: A systematic review of deepfake detection and generation techniques using artificial intelligence,” Expert Syst. Appl., vol. 252, p. 124260, Oct. 2024, doi: 10.1016/j.eswa.2024.124260.
M. Masood, M. Nawaz, K. M. Malik, A. Javed, and A. Irtaza, “Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward,” Appl. Intell., vol. 53, pp. 3974–4026, 2023, doi: 10.1007/s10489-022-03766-z.
J. J. Bird and A. Lotfi, “Real-time Detection of AI-Generated Speech for DeepFake Voice Conversion,” Aug. 2023, [Online]. Available: http://arxiv.org/abs/2308.12734
Z. Xu, X. Wen, G. Zhong, X. Pan, and C. Zhang, “Public perception towards deepfake through topic modelling and sentiment analysis of social media data,” Soc. Netw. Anal. Min., vol. 15, no. 16, pp. 1–13, 2025, doi: 10.1007/s13278-025-01445-8.
M. Wack and S. Prochaska, “Making sense of AI-generated disinformation: How audience interpretations influence the impact of deepfakes in Kenya,” Soc. Media Soc., vol. 12, no. 3, Jul. 2026, doi: 10.1177/20563051261462092.
J. Wang, X. Wang, and A. Yu, “Tackling misinformation in mobile social networks: a BERT-LSTM approach for enhancing digital literacy,” Sci. Rep., vol. 15, no. 1, Art. no. 1118, Jan. 2025, doi: 10.1038/s41598-025-85308-4.
Z. Fang, H. Zhang, J. He, Z. Qi, and H. Zheng, “Semantic and contextual modeling for malicious comment detection with BERT-BiLSTM,” arXiv:2503.11084, Mar. 2025.
A. Bustamin, A. A. Prayogi, D. Siswanto, M. Rafrin, and A. Nurdin, "Text normalization for Indonesian slang words in sentiment analysis development," ICIC Express Letters, Part B: Applications, vol. 16, no. 2, pp. 121–129, Feb. 2025, doi: 10.24507/icicelb.16.02.121.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
D. Jurafsky and J. H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, with Language Models, 3rd ed. draft, 2025. [Online]. Available: https://web.stanford.edu/~jurafsky/slp3/
A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, Dive into Deep Learning. 2023.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Comparative IndoBERT, Logistic Regression, SVM, Random Forest, and Naïve Bayes for TikTok Deepfake Misinformation
Pages: 833-845
Copyright (c) 2026 Tri Pidianto, Iwan Setiawan Wibisono
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
