Sentiment Classification on Indonesian Game Sequels: A Comparative Analysis of SVM and Naive Bayes on Coffee Talk Franchise Reviews
- Nanda Yuris Riziq Universitas Dian Nuswantoro, Semarang, Indonesia
-
Edy Mulyanto
*
Universitas Dian Nuswantoro, Semarang,
Indonesia
(*) Corresponding Author
Abstract
User reviews on Steam are a critical source of feedback for game developers, yet manual sentiment analysis at scale is impractical. This study aims to compare Support Vector Machine (SVM), Multinomial Naive Bayes (MNB), and Complement Naive Bayes (CNB) for binary sentiment classification and to analyze sequel reception patterns through cross-game evaluation. Reviews were preprocessed with negation-aware stopword removal and WordNet lemmatization, then vectorized with TF-IDF unigram and bigram features. Four scenarios were evaluated: two within-game baselines, a cross-game generalization, and a combined evaluation. Class imbalance was handled at the model level via class weighting for SVM and the CNB variant. Macro-averaged F1-Score was the primary metric. SVM consistently outperformed both Naive Bayes variants, achieving macro-F1 of 0.81 within-game and 0.75 cross-game. MNB collapsed to majority-class prediction across all scenarios; in S2, all three models also failed on the minority class due to the small test partition (n=6). The cross-game result indicates that sentiment patterns transfer reasonably from the original game to its sequel, with the performance drop concentrated in the minority class. These findings offer practical guidance for Indonesian game developers monitoring sequel reception through automated sentiment analysis.
Downloads
References
ANTARA News, “Indonesia’s game industry levels up,” ANTARA News. Accessed: May 06, 2026. [Online]. Available: https://en.antaranews.com/news/321747/indonesias-game-industry-levels-up
D. W. Nugraha, “”Coffee Talk” dan Asa Industri Gim Lokal,” Kompas.id. Accessed: May 07, 2026. [Online]. Available: https://www.kompas.id/artikel/coffee-talk-dan-asa-industri-gim-lokal
Y. Pratomo and M. F. Nurhappy, “Toge Productions Umumkan Game ‘Coffee Talk Tokyo’, Rilis 2025,” Kompas.com. Accessed: May 07, 2026. [Online]. Available: https://tekno.kompas.com/read/2024/08/28/21000027/toge-productions-umumkan-game-coffee-talk-tokyo-rilis-2025
D. Lin, C.-P. Bezemer, Y. Zou, and A. E. Hassan, “An empirical study of game reviews on the Steam platform,” Empirical Software Engineering, vol. 24, no. 1, pp. 170–207, Feb. 2019, doi: 10.1007/s10664-018-9627-4.
B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques,” in Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP ’02, Morristown, NJ, USA: Association for Computational Linguistics, 2002, pp. 79–86. doi: 10.3115/1118693.1118704.
Y. HaCohen-Kerner, D. Miller, and Y. Yigal, “The influence of preprocessing on text classification using a bag-of-words representation,” PLOS ONE, vol. 15, no. 5, p. e0232525, May 2020, doi: 10.1371/journal.pone.0232525.
A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, and F. Herrera, Learning from Imbalanced Data Sets. Cham: Springer International Publishing, 2018. doi: 10.1007/978-3-319-98074-4.
M. Aulia and A. Hermawan, “Analisis Perbandingan Algoritma SVM, Naïve Bayes, dan Perceptron untuk Analisis Sentimen Ulasan Produk Tokopedia,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 7, no. 4, p. 1850, Oct. 2023, doi: 10.30865/mib.v7i4.6839.
I. Kurniawan, A. L. Hananto, S. Hilabi, A. Hananto, B. Priyatna, and A. Y. Rahman, “Perbandingan Algoritma Naive Bayes Dan SVM Dalam Sentimen Analisis Marketplace Pada Twitter,” JATISI (Jurnal Teknik Informatika dan Sistem Informasi), vol. 10, no. 1, pp. 731–740, Mar. 2023.
I. Guspian and M. H. Basri, “Comparison of Naive Bayes and SVM Algorithms in Sentiment Analysis for the Optimization of Hotel Operational Services in Central Bangka Regency,” Jutisi : Jurnal Ilmiah Teknik Informatika dan Sistem Informasi, vol. 14, no. 2, p. 1315, Aug. 2025, doi: 10.35889/jutisi.v14i2.3107.
L. B. Ilmawan and M. A. Mude, “Perbandingan Metode Klasifikasi Support Vector Machine dan Naïve Bayes untuk Analisis Sentimen pada Ulasan Tekstual di Google Play Store,” ILKOM Jurnal Ilmiah, vol. 12, no. 2, pp. 154–161, Aug. 2020, doi: 10.33096/ilkom.v12i2.597.154-161.
P. R. Sari, D. R. Indah, E. Rasywir, M. A. Firdaus, and G. Athalina, “Comparison of Naive Bayes and SVM Algorithms for Sentiment Analysis of PUBG Mobile on Google Play Store,” SISTEMASI, vol. 13, no. 6, p. 2767, Nov. 2024, doi: 10.32520/stmsi.v13i6.4814.
Heti Aprilianti, Khothibul Umam, and Maya Rini Handayani, “Comparative Study of SVM, KNN, and Naïve Bayes for Sentiment Analysis of Religious Application Reviews,” Journal of Applied Informatics and Computing, vol. 9, no. 3, pp. 920–927, Jun. 2025, doi: 10.30871/jaic.v9i3.9482.
J. W. Iskandar and Y. Nataliani, “Perbandingan Naïve Bayes, SVM, dan k-NN untuk Analisis Sentimen Gadget Berbasis Aspek,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 5, no. 6, pp. 1120–1126, Dec. 2021, doi: 10.29207/resti.v5i6.3588.
M. I. Fikri, T. S. Sabrila, and Y. Azhar, “Perbandingan Metode Naïve Bayes dan Support Vector Machine pada Analisis Sentimen Twitter,” SMATIKA JURNAL, vol. 10, no. 02, pp. 71–76, Dec. 2020, doi: 10.32664/smatika.v10i02.455.
P. D. Michailidis, “A Comparative Study of Sentiment Classification Models for Greek Reviews,” BDCC, vol. 8, no. 9, p. 107, Sep. 2024, doi: 10.3390/bdcc8090107.
A. Pangestu, Y. Tajul Arifin, and R. Ade Safitri, “Analisis Sentimen Review Publik Pengguna Game Online Pada Platform Steam Menggunakan Algoritma Naïve Bayes,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 7, no. 6, pp. 3106–3113, Jan. 2024, doi: 10.36040/jati.v7i6.8829.
R. Taqiuddin, F. A. Bachtiar, and W. Purnomo, “Opinion Spam Classification on Steam Review using Support Vector Machine with Lexicon-Based Features,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Nov. 2021, doi: 10.22219/kinetik.v6i4.1323.
M. Y. Febrianta, S. Widiyanesti, and S. R. Ramadhan, “Analisis Ulasan Indie Video Game Lokal pada Steam Menggunakan Analisis Sentimen dan Pemodelan Topik Berbasis Latent Dirichlet Allocation,” J. Animation and Games stud., vol. 7, no. 2, pp. 117–144, Oct. 2021, doi: 10.24821/jags.v7i2.5162.
J. Rennie, L. Shih, and J. Teevan, Tackling the Poor Assumptions of Naive Bayes Text Classifiers. 2003.
Azwan Triyadi, “Public Sentiment Analysis About Neuralink from Twitter Using Naïve Bayes: Multinomial, Gaussian and Complement,” The Indonesian Journal of Computer Science, vol. 13, no. 5, Oct. 2024, doi: 10.33022/ijcs.v13i5.4278.
K. Sparck Jones, “A Statistical Interpretation Of Term Specificity And Its Application In Retrieval,” Journal of Documentation, vol. 28, no. 1, pp. 11–21, Jan. 1972, doi: 10.1108/eb026526.
G. Salton and C. Buckley, “Term-weighting approaches in automatic text retrieval,” Information Processing & Management, vol. 24, no. 5, pp. 513–523, Jan. 1988, doi: 10.1016/0306-4573(88)90021-0.
C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge University Press, 2008. doi: 10.1017/CBO9780511809071.
S. Ranjan and S. Mishra, “Comparative Sentiment Analysis of App Reviews,” Jun. 2020.
D. Jurafsky and J. H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models. 2026.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.
C. Suhaeni and H.-S. Yong, “Enhancing Imbalanced Sentiment Analysis: A GPT-3-Based Sentence-by-Sentence Generation Approach,” Applied Sciences, vol. 14, no. 2, p. 622, Jan. 2024, doi: 10.3390/app14020622.
J. Al Mursyidy Fadhlurrahman, N. A. Herawati, H. R. Widya Aulya, I. Puspasari, and N. P. Utama, “Sentiment Analysis of Game Reviews on STEAM using BERT, BiLSTM, and CRF,” in 2023 International Conference on Electrical Engineering and Informatics (ICEEI), Bandung, Indonesia: IEEE, Oct. 2023, pp. 1–6. doi: 10.1109/ICEEI59426.2023.10346219.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Sentiment Classification on Indonesian Game Sequels: A Comparative Analysis of SVM and Naive Bayes on Coffee Talk Franchise Reviews
Pages: 151-160
Copyright (c) 2026 Nanda Yuris Riziq, Edy Mulyanto
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
