Perbandingan Naïve Bayes, SVM, dan XGBoost dengan SMOTE untuk Analisis Sentimen Ulasan DANA dan OVO

Abstract

User reviews of the DANA and OVO digital wallet applications provide valuable insights into user satisfaction and service-related issues. However, the large volume of review data and the imbalanced distribution of sentiment classes pose significant challenges for automated sentiment analysis. This study aims to compare the performance of Naïve Bayes, Support Vector Machine (SVM), and XGBoost for sentiment classification of user reviews, while evaluating the effectiveness of the Synthetic Minority Over-sampling Technique (SMOTE) in handling class imbalance for each algorithm. The dataset consists of 4,987 user reviews classified into three sentiment categories: positive, negative, and neutral. The research process includes text preprocessing, TF-IDF feature extraction, model training, and performance evaluation using a confusion matrix and standard classification metrics. The experimental results indicate that SMOTE significantly improves the performance of the Naïve Bayes classifier, whereas it provides little to no performance improvement for SVM and XGBoost; therefore, the final SVM and XGBoost models are trained using the original dataset. Among the evaluated algorithms, SVM achieves the best overall performance with an accuracy of 85.14% and a macro F1-score of 77.60%, followed by XGBoost with an accuracy of 82.32%. Furthermore, application-specific analysis reveals that the model achieves higher accuracy on DANA reviews (87.55%) than on OVO reviews (82.13%), suggesting differences in linguistic characteristics and sentiment distributions across the two platforms. This study provides a systematic comparison of machine learning algorithms for Indonesian digital wallet sentiment analysis and demonstrates that selective application of data balancing techniques can improve classification performance without necessarily benefiting all algorithms.