Comparison of Random Forest and XGBoost Methods Based on Hyperparameter Tuning for Classification of Customer Churn Rate of Telecommunication Providers
- Abdul Karim Universitas Labuhanbatu, Rantauprapat, Indonesia
-
Muhammad Hidayatullah
*
Universitas Teknologi Sumbawa, Sumbawa,
Indonesia
- Nora Dery Sofya Universitas Teknologi Sumbawa, Sumbawa, Indonesia
- Erwin Mardinata Universitas Teknologi Sumbawa, Sumbawa, Indonesia
- Shinta Esabella Universitas Teknologi Sumbawa, Sumbawa, Indonesia (*) Corresponding Author
Abstract
Customer churn represents one of the most critical challenges in the telecommunications industry, as the cost of acquiring new customers significantly outweighs the expense of retaining existing ones. High churn rates directly impact corporate revenue stability and market competitiveness, necessitating the development of precise predictive systems. This study presents a comprehensive comparative analysis of two prominent ensemble learning algorithms, Random Forest (RF) and Extreme Gradient Boosting (XGBoost), to establish a robust predictive framework for identifying potential churners using a large-scale Telco subscriber dataset. To ensure the reliability and scientific validity of the comparison, the research methodology incorporates the Synthetic Minority Over-sampling Technique (SMOTE) to rigorously address the inherent class imbalance within the dataset, ensuring that the minority churn class is adequately represented during the training phase to avoid model bias. Furthermore, a systematic hyperparameter tuning process was executed via GridSearchCV, exploring multiple combinations of estimators, depth, and learning rates to identify the optimal configurations for both algorithms. The experimental results reveal that while both models are highly effective, Random Forest slightly outperformed XGBoost, achieving an overall accuracy of 77.54% and a balanced F1-score of 0.616, compared to XGBoost’s accuracy of 76.54% and F1-score of 0.605. Notably, although both models demonstrated an identical recall rate of 67.64%, Random Forest exhibited superior precision (56.47% vs. 54.76%), which is vital for minimizing false positives and ensuring cost-effective retention campaigns. Feature importance analysis, conducted through Gini impurity and gain metrics, further identified tenure, total charges, and month-to-month contract types as the primary drivers of customer attrition. This study concludes that an optimized Random Forest model provides a more stable and accurate framework for telecommunication providers to proactively mitigate customer turnover. The findings offer valuable business intelligence, allowing stakeholders to transition from reactive measures to proactive, data-driven loyalty programs that enhance long-term business sustainability.
Downloads
References
P. Wachwanakijkul, S. Junsiritrakhoon, N. Kantanantha, G. Narayanamurthy, and P. Jarumaneeroj, “Data-driven approaches to predicting customer churn in a non-contractual car-sharing company,” Transp. Res. Interdiscip. Perspect., vol. 33, no. October 2024, pp. 1–20, 2025, doi: 10.1016/j.trip.2025.101600.
H. GhorbanTanhaei, P. Boozary, S. Sheykhan, M. Rabiee, F. Rahmani, and I. Hosseini, “Predictive analytics in customer behavior: Anticipating trends and preferences,” Results in Control and Optimization, vol. 17, no. September, pp. 1–17, 2024, doi: 10.1016/j.rico.2024.100462.
L. Theodorakopoulos, A. Theodoropoulou, and C. Klavdianos, “Big Data Analytics and AI for Consumer Behavior in Digital Marketing: Applications, Synthetic and Dark Data, and Future Directions,” Big Data and Cognitive Computing, vol. 10, no. 2, pp. 1–34, 2026, doi: 10.3390/bdcc10020046.
M. Imani, A. Beikmohammadi, and H. R. Arabnia, “Comprehensive Analysis of Random Forest and XGBoost Performance with SMOTE, ADASYN, and GNUS Under Varying Imbalance Levels,” Technologies (Basel)., vol. 13, no. 3, pp. 1–40, 2025, doi: 10.3390/technologies13030088.
H. N. Noura, T. Chu, Z. Allal, O. Salman, and K. Chahine, “A comparative study of ensemble methods and multi-output classifiers for predictive maintenance of hydraulic systems,” Results in Engineering, vol. 24, no. September, pp. 1–20, 2024, doi: 10.1016/j.rineng.2024.102900.
A. S. Assiri, “An Optimized Customer Churn Prediction Approach Based on Regularized Bidirectional Long Short-Term Memory Model,” Computers, Materials & Continua, vol. 86, no. 1, pp. 1–21, 2026, doi: 10.32604/cmc.2025.069826.
S. Shanmugam, E. Elavarasan, N. Madhavarao Seshadri, D. Ashokkumar, S. Senthilkumar, and T. Mohanavelu, “A Segmented Machine Learning Approach to Predicting and Mitigating Churn in the Gig Economy,” Journal of Theoretical and Applied Electronic Commerce Research , vol. 21, no. 3, pp. 1–25, 2026, doi: 10.3390/jtaer21030093.
S. Matharaarachchi, M. Domaratzki, and S. Muthukumarana, “Enhancing SMOTE for imbalanced data with abnormal minority instances,” Machine Learning with Applications, vol. 18, no. December 2023, pp. 1–31, 2024, doi: 10.1016/j.mlwa.2024.100597.
Y. Zhang, L. Deng, and B. Wei, “Imbalanced Data Classification Based on Improved Random-SMOTE and Feature Standard Deviation,” Mathematics, vol. 12, no. 11, pp. 1–17, 2024, doi: 10.3390/math12111709.
H. Jiang, Y. Xia, C. Yu, Z. Qu, and H. Li, “On the implications of artificial intelligence methods for feature engineering in reliability sector with computer knowledge graph,” Alexandria Engineering Journal, vol. 119, no. December 2024, pp. 587–597, 2025, doi: 10.1016/j.aej.2025.01.093.
F. Gao and M. Abisado, “Enhanced Feature Engineering Symmetry Model Based on Novel Dolphin Swarm Algorithm,” Symmetry (Basel)., vol. 17, no. 10, pp. 1–26, 2025, doi: 10.3390/sym17101736.
P. Ziolkowski, “Computational Complexity and Its Influence on Predictive Capabilities of Machine Learning Models for Concrete Mix Design,” Materials, vol. 16, no. 17, pp. 1–36, 2023, doi: 10.3390/ma16175956.
G. Croitoru, A. Capatina, N. V. Florea, F. Codignola, and D. Sokolic, “A cross-cultural analysis of perceived value and customer loyalty in restaurants,” European Research on Management and Business Economics, vol. 30, no. 3, pp. 1–16, 2024, doi: 10.1016/j.iedeen.2024.100265.
J. Tang, “Unlocking Retail Insights: Predictive Modeling and Customer Segmentation Through Data Analytics,” Journal of Theoretical and Applied Electronic Commerce Research, vol. 20, no. 2, pp. 1–20, 2025, doi: 10.3390/jtaer20020059.
M. Fang, H. Shi, H. Li, and T. Liu, “Application of Machine Learning for Productivity Prediction in Tight Gas Reservoirs,” Energies (Basel)., vol. 17, no. 8, pp. 1–27, 2024, doi: 10.3390/en17081916.
O. G. Al-Salih, D. Guangjian, W. J. Al-Mudhafar, and D. A. Wood, “Using extreme gradient boosting with Optuna hyperparameter tuning for efficient lost circulation prediction,” Energy Geoscience, vol. 7, no. 2, pp. 1–20, 2026, doi: 10.1016/j.engeos.2026.100540.
D. Sun, P. Zheng, J. Zhang, and L. Cheng, “Optimized Gradient Boosting Framework for Data-Driven Prediction of Concrete Compressive Strength,” Buildings, vol. 15, no. 20, pp. 1–20, 2025, doi: 10.3390/buildings15203761.
R. Suguna, J. Suriya Prakash, H. Aditya Pai, T. R. Mahesh, V. Vinoth Kumar, and T. E. Yimer, “Mitigating class imbalance in churn prediction with ensemble methods and SMOTE,” Sci. Rep., vol. 15, no. 1, pp. 1–20, 2025, doi: 10.1038/s41598-025-01031-0.
B. S. Priya, G. Chitra, and R. Ramalakshmi, “Performance comparison of sampling techniques with machine learning algorithms for churn prediction in telecommunication,” Franklin Open, vol. 13, no. August, pp. 1–14, 2025, doi: 10.1016/j.fraope.2025.100402.
M. Z. Abedin, C. Guotai, P. Hajek, and T. Zhang, “Combining weighted SMOTE with ensemble learning for the class-imbalanced prediction of small business credit risk,” Complex and Intelligent Systems, vol. 9, no. 4, pp. 3559–3579, 2023, doi: 10.1007/s40747-021-00614-4.
O. R. Olaniran, A. R. R. Alzahrani, N. M. S. Alharbi, and A. A. Alzahrani, “Random Generalized Additive Logistic Forest: A Novel Ensemble Method for Robust Binary Classification,” Mathematics, vol. 13, no. 7, pp. 1–25, 2025, doi: 10.3390/math13071214.
X. Zhang et al., “The XGBoost wind speed prediction model based on VMD-LSTM error correction,” Renew. Energy, vol. 267, no. June 2025, pp. 1–13, 2026, doi: 10.1016/j.renene.2026.125708.
S. R. Al-Taai, N. M. Azize, Z. A. Thoeny, H. Imran, L. F. A. Bernardo, and Z. Al-Khafaji, “XGBoost Prediction Model Optimized with Bayesian for the Compressive Strength of Eco-Friendly Concrete Containing Ground Granulated Blast Furnace Slag and Recycled Coarse Aggregate,” Applied Sciences (Switzerland), vol. 13, no. 15, pp. 1–23, 2023, doi: 10.3390/app13158889.
S. K. Wagh, A. A. Andhale, K. S. Wagh, J. R. Pansare, S. P. Ambadekar, and S. H. Gawande, “Customer churn prediction in telecom sector using machine learning techniques,” Results in Control and Optimization, vol. 14, no. March 2023, pp. 1–16, 2024, doi: 10.1016/j.rico.2023.100342.
M. Imani, M. Joudaki, A. Beikmohammadi, and H. R. Arabnia, “Customer Churn Prediction: A Systematic Review of Recent Advances, Trends, and Challenges in Machine Learning and Deep Learning,” Mach. Learn. Knowl. Extr., vol. 7, no. 3, pp. 1–38, 2025, doi: 10.3390/make7030105.
X. Wang and D. Hou, “Enhancing Keystroke Dynamics Authentication with Ensemble Learning and Data Resampling Techniques,” Electronics (Switzerland), vol. 13, no. 22, pp. 1–22, 2024, doi: 10.3390/electronics13224559.
I. Zerine et al., “Explainable churn prediction in telecom with tabular ML five model benchmark and SHAP analysis,” Discover Artificial Intelligence, vol. 6, no. 1, pp. 1–19, 2026, doi: 10.1007/s44163-026-00983-0.
A. Barsotti et al., “A Decade of Churn Prediction Techniques in the TelCo Domain: A Survey,” SN Comput. Sci., vol. 5, no. 4, pp. 1–15, 2024, doi: 10.1007/s42979-024-02722-7.
M. K. Banjanin, M. Stojčić, D. Danilović, Z. Ćurguz, M. Vasiljević, and G. Puzić, “Classification and Prediction of Sustainable Quality of Experience of Telecommunication Service Users Using Machine Learning Models,” Sustainability (Switzerland), vol. 14, no. 24, pp. 1–29, 2022, doi: 10.3390/su142417053.
P. B. Pires, B. M. Perestrelo, and J. D. Santos, “Measuring Customer Experience in E-Retail,” Adm. Sci., vol. 15, no. 11, pp. 1–33, 2025, doi: 10.3390/admsci15110434.
S. Mishra et al., “Agent-based modeling: Insights into consumer behavior, urban dynamics, grid management, and market interactions,” Energy Strategy Reviews, vol. 57, no. December 2024, pp. 1–19, 2025, doi: 10.1016/j.esr.2024.101613.
Q. Y. Huang, N. D. Dizon, N. Jeyakumar, and V. Jeyakumar, “A distributionally robust machine learning model of simultaneous classification and feature selection under data uncertainty: Theory, methods, and application to the identification of Alzheimer’s disease using handwriting,” EURO Journal on Computational Optimization, vol. 13, no. July, pp. 1–22, 2025, doi: 10.1016/j.ejco.2025.100111.
M. Shahabikargar, A. Beheshti, X. Zhang, J. Foo, and A. Jolfaei, “A comprehensive survey on customer churn analysis studies,” Journal of Information and Telecommunication, vol. 10, no. 1, pp. 24–70, 2025, doi: 10.1080/24751839.2025.2528440.
M. Madanchian, “The Role of Complex Systems in Predictive Analytics for E-Commerce Innovations in Business Management,” Systems, vol. 12, no. 10, pp. 1–20, 2024, doi: 10.3390/systems12100415.
H. Çelikten, “Evaluating machine learning (RF, XGBoost) and statistical model (MLR) for PM10 and air quality prediction: A case from Kars, Türkiye,” Atmos. Pollut. Res., vol. 17, no. 6, pp. 1–15, 2026, doi: 10.1016/j.apr.2026.102975.
J. B. Ruhland, I. Masoudian, and D. Heider, “Enhancing deep neural network training through learnable adaptive normalization,” Knowl. Based. Syst., vol. 326, no. July, pp. 1–9, 2025, doi: 10.1016/j.knosys.2025.113968.
P. Koukaras and C. Tjortjis, “Data Preprocessing and Feature Engineering for Data Mining: Techniques, Tools, and Best Practices,” AI (Switzerland), vol. 6, no. 10, pp. 1–40, 2025, doi: 10.3390/ai6100257.
Y. B. Wah et al., “Machine Learning and Synthetic Minority Oversampling Techniques for Imbalanced Data: Improving Machine Failure Prediction,” Computers, Materials and Continua, vol. 75, no. 3, pp. 4821–4841, 2023, doi: 10.32604/cmc.2023.034470.
S. V. Oprea and A. Bâra, “Customer-Centric Decision-Making with XAI and Counterfactual Explanations for Churn Mitigation,” Journal of Theoretical and Applied Electronic Commerce Research, vol. 20, no. 2, pp. 1–25, 2025, doi: 10.3390/jtaer20020129.
J. H. Joloudari, A. Marefat, M. A. Nematollahi, S. S. Oyelere, and S. Hussain, “Effective Class-Imbalance Learning Based on SMOTE and Convolutional Neural Networks,” Applied Sciences (Switzerland), vol. 13, no. 6, pp. 1–34, 2023, doi: 10.3390/app13064006.
K. Mokgwatjane and T. Paepae, “An explainable ensemble machine learning approach for multi-domain, multiclass sentiment analysis in Amazon product reviews,” Machine Learning with Applications, vol. 23, no. August 2025, pp. 1–17, 2026, doi: 10.1016/j.mlwa.2025.100825.
M. O. Ajinaja et al., A Comparative Evaluation of Probabilistic and Transformer-Based Topic Models Across Diverse and Multilingual Text Corpora, vol. 58, no. 1. 2026. doi: 10.1007/s11063-025-11820-3.
P. Boozary, S. Sheykhan, H. GhorbanTanhaei, and C. Magazzino, “Enhancing customer retention with machine learning: A comparative analysis of ensemble models for accurate churn prediction,” International Journal of Information Management Data Insights, vol. 5, no. 1, pp. 1–15, 2025, doi: 10.1016/j.jjimei.2025.100331.
Y. Xia, S. Jiang, L. Meng, and X. Ju, “XGBoost-B-GHM: An Ensemble Model with Feature Selection and GHM Loss Function Optimization for Credit Scoring,” Systems, vol. 12, no. 7, pp. 1–26, 2024, doi: 10.3390/systems12070254.
S. K. Kiangala and Z. Wang, “An effective adaptive customization framework for small manufacturing plants using extreme gradient boosting-XGBoost and random forest ensemble learning algorithms in an Industry 4.0 environment,” Machine Learning with Applications, vol. 4, no. December 2020, pp. 1–15, 2021, doi: 10.1016/j.mlwa.2021.100024.
C. Mueangphaen, W. Kusonkhum, K. Kuntiyawichai, T. Pannachet, R. Nuntasarn, and M. Boonpichetvong, “Comparative multi-algorithm AI framework for real-time carbon emission optimization in a medium-scale irrigation project in Thailand,” Environ. Impact Assess. Rev., vol. 118, no. November 2025, pp. 1–13, 2026, doi: 10.1016/j.eiar.2025.108276.
K. Sandunil, Z. Bennour, H. Ben Mahmud, and A. Giwelli, “Effects of tuning decision trees in random forest regression on predicting porosity of a hydrocarbon reservoir. A case study: volve oil field, north sea,” Energy Advances, vol. 3, no. 9, pp. 2335–2347, 2024, doi: 10.1039/d4ya00313f.
R. Agrawal et al., “Improving Predictive Performance in Telecom Churn Modeling with Hybrid SMOTE and GAN-Based Synthetic Data Generation,” International Journal of Computational Intelligence Systems, vol. 19, no. 1, pp. 1–23, 2026, doi: 10.1007/s44196-026-01204-3.
M. Risha and P. Liu, “Comparative machine learning facies prediction using ensemble boosting models and support vector machine versus unsupervised clustering,” Discover Geoscience, vol. 4, no. 1, pp. 1–22, 2026, doi: 10.1007/s44288-026-00398-5.
J. I. Iturbe-Araya and H. Rifà-Pous, “Hyperparameter Optimization and Evaluation Metrics for Unsupervised Anomaly-Based Cyberattack Detection in Imbalanced Smart Home Datasets,” Journal of Network and Systems Management, vol. 33, no. 4, pp. 1–37, 2025, doi: 10.1007/s10922-025-09973-6.
M. P. Pretel et al., “Machine Learning Models for Predicting Professional Disqualification in Peruvian Association Members,” Data (Basel)., vol. 11, no. 98, pp. 1–20, 2026, doi: 10.3390/data11050098.
N. Salaeh et al., “Resampling-driven machine learning models for enhanced high streamflow forecasting,” Water Cycle, vol. 7, no. July 2025, pp. 99–119, 2026, doi: 10.1016/j.watcyc.2025.07.001.
M. Martinović, K. Dokic, and D. Pudić, “Comparative Analysis of Machine Learning Models for Predicting Innovation Outcomes: An Applied AI Approach,” Applied Sciences (Switzerland), vol. 15, no. 7, pp. 1–44, 2025, doi: 10.3390/app15073636.
P. A. H. Pham and N. D. Hoang, “Metaheuristic optimization of extreme gradient boosting machine for enhanced prediction of lateral strength of reinforced concrete columns under cyclic loadings,” Results in Engineering, vol. 24, no. July, pp. 1–18, 2024, doi: 10.1016/j.rineng.2024.103125.
V. Chang, K. Hall, Q. A. Xu, F. O. Amao, M. A. Ganatra, and V. Benson, “Prediction of Customer Churn Behavior in the Telecommunication Industry Using Machine Learning Models,” Algorithms, vol. 17, no. 6, 2024, doi: 10.3390/a17060231.
S. S. Poudel, S. Pokharel, and M. Timilsina, “Explaining customer churn prediction in telecom industry using tabular machine learning models,” Machine Learning with Applications, vol. 17, no. March, pp. 1–9, 2024, doi: 10.1016/j.mlwa.2024.100567.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Comparison of Random Forest and XGBoost Methods Based on Hyperparameter Tuning for Classification of Customer Churn Rate of Telecommunication Providers
Pages: 459-472
Copyright (c) 2026 Abdul Karim, Muhammad Hidayatullah, Nora Dery Sofya, Erwin Mardinata, Shinta Esabella
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
