Perbandingan XGBoost dan Random Forest Untuk Prediksi Pembatalan Pesanan Marketplace Berbasis SHAP
-
Dian Pertiwi
*
Universitas Dian Nuswantoro, Semarang,
Indonesia
- Usman Sudibyo Universitas Dian Nuswantoro, Semarang, Indonesia (*) Corresponding Author
Abstract
The high rate of order cancellations in marketplaces is a challenge that can impact operational efficiency, inventory management, and customer service quality. The ability to identify orders potentially canceled early is crucial so businesses can take more appropriate mitigation measures. The purpose of this study is to compare the performance of the Random Forest, and XGBoost algorithms in the marketplace order cancellations prediction process and to interpret the factors that influence predictions using SHAP (SHapley Additive exPlanations)-based explainable machine learning. The study used a marketplace transaction dataset that had undergone data cleaning, feature transformations, and class through the application of the Synthetic Minority Oversampling Technique (SMOTE) method. Next, the model performance is evaluated using the Accuracy, Precision, Recall, and F1-Score metrics. Four model scenarios were tested: Random Forest + SMOTE, Random Forest Tuned + SMOTE, XGBoost + SMOTE, and XGBoost Tuned + SMOTE. Based on the research results, it is indicated that the XGBoost Tuned + SMOTE model produces the best performance with accuracy values reaching 86.62%, precision reaching 51.43%, recall reaching 34.95%, and F1-Score reaching 41.62%. Meanwhile, the XGBoost + SMOTE model produced the highest recall of 37.09% with an F1-Score of 41.48%. Considering that the research objective is to detect potentially canceled orders in imbalance data, the XGBoost + SMOTE model was chosen as the best model because it produced the highest recall of 37,09% with a competitive F1-Score of 41,48%, making it more capable of detecting minority classes compared to other models. SHAP analysis showed that several transaction features contribute more dominantly to the prediction results produced by the model. These findings can be used as a basis for decision-making to reduce the risk of order cancellations and improve the effectiveness of marketplace operations.
Downloads
References
S. Wang, Y. Ye, B. Ning, J. H. Cheah, and X. J. Lim, “Why Do Some Consumers Still Prefer In-Store Shopping? An Exploration of Online Shopping Cart Abandonment Behavior,” Front. Psychol., vol. 12, no. January, pp. 1–14, 2022, doi: 10.3389/fpsyg.2021.829696.
T. Le Tan, K. Nguyen Chau Ngoc, H. L. T. Thanh, H. N. T. Thu, and U. V. T. Hoang, “Enhancing Repurchase Intention on Digital Platforms Based on Shopping Well-Being Through Shopping Value, Trust and Impulsive Buying,” SAGE Open, vol. 14, no. 3, pp. 1–25, 2024, doi: 10.1177/21582440241278454.
M. R. Nuryana, Muhammad Nur Aulia Rahman, and R. S. Hadikusuma, “Predicting Successful Timely Order Completion in E-Commerce Using XGboost and Catboost Models,” J. INSTEK (Informatika Sains dan Teknol., vol. 11, no. 1, pp. 69–77, 2026, doi: 10.24252/instek.v11i1.63315.
R. N. S. Hakim and A. Asmunin, “Optimasi Algoritma Random Forest dengan Teknik Boosting dalam Prediksi Churn Pelanggan di Industri Telekomunikasi,” J. Manag. Inform., vol. 13, no. 2, pp. 1–9, 2024, [Online]. Available: https://ejournal.unesa.ac.id/index.php/jurnal-manajemen-informatika/article/view/59841
M. R. Atsauri, H. Mawengkang, and S. Efendi, “Enhancing Unbalanced Data Classification with Cross-Validation and Extreme Gradient Boosting: A Comprehensive Analysis,” J. Informatics Telecommun. Eng., vol. 7, no. 1, pp. 30–42, 2023, doi: 10.31289/jite.v7i1.8690.
D. Elreedy, A. F. Atiya, and F. Kamalov, “A theoretical distribution analysis of synthetic minority oversampling technique (SMOTE) for imbalanced learning,” Mach. Learn., vol. 113, no. 7, pp. 4903–4923, 2024, doi: 10.1007/s10994-022-06296-4.
Y. Han, Z. Wei, and G. Huang, “An imbalance data quality monitoring based on SMOTE-XGBOOST supported by edge computing,” Sci. Rep., vol. 14, no. 1, pp. 1–13, 2024, doi: 10.1038/s41598-024-60600-x.
J. Starcke, J. Spadafora, J. Spadafora, P. Spadafora, and M. Toma, “The Effect of Data Leakage and Feature Selection on Machine Learning Performance for Early Parkinson’s Disease Detection,” Bioengineering, vol. 12, no. 8, 2025, doi: 10.3390/bioengineering12080845.
F. Koten, I. W. Sudiarsa, N. K. Trisnawati, M. M. Palo Pera, and M. A. Nidin, “Prediksi Pembatalan Pesanan E-Commerce Indonesia Mengunakan Random Forest dan SMOTE,” J. Manaj. Akunt. danIlmu Ekon., vol. 3, no. 1, pp. 8–14, 2026, doi: 10.70585/jumali.v3i1.192.
A. C. Dewi, A. Hermawan, and D. Avianto, “Classification of Customers’ Repeat Order Probability Using Decision Tree, Naïve Bayes and Random Forest,” J. Pilar Nusa Mandiri, vol. 20, no. 1, pp. 52–59, 2024, doi: 10.33480/pilar.v20i1.5243.
S. I. Mohammadpour, M. Khedmati, and M. J. H. Zada, “Classification of truck-involved crash severity: Dealing with missing, imbalanced, and high dimensional safety data,” PLoS One, vol. 18, no. 3, pp. 1–22, 2023, doi: 10.1371/journal.pone.0281901.
P. Netayawijit, W. Chansanam, and K. Sorn-In, “Interpretable Machine Learning Framework for Diabetes Prediction: Integrating SMOTE Balancing with SHAP Explainability for Clinical Decision Support,” Healthc., vol. 13, no. 20, pp. 1–26, 2025, doi: 10.3390/healthcare13202588.
M. K. Dahouda and I. Joe, “A Deep-Learned Embedding Technique for Categorical Features Encoding,” IEEE Access, vol. 9, pp. 114381–114391, 2021, doi: 10.1109/ACCESS.2021.3104357.
Q. Dong, “Leakage Prediction in Machine Learning Models When Using Data from Sports Wearable Sensors,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/5314671.
A. Ichwani, R. I. Kesuma, A. Setiawan, I. E. Wicaksono, and R. Hanifah, “Preventing Data Leakage in Classification via Integrated Machine Learning Pipelines: Preprocessing, Feature Transformation, and Hyperparameter Tuning,” J. Tek. Inform., vol. 7, no. 1, pp. 391–410, 2026, doi: 10.52436/1.jutif.2026.7.1.5490.
L. Chmielowski, M. Kucharzak, and R. Burduk, “Novel method of building train and test sets for evaluation of machine learning models related to software bugs assignment,” Sci. Rep., vol. 13, no. 1, pp. 1–11, 2023, doi: 10.1038/s41598-023-48617-0.
A. Demircioğlu, “Applying oversampling before cross-validation will lead to high bias in radiomics,” Sci. Rep., vol. 14, no. 1, pp. 1–11, 2024, doi: 10.1038/s41598-024-62585-z.
J. T. Hancock, T. M. Khoshgoftaar, and J. M. Johnson, “Evaluating classifier performance with highly imbalanced Big Data,” J. Big Data, vol. 10, no. 1, 2023, doi: 10.1186/s40537-023-00724-5.
S. Roumeliotis et al., “ROC curve analysis: a useful statistic multi-tool in the research of nephrology,” Int. Urol. Nephrol., vol. 56, no. 8, pp. 2651–2658, 2024, doi: 10.1007/s11255-024-04022-8.
A. Nambiar, S. Harikrishnaa, and S. Sharanprasath, “Model-agnostic explainable artificial intelligence tools for severity prediction and symptom analysis on Indian COVID-19 data,” Front. Artif. Intell., vol. 6, 2023, doi: 10.3389/frai.2023.1272506.
A. R. Salehi and M. Khedmati, “A cluster-based SMOTE both-sampling (CSBBoost) ensemble algorithm for classifying imbalanced data,” Sci. Rep., vol. 14, no. 1, pp. 1–17, 2024, doi: 10.1038/s41598-024-55598-1.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Perbandingan XGBoost dan Random Forest Untuk Prediksi Pembatalan Pesanan Marketplace Berbasis SHAP
Pages: 714-726
Copyright (c) 2026 Dian Pertiwi, Usman Sudibyo
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
