Improved Sentiment Classification Using Multilingual BERT with Enhanced Performance Evaluation for Hotel Guest Review Analysis
-
Yerik Afrianto Singgalen
*
Atma Jaya Catholic University of Indonesia, Jakarta,
Indonesia
(*) Corresponding Author
Abstract
Sentiment analysis in hotel guest reviews has become essential for evaluating customer satisfaction and service quality. This study improves sentiment classification accuracy by utilizing the Multilingual BERT model with an improved performance evaluation framework. Using the Knowledge Discovery in Databases (KDD) methodology, this research involves data selection, preprocessing, transformation, sentiment classification, and performance evaluation. A dataset of 715 hotel reviews from Qubika Boutique Hotel, sourced from Agoda, was used to assess the model's effectiveness. The classification results showed high accuracy in identifying positive sentiment, with 98% precision, 97% memory, and 98% F1 score, as observed in 432 correctly classified reviews. However, challenges were identified in the classification of neutral sentiment, which achieved a precision of 87% with 127 correctly classified cases, and negative sentiment, where the accuracy was 92%, with 104 correctly identified reviews. The overlap in confidence scores, especially in the range of 0.4-0.6 between neutral and negative sentiment, highlights the need for improved contextual embedding and hybrid modeling techniques. The sentiment distribution analysis revealed that 60-70% of reviews were positive, 20-30% neutral, and 10-15% indicated dissatisfaction, underscoring the need for targeted service improvement. These findings provide valuable insights for data-driven decision-making in hospitality management, enabling businesses to strengthen service power and address critical areas of concern. Future research should focus on refining model interpretability, expanding multilingual datasets, and integrating real-time sentiment analysis to improve classification performance. Strengthening these aspects will contribute to a more robust and scalable sentiment analysis framework, ensuring greater precision in capturing the guest experience and optimizing service strategies in the hospitality industry.
Downloads
References
O.A. George and CMQ Ramos, "Sentiment analysis applied to tourism: exploring tourist-generated content in the case of health tourism destinations," J. Spa International Health, vol. 7, no. 2, hlm. 139–161, 2024, doi: 10.1080/24721735.2024.2352979.
A. Ameur, S. Hamdi, and S. Ben Yahia, "An enhanced multilabel learning approach for the detection of Arabic aspect categories of hotel reviews," Computing. Intell., Vol. 40, No. 1, 2024, doi:10.1111/coin.12609.
Y. Wu, J. Wang, Y. Xia, Q. Li, and Y. Pan, "Sensing the distribution of hotel customers and the variation in their sentiment using online travel agency data: the case of a Shanghai star hotel," Ann. GIS, vol. 30, no. 3, hlm. 323–343, 2024, doi: 10.1080/19475683.2024.2335976.
J. Wang, "Hotel Room Experience Design Based on Virtual Reality Technology," A. Electricity. System., vol. 20, no. 1, hlm. 206–218, 2024, doi: 10.52783/jes.677.
Y. A. Singgalen, S. Y. Wahyuningtyas, Y. E. Widodo, M. N. A. Dasra, and R. W. Setiawan, "Discovery of Knowledge in Databases for Improving Hotel Service Quality Through a Data Mining Approach," J. Teor. Inf. Technol App., vol. 102, no. 24, pp. 9004–9020, 2024, [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85213991405&origin=inward
J. L. Nicolau, Z. Xiang, and D. Wang, "Daily online review sentiment and hotel performance," Int. J. Contemporary. Manag Hospital., vol. 36, no. 3, hlm. 790–811, 2024, doi: 10.1108/IJCHM-05-2022-0594.
M.G. Gîngioveanu Lupulescu, V.M. Dincă, SD Taranu, and B.A. Blănuță, "Data-Driven Insights from 10,000 Reviews: Driving Sustainability through Rapid Adaptation to Guest Feedback," Defend. , vol. 16, no. 7, 2024, doi: 10.3390/SU16072759.
JT Hsueh and S.H. Hsu, "Turning negative reviews into operational insights: The role of ABSS-GPT in informing hotel decisions," J. Decis. System., 2024, doi: 10.1080/12460125.2024.2428977.
M.J. Sánchez-Franco and S. Rey-Tienda, "The role of user-generated content in tourism decision-making: a case study of Andalusia, Spain," Manag. Results., Vol. 62, No. 7, hlm. 2292–2328, January 2024, Yogurt: 10.1108/MD-06-2023-0966.
S. Gupta and R. Jaiswal, "How We Can Improve Hospitality Excellence for Sustainable Development Using Machine Learning," J. Hosp. Tour. Emerged., 2024, doi: 10.1080/10963758.2024.2420267.
LC Cheng, H.Y. Huang, and Y.W. Huang, "A multi-task China aspect-based sentiment analysis framework for service improvement: a case study on BNB reviews," Electron. Comes. Res., 2024, doi: 10.1007/s10660-024-09871-0.
N.K. Boparai, H. Aggarwal, and R. Rani, "Analyzing review fuzzy semantics for multi-criteria recommendations," Data Knowledge. Eng., Vol. 152, 2024, doi: 10.1016/J. Donork.2024.102314.
F. Jeribi, U. Perumal, and M. H. Alhameed, "A Recommendation System for Sustainable Day and Night Cultural Tourism Using Recurrent Neural Networks Centered on Average Marked Errors for Riyadh Historic Sites," Defend. , vol. 16, no. 13, 2024, doi: 10.3390/SU16135566.
S. Bhowmik, R. Sadik, W. Akanda, and J. R. Pavel, "Sentiment analysis with hotel customer reviews using FNet," Cow. Electricity. Eng. Informatics, vol. 13, no. 2, no. 1298–1306, 2024, doi: 10.11591/eei.v13i2.6301.
Y. Andriyana et al., "Spatial Durbin Model with Expansion Using the Casetti Approach: A Case Study of Rainfall Prediction in Java, Indonesia," Mathematics, vol. 12, no. 15, 2024, doi: 10.3390/math12152304.
G.D. Mendonça, S.R. de M. Oliveira, OFlima, and PTV de Resende, "Intelligent algorithms applied to air transport delay prediction," Int. J. Phys. Distrib. Logistik. Manag.Vol. 54, No. 1 hlm. 61–91, January 2024, doi:10.1108/IJPDLM-10-2022-0328.
S. Khlamov, V. Savanevych, T. Trunova, Z. Deineko, O. Vovk, and R. Gerasimenko, "Automatic Data Mining of Reference Stars from Astronomical CCD Frames," CEUR Workshop Proceedings, vol. 3668. pp. 83–97, 2024. [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85191652348&origin=inward
K. Talebi, Z. Torabi, and N. Daneshpour, "An ensemble model based on CNN and LSTM for dropout prediction in MOOC," Application Expert System., vol. 235, 2024, doi: 10.1016/j.eswa.2023.121187.
D. Srivastava, R. Singh, C. Chakraborty, S. K. Maakar, A. Makkar, and D. Sinwar, "A framework for detecting cyberattacks with intrusion detection dataset classification," Microprocesses. Microsystems., vol. 105, 2024, doi: 10.1016/j.micpro.2023.104964.
M. N. Razali, S. A. Manaf, R. B. Hanapi, M. R. Salji, L. W. Chiat, and K. Nisar, "Improving the Classification of Minority Sentiment in Gastronomic Tourism: A Hybrid Sentiment Analysis Framework with Data Augmentation, Feature Engineering, and Business Intelligence," IEEE Access, vol. 12, no. December 2023, hlm. 49387–49407, 2024, doi: 10.1109/ACCESS.2024.3362730.
T. Mahmud, M. Ptaszynski, and F. Masui, "A Complete Study of Machine Learning and Deep Learning Methods for the Detection of Multilingual Cyberbullying in Bangla and Chittagonian Texts," Electron., Vol. 13, No. 9, 2024, doi: 10.3390/es13091677.
D. Gupta et al., "True and Fraudulent Hotel Reviews Based on Deep Learning," Defend. , vol. 16, no. 11, 2024, doi: 10.3390/SU16114514.
M. Ijaz, N. Anwar, M. Safran, S. Alfarhood, T. Sadad, and Imran, "Domain adaptive learning for multi-domain sentiment classification on big data," PLoS One, Vol. 19, no. 4 April 2024, Yogurt: 10.1371/Journal.Pone.0297028.
N. Habbat, H. Anoun, L. Hassouni, and H. Nouri, "Hotel Demand Forecasting through Booking Comments Using Sentiment Analysis and Topic Modeling Techniques," Advancements in Science, Technology, and Innovation. pp. 113–122, 2024. doi: 10.1007/978-3-031-46849-0_13.
N. Habbat and H. Nouri, "Unlocking travel narratives: a blend of deep learning composing ensemble and neural topic modeling for enhanced analysis of tourism commentary," Soc. Netw. Anal. Min., Vol. 14, No. 1, 2024, doi:10.1007/S13278-024-01256-3.
W. Jin et al., "Improving rural B&B management through machine learning and evolutionary games: A case study of rural revitalization in Yunnan, China," PLoS One, vol. 19, no. 3 March, 2024, doi: 10.1371/journal.pone.0294267.
M. Maryamah, G. Wilsen, C. T. Suhalim, R. Septiana, A. Fajar, and M. I. Solihin, "Hybrid Information Retrieval with Masked and Permuted Language Modeling (MPNet) and BM25L for Indonesian Drug Data Collection," in KST 2024 - 16th International Conference on Smart Knowledge and Technology2024, pp. 242–247. doi:10.1109/KST61284.2024.10499674.
A. Riyadi, M. Kovacs, U. Serdült, and V. Kryssanov, "IndoGovBERT: A Domain-Specific Language Model for Processing Indonesian Government SDG Documents," Big Data Cogn. Computing., Vol. 8, No. 11, 2024, doi: 10.3390/BDCC8110153.
H. Al-Jarrah, M. Al-Smadi, M. Hammad, and F. Shannaq, "Using Deep Learning Techniques to Detect Hate and Abusive Language in Arabic Tweets," Int. J. Intell. Eng. Syst., vol. 17, no. 5, hlm. 553–569, 2024, doi:10.22266/ijies2024.1031.43.
E. Raja, B. Soni, and S. K. Borgohain, "Harnessing heterogeneity: A multi-embedding ensemble approach to detecting fake news in Dravidian language," Computing. Electricity. Eng., vol. 120, 2024, doi: 10.1016/j.compeleceng.2024.109661.
M. E. Hassan, M. Hussain, I. Maab, U. Habib, M. A. Khan, and A. Masood, "Detection of Sarcasm in Urdu Tweets Using Deep Learning and a Transformer-Based Hybrid Approach," IEEE Access, vol. 12, hlm. 61542–61555, 2024, doi: 10.1109/ACCESS.2024.3393856.
K. Kim et al., "A Multifaceted Natural Language Processing Task-Based Evaluation of Two-Way Encoder Representations from Transformers Models for Bilingual Clinical Records (Korean and English): Algorithm Development and Validation," JMIR Med. Informatics, vol. 12, 2024, doi: 10.2196/52897.
A. Dhakshina Moorthy, D. Kavitha, R. Logeshwaran, N.V. Vishnu Kumar, and V. Karthick, "PSFAS: A Progressive Student Feedback Analysis System to Improve Teaching Learning with Intelligent Processing of Open Responses," J. Appl. Res. Tinggi. Emerged.2024, Doi: 10.1108/Jarhe-04-2024-0157.
D. Karpov and M. Burtsev, "Monolingual and Cross-Language Knowledge Transfer for Topic Classification," J. Math. Sci. (United States), vol. 285, no. 1, hlm. 36–48, 2023, doi:10.1007/s10958-024-07421-5.
KA Alshaikh, OA Almatrafi, and YB Abushark, "A BERT-Based Model for Aspect Based Sentiment Analysis to Analyze Arabic Open Survey Responses: A Case Study," IEEE Access, vol. 12, no. January, hlm. 2288–2302, 2024, doi: 10.1109/ACCESS.2023.3348342.
A.H. Aljammal, I. Al-Oqily, M. Obiedat, A. Qawasmeh, S. Taamneh, and F.I. Wedyan, "Detection of anomalous intrusion using machine learning-IG-R based on NSL-KDD datasets," Cow. Electricity. Eng. Informatics, vol. 13, no. 6, no. 4466–4474, 2024, doi: 10.11591/eei.v13i6.7308.
B. Al-Fuhaidi, Z. Farae, F. Al-Fahaidy, G. Nagi, A. Ghallab, and A. Alameri, "Anamal-Based Intrusion Detection System in Wireless Sensor Networks Using Machine Learning Algorithms," Application. Computing. Intell. Soft computing., vol. 2024, 2024, doi: 10.1155/2024/2625922.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Improved Sentiment Classification Using Multilingual BERT with Enhanced Performance Evaluation for Hotel Guest Review Analysis
Pages: 484-496
Copyright (c) 2025 Yerik Afrianto Singgalen
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
