Enhancing Sentiment Analysis Effectiveness with LSTM Variants, and Stratified K-Fold on Imbalanced Dataset
-
Rifki Andriyanto
*
Universitas AMKOM Yogyakarta,
Indonesia
https://orcid.org/0009-0001-1305-5022
- Kusrini Kusrini Universitas Amikom Yogyakarta, Indonesia (*) Corresponding Author
Abstract
Sentiment analysis on hotel reviews often faces the challenge of class imbalance, where positive reviews significantly outnumber negative or neutral ones. This study aims to improve the effectiveness of sentiment analysis models on imbalanced hotel reviews by examining combinations of word embedding methods (FastText, Word2Vec, Doc2Vec) and model architectures (LSTM, BiLSTM, BiLSTM-Attention). Class imbalance is addressed using SMOTE, and model evaluation is conducted using Stratified K Fold cross-validation. Results show that Doc2Vec consistently outperforms FastText and Word2Vec as a word embedding method, especially when combined with the BiLSTM-Attention architecture. The use of SMOTE and Stratified K Fold also proves effective in improving model performance on imbalanced datasets. This study concludes that the selection of appropriate word embedding methods and model architectures, along with the implementation of class imbalance techniques, is crucial in developing effective and robust sentiment analysis models for hotel reviews.
Downloads
References
A. Al Hamoud, A. Hoenig, and K. Roy, “Sentence subjectivity analysis of a political and ideological debate dataset using LSTM and BiLSTM with attention and GRU models,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 10, pp. 7974–7987, Aug. 2022, doi: 10.1016/j.jksuci.2022.07.014.
J. Sangeetha and U. Kumaran, “A hybrid optimization algorithm using BiLSTM structure for sentiment analysis,” Measurement: Sensors, vol. 25, Aug. 2023, doi: 10.1016/j.measen.2022.100619.
Y. Pei, S. Chen, Z. Ke, W. Silamu, and Q. Guo, “AB-LaBSE: Uyghur Sentiment Analysis via the Pre-Training Model with BiLSTM,” Applied Sciences (Switzerland), vol. 12, no. 3, Aug. 2022, doi: 10.3390/app12031182.
B. A. Chandio, A. S. Imran, M. Bakhtyar, S. M. Daudpota, and J. Baber, “Attention-Based RU-BiLSTM Sentiment Analysis Model for Roman Urdu,” Applied Sciences (Switzerland), vol. 12, no. 7, Aug. 2022, doi: 10.3390/app12073641.
X. Li, Y. Lei, and S. Ji, “BERT- and BiLSTM-Based Sentiment Analysis of Online Chinese Buzzwords,” Future Internet, vol. 14, no. 11, Aug. 2022, doi: 10.3390/fi14110332.
Y. Yuan, W. Wang, G. Wen, Z. Zheng, and Z. Zhuang, “Sentiment Analysis of Chinese Product Reviews Based on Fusion of DUAL-Channel BiLSTM and Self-Attention,” Future Internet, vol. 15, no. 11, Aug. 2023, doi: 10.3390/fi15110364.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput, vol. 9, no. 8, pp. 1735–1780, 1997.
M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.
P. Wu, X. Li, C. Ling, S. Ding, and S. Shen, “Sentiment classification using attention mechanism and bidirectional long short-term memory network,” Appl Soft Comput, vol. 112, p. 107792, Aug. 2021, doi: 10.1016/J.ASOC.2021.107792.
D. Dablain, B. Krawczyk, and N. V Chawla, “DeepSMOTE: Fusing deep learning and SMOTE for imbalanced data,” IEEE Trans Neural Netw Learn Syst, vol. 34, no. 9, pp. 6390–6404, 2022.
T. R. Mahesh, O. Geman, M. Margala, M. Guduri, and others, “The stratified K-folds cross-validation and class-balancing methods with high-performance ensemble classifiers for breast cancer classification,” Healthcare Analytics, vol. 4, p. 100247, 2023.
D. Jurafsky, Speech & language processing. Pearson Education India, 2000.
P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching Word Vectors with Subword Information,” Aug. 2016.
F. Morin and Y. Bengio, “Hierarchical probabilistic neural network language model,” in International workshop on artificial intelligence and statistics, 2005, pp. 246–252.
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” Adv Neural Inf Process Syst, vol. 26, 2013.
K. I. Gunawan and J. Santoso, “Multilabel text classification menggunakan svm dan doc2vec classification pada dokumen berita bahasa indonesia,” Journal of Information System, Graphics, Hospitality and Technology, vol. 3, no. 01, pp. 29–38, 2021.
S. Khotijah, J. Tirtawangsa, and A. A. Suryani, “Using lstm for context based approach of sarcasm detection in twitter,” in Proceedings of the 11th international conference on advances in information technology, 2020, pp. 1–7.
A. Graves, A. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in 2013 IEEE international conference on acoustics, speech and signal processing, 2013, pp. 6645–6649.
J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural networks, vol. 61, pp. 85–117, 2015.
K. Wang, J. He, and L. Zhang, “Sequential weakly labeled multiactivity localization and recognition on wearable sensors using recurrent attention networks,” IEEE Trans Hum Mach Syst, vol. 51, no. 4, pp. 355–364, 2021.
X. Ma and E. H. Hovy, “End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF,” CoRR, vol. abs/1603.01354, 2016, [Online]. Available: http://arxiv.org/abs/1603.01354
N. C. Dang, M. N. Moreno-Garc’ia, and F. la Prieta, “Sentiment analysis based on deep learning: A comparative study,” Electronics (Basel), vol. 9, no. 3, p. 483, 2020.
X. Yao, “Attention-based BiLSTM neural networks for sentiment classification of short texts,” in Proc. Int. Conf. Inf. Sci. Cloud Comput, 2017, pp. 110–117.
S. Sukhbaatar, J. Weston, R. Fergus, and others, “End-to-end memory networks,” Adv Neural Inf Process Syst, vol. 28, 2015.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Enhancing Sentiment Analysis Effectiveness with LSTM Variants, and Stratified K-Fold on Imbalanced Dataset
Pages: 981-990
Copyright (c) 2024 Rifki Andriyanto
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
