Analisis Komparatif Konfigurasi Multilayer Perceptron pada Classifier Head RoBERTa untuk Klasifikasi Ujaran Kebencian
- Ikhwan Habibi Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
-
Surya Agustian
*
Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru,
Indonesia
- Jasril Jasril Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
- Muhammad Affandes Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia (*) Corresponding Author
Abstract
The widespread dissemination of hate speech and offensive language on social media has increased the demand for accurate automated text classification systems. Although RoBERTaForSequenceClassification has been widely used for various for text classification task, the effect of its default classifier head configuration on classification performance has not yet been systematically evaluated. As the main contribution, this study conducts a controlled evaluation of 32 multilayer perceptron (MLP)-based classifier head configurations, varying the number of hidden layers, activation functions, and dropout rates, against the default classifier head on the English HASOC 2021 dataset for two subtasks: binary and multiclass classification. Each configuration was evaluated using Stratified 5-Fold Cross-Validation with Macro-F1 as the evaluation metric, after which the best-performing configuration was further evaluated on an independent test set. For the binary task, the best configuration achieved a test Macro-F1 of 80.90%, about 0.3 percentage points higher than the baseline's 80.59%. For the multiclass task, the configuration with the highest validation performance instead achieved a test Macro-F1 of 65.68%, about 0.4 percentage points lower than the baseline's 66.11%, showing that an advantage observed during cross-validation does not always hold on the test set. Further analysis revealed that excessively deep hidden layers combined with aggressive dimensional compression can sharply degrade performance on the multiclass task. These findings indicate that the effect of classifier head configuration is small and task-dependent, so systematic evaluation remains necessary before adopting a given configuration in place of the default classifier head when fine-tuning RoBERTa-based models.
Downloads
References
Agustian, S., Saputra, R., & Fadhilah, A. (2021). Feature Selection with Pretrained-BERT for Hate Speech and Offensive Content Identification in English and Hindi Languages. Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, CEUR Workshop Proceedings, 3159, 508–516. https://ceur-ws.org/Vol-3159/T1-50.pdf
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., Deng, D., & Lindauer, M. (2023). Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. WIREs Data Mining and Knowledge Discovery, 13(2), e1484. https://doi.org/10.1002/widm.1484
Bölücü, N., & Canbay, P. (2021). Hate Speech and Offensive Content Identification with Graph Convolutional Networks. Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, CEUR Workshop Proceedings, 3159, 44–51. https://ceur-ws.org/Vol-3159/T1-4.pdf
Cinelli, M., Pelicon, A., Mozetič, I., Quattrociocchi, W., Novak, P. K., & Zollo, F. (2021). Dynamics of online hate and misinformation. Scientific Reports, 11(1), 22083. https://doi.org/10.1038/s41598-021-01487-w
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022a). Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503, 92–108. https://doi.org/10.1016/j.neucom.2022.06.111
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022b). Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503, 92–108. https://doi.org/https://doi.org/10.1016/j.neucom.2022.06.111
ElSherief, M., Ziems, C., Muchlinski, D., Anupindi, V., Seybolt, J., De Choudhury, M., & Yang, D. (2021). Latent Hatred: A Benchmark for Understanding Implicit Hate Speech. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 345–363. https://doi.org/10.18653/v1/2021.emnlp-main.29
Glazkova, A. V, Kadantsev, M., & Glazkov, M. (2021). Fine-tuning of Pre-trained Transformers for Hate Offensive and Profane Content Detection in English and Marathi. Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, CEUR Workshop Proceedings, 3159, 52–62. https://ceur-ws.org/Vol-3159/T1-5.pdf
Kenneweg, P., Schröder, S., & Hammer, B. (2022). Neural Architecture Search for Sentence Classification with BERT. ESANN 2022 Proceedings, 417–422. https://doi.org/10.14428/esann/2022.ES2022-45
Kui, Y. (2021). Detect Hate and Offensive Content in English and Indo-Aryan Languages based on Transformer. Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, CEUR Workshop Proceedings, 3159, 191–199. https://ceur-ws.org/Vol-3159/T1-19.pdf
Lee, M. (2023). Mathematical Analysis and Performance Evaluation of the GELU Activation Function in Deep Learning. Journal of Mathematics, 2023, 1–13. https://doi.org/10.1155/2023/4229924
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv E-Prints, arXiv:1907.11692. http://arxiv.org/abs/1907.11692
Masud, S., Khan, M. A., Goyal, V., Akhtar, M. S., & Chakraborty, T. (2024). Probing Critical Learning Dynamics of PLMs for Hate Speech Detection. Findings of the Association for Computational Linguistics: EACL 2024, 826–845. https://doi.org/10.18653/v1/2024.findings-eacl.55
Mitra, A., & Sankhala, P. (2021). Multilingual Hate Speech and Offensive Content Detection using Modified Cross-entropy Loss. Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, Gandhinagar, India, December 13-17, 2021, CEUR Workshop Proceedings, 3159, 349–356. https://ceur-ws.org/Vol-3159/T1-34.pdf
Mnassri, K., Farahbakhsh, R., Chalehchaleh, R., Rajapaksha, P., Jafari, A. R., Li, G., & Crespi, N. (2024). A survey on multi-lingual offensive language detection. PeerJ Computer Science, 10, e1934. https://doi.org/10.7717/peerj-cs.1934
Modha, S., Mandl, T., Shahi, G. K., Madhu, H., Satapara, S., Ranasinghe, T., & Zampieri, M. (2021). Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages and Conversational Hate Speech. Forum for Information Retrieval Evaluation, 1–3. https://doi.org/10.1145/3503162.3503176
Ocampo, N., Sviridova, E., Cabrio, E., & Villata, S. (2023). An In-depth Analysis of Implicit and Subtle Hate Speech Messages. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 1997–2013. https://doi.org/10.18653/v1/2023.eacl-main.147
Pérez, J. M., Luque, F. M., Zayat, D., Kondratzky, M., Moro, A., Serrati, P. S., Zajac, J., Miguel, P., Debandi, N., Gravano, A., & Cotik, V. (2023). Assessing the Impact of Contextual Information in Hate Speech Detection. IEEE Access, 11, 30575–30590. https://doi.org/10.1109/ACCESS.2023.3258973
Roychowdhury, S., & Gupta, V. (2023). Data-Efficient Methods For Improving Hate Speech Detection. Findings of the Association for Computational Linguistics: EACL 2023, 125–132. https://doi.org/10.18653/v1/2023.findings-eacl.9
Salehin, I., & Kang, D.-K. (2023). A Review on Dropout Regularization Approaches for Deep Neural Networks within the Scholarly Domain. Electronics, 12(14), 3106. https://doi.org/10.3390/electronics12143106
Subramanian, M., Easwaramoorthy Sathiskumar, V., Deepalakshmi, G., Cho, J., & Manikandan, G. (2023). A survey on hate speech detection and sentiment analysis using machine learning and deep learning models. Alexandria Engineering Journal, 80, 110–121. https://doi.org/10.1016/j.aej.2023.08.038
Szeghalmy, S., & Fazekas, A. (2023). A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning. Sensors, 23(4), 2333. https://doi.org/10.3390/s23042333
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł. ukasz, & Polosukhin, I. (2017). Attention is All you Need. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
Yadav, A., Khan, F. A., & Singh, V. (2024). A Multi-Architecture Approach for Offensive Language Identification Combining Classical Natural Language Processing and BERT-Variant Models. Applied Sciences, 14(23), 11206. https://doi.org/10.3390/app142311206
Yin, W., & Zubiaga, A. (2021). Towards generalisable hate speech detection: a review on obstacles and solutions. PeerJ Computer Science, 7, e598. https://doi.org/10.7717/peerj-cs.598
Zaghouani, W., Mubarak, H., & Biswas, Md. R. (2024). So Hateful! Building a Multi-Label Hate Speech Annotated Arabic Dataset. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 15044–15055. https://aclanthology.org/2024.lrec-main.1308/
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Analisis Komparatif Konfigurasi Multilayer Perceptron pada Classifier Head RoBERTa untuk Klasifikasi Ujaran Kebencian
Pages: 885-898
Copyright (c) 2026 Ikhwan Habibi, Surya Agustian, Jasril Jasril, Muhammad Affandes
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
