Verifikasi Kesesuaian Materi Pembelajaran Menggunakan Model Bidirectional Encoder Representations from Transformers (BERT) dan Semantic Textual Similarity
-
Riani Saputri Abadi
*
Institut Teknologi PLN, Jakarta Barat,
Indonesia
(*) Corresponding Author
Abstract
The challenge in the education domain is ensuring that learning can be evaluated effectively and in a structured manner to improve and strengthen the quality of education standards in achieving optimal learning. In this study, an implementation was carried out to evaluate learning outcomes based on Natural Language Processing using BERT (IndoBERT) and Cosine similarity to assess the consistency and accuracy of learning materials with BAKP and RPS. IndoBERT is used to extract embedding vectors as contextual semantic representations from documents, and the similarity level is calculated using Cosine Similarity between the contents of BAKP and RPS to ensure the achievement of learning objectives. The research methodology consists of data collection, pre-processing, tokenization, and sentence embedding using IndoBERT, calculating the similarity level, and evaluating model performance. The results showed that implementing the IndoBERT model produced a good level of similarity with a value above the threshold, which was 0.50, with a Cosine Similarity result of 0.674 and a performance evaluation of 100%. This approach can provide the potential for automation of the higher education quality assurance process for academic evaluation based on BAKP and RPS so that learning materials are always relevant and updated with industry needs.
Downloads
References
UNESCO, Global Education Monitoring Report 2024: Pacific: Technology in education – A tool on whose terms. GEM Report UNESCO; Commonwealth of Learning, 2024. doi: 10.54676/FILL2531.
Permendikbud, “Permendikbud-Nomor-3-Tahun-2020,” 2020.
World Economic Forum, “Our Institutional Framework Leadership and Governance Policies, Terms and Codes.” Accessed: Feb. 27, 2025. [Online]. Available: https://www.weforum.org/stories/2024/08/global-youth-employment-future-jobs/
Kemendikbud, “Keputusan Menteri Pendidikan Dan Kebudayaan Republik Indonesia-Nomor-3_M_2021,” 2021, Accessed: Mar. 27, 2024. [Online]. Available: jdih.kemdikbud.go.id
M. Pratiwi, D. Yuliana Fitri, and A. Cesaria, “The Development of Inquiry-Based Teaching Materials for Basic Algebra Courses: Integration with Guided Note-Taking Learning Models,” MATHEMATICS TEACHING RESEARCH JOURNAL, vol. 192, no. 4, 2022.
M. H. Husni, A. Agung Ngurah Sedana Putra, and D. Purnama Dewata, “Development of Digitalization of Semester Learning Plans for Room Reservation Courses in the Room Division Study Program Lombok Tourism Polytechnic,” International Journal of Humanities Education and Social Sciences, Vol 3, No 5, 2024, doi.org/10.55227/ijhess.v3i5.844
Kementistekdikti RI, “Standar Nasional Pendidikan Tinggi_PERMENRISTEKDIKTI_Nomor_44_Tahun_2015_SNPT,” Dec. 2015.
V. Efimov, “Large Language Models: BERT — Bidirectional Encoder Representations from Transformer.” Accessed: Aug. 24, 2024. [Online]. Available: https://towardsdatascience.com/bert-3d1bf880386a
D. Y. Dengyun Zhu, Hailong Gai, Fucheng Wan, “Semantic Similarity Caculating based on BERT,” Journal of Electrical Systems, vol. 20, no. 2, pp. 73–79, Apr. 2024, doi: 10.52783/jes.1099.
D. Viji and S. Revathy, “A hybrid approach of Weighted Fine-Tuned BERT extraction with deep Siamese Bi – LSTM model for semantic text similarity identification,” Multimed Tools Appl, vol. 81, no. 5, pp. 6131–6157, Feb. 2022, doi: 10.1007/s11042-021-11771-6.
B. Whalley, D. France, J. Park, A. Mauchline, and K. Welsh, “Towards flexible personalized learning and the future educational system in the fourth industrial revolution in the wake of Covid-19,” High Educ Pedagog, vol. 6, no. 1, pp. 79–99, 2021, doi: 10.1080/23752696.2021.1883458.
N. L. Rane, “Education 4.0 and 5.0: integrating Artificial Intelligence (AI) for personalized and adaptive learning,” Journal of Artificial Intelligence and Robotics, Jun. 2024, doi: 10.61577/jaiar.2024.100006.
L. I. González‐pérez and M. S. Ramírez‐montoya, “Components of Education 4.0 in 21st Century Skills Frameworks: Systematic Review,” MDPI, Feb. 01, 2022,. doi: 10.3390/su14031493.
Sudarshan Joshi, Akshay Bachkar, Omkar Awaje, Rhutuj Bhoir, and Kimaya Urane, “Automated Answersheet Evaluation using BERT,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 10, no. 3, pp. 624–631, Jun. 2024, doi: 10.32628/cseit2410337.
N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” pp. 3982–3992, 2019, doi: 10.18653/v1/D19-1410.
V. Goel, D. Sahnan, V. V, G. Sharma, D. Dwivedi, and M. Mohania, “K-12BERT: BERT for K-12 education,” May 2022, [Online]. Available: http://arxiv.org/abs/2205.12335
Anugerah Simanjuntak et al., “Research and Analysis of IndoBERT Hyperparameter Tuning in Fake News Detection,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 13, no. 1, pp. 60–67, Feb. 2024, doi: 10.22146/jnteti.v13i1.8532.
S. Raaijmakers, Deep Learning for Natural Language Processing. Shelter Island: Manning Publications Co, 2022.
D. Sutrisno, A. Yuliana Dewi, I. Rosyadi, and M. Informatika, “Rancang Bangun Aplikasi Instrumen Perkuliahan Pada Fakultas Teknik dan Ilmu Komputer UMPP Berbasis WEB,” SURYA INFORMATIKA, vol. 13, no. 1, May 2023.
D. Firmansyah and Dede, “Teknik Pengambilan Sampel Umum dalam Metodologi Penelitian: Literature Review,” Jurnal Ilmiah Pendidikan Holistik (JIPH), vol. 1, no. 2, pp. 85–114, Aug. 2022, doi: 10.55927/jiph.v1i2.937.
A. Scarlatos, C. Brinton, and A. Lan, “Process-BERT: A Framework for Representation Learning on Educational Process Data,” Apr. 2022, [Online]. Available: http://arxiv.org/abs/2204.13607
K. Datchanamoorthy, A. Mala. G. S, and Padmavathi. B, “Text Mining: Clustering Using Bert And Probabilistic Topic Modeling,” Social Informatics Journal, vol. 2, no. 2, pp. 1–13, Dec. 2023, doi: 10.58898/sij.v2i2.01-13.
Denis. Rothman, Transformers for Natural Language Processing Rothman, Denis. Packt Publishing, 2021.
sbert.net, “Semantic Textual Similiarity.” Accessed: Aug. 24, 2024. [Online]. Available: https://sbert.net/examples/training/sts/README.html
E. Dave and A. Chowanda, “Indonesian personal financial entity extraction using indoBERT-BiGRU-CRF model,” J Big Data, vol. 11, no. 1, Dec. 2024, doi: 10.1186/s40537-024-00987-6.
F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP,” pp. 757–770, Nov. 2020, doi: 10.48550/arXiv.2011.00677.
D. Suhartono, M. R. N. Majiid, and R. Fredyan, “Towards automatic question generation using pre-trained model in academic field for Bahasa Indonesia,” Educ Inf Technol (Dordr), Nov. 2024, doi: 10.1007/s10639-024-12717-9.
S. Sofiana, Konsep-BERT pada Natural Language Processing. Eureka Media Aksara, 2024.
D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: state of the art, current trends and challenges,” Multimed Tools Appl, vol. 82, no. 3, pp. 3713–3744, Jan. 2023, doi: 10.1007/s11042-022-13428-4.
Y. T. Jin, J. B. You, S. Wakamiya, and H. Y. Kwon, “Analyzing user reactions using relevance between location information of tweets and news articles,” EPJ Data Sci, vol. 13, no. 1, Dec. 2024, doi: 10.1140/epjds/s13688-024-00465-2.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Verifikasi Kesesuaian Materi Pembelajaran Menggunakan Model Bidirectional Encoder Representations from Transformers (BERT) dan Semantic Textual Similarity
Pages: 2723-2734
Copyright (c) 2025 Riani Saputri Abadi
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
