User-Based Collaborative Filtering for Adaptive Assessment of Students' Critical Thinking: A Systematic Literature Review
- Elsha Rahma Listyanti Universitas Pendidikan Indonesia, West Java, Indonesia
-
Erna Piantari
*
Universitas Pendidikan Indonesia, West Java,
Indonesia
- H Eka Fitrajaya Rahman Universitas Pendidikan Indonesia, West Java, Indonesia (*) Corresponding Author
Abstract
Assessment is an essential part of the learning process. Standardized assessment often fails to identify students’ cognitive abilities, including critical thinking skills, on an individual basis. Recent research shows growing potential in adaptive assessment through machine learning, particularly user-based collaborative filtering, though its application to critical-thinking instruments remains limited. This study analyzes research trends in the application of an adaptive assessment system based on User-Based Collaborative Filtering (UBCF) as a context-specific question recommendation mechanism that maintains measurement equity. The method used was a Systematic Literature Review following PRISMA 2020, searching four databases (Google Scholar, Semantic Scholar, IEEE Xplore, and GARUDA), yielding 25 final articles from 2,573 records. Findings reveal three results. First, UBCF is reliable as an educational recommendation mechanism, but its use for question selection in assessment remains unexplored, though Elo-rating and hybrid CF-IRT combinations have begun touching this domain. Second, adaptive assessment is consistently more effective than conventional assessment in measuring cognitive ability, provided the mechanism maintains conditional fairness. Third, critical thinking measurement in vocational programming education still relies on uniform conventional instruments, without integrating recommendation systems. This study concludes that UBCF can fill this gap through context-based question adjustment using similarity of student response patterns, while maintaining cognitive equity. Further research is needed for empirical validation in vocational classrooms. This study contributes a research gap map and a context-based adaptivity framework for UBCF-based assessment systems.
Downloads
References
Adeoye, M. A., Prastikawati, E. F., & Riwayatiningsih, R. (2025). Beyond traditional testing: Exploring the efficacy of adaptive assessments in shaping future learners. Journal of Nonformal Education, 11(1), 153–168.
Agustian, E. R., Munir, & Nugroho, E. P. (2020). Sistem rekomendasi film menggunakan metode collaborative filtering dan k-nearest neighbors. JATIKOM: Jurnal Aplikasi dan Teori Ilmu Komputer. https://ejournal.upi.edu/index.php/JATIKOM
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. AERA.
Aprilianti, A. R., & Siswandari. (2024). Keefektifan model problem based learning (PBL) untuk meningkatkan kemampuan berpikir kritis siswa SMK. Jurnal Pendidikan dan Pembelajaran Indonesia (JPPI), 4(3), 1255–1266. https://doi.org/10.53299/jppi.v4i3.733
Arndt, D. M., Martins, R. M., & Hauck, J. C. R. (2025). Critical thinking assessment in K-12 computing education: A systematic mapping. Informatics in Education, 24(1), 1–44. https://doi.org/10.15388/infedu.2025.02
Bachtiar, Nirmala, S. D., Munawir, A., & Mujiono. (2024). AI-driven in education settings: Transforming educational landscapes for the future. Jurnal BUDIMAS, 6(3).
Contrino, M. F., Reyes-Millán, M., Vázquez-Villegas, P., & Membrillo-Hernández, J. (2024). Using an adaptive learning tool to improve student performance and satisfaction in online and face-to-face education. Smart Learning Environments, 11, Article 6. https://doi.org/10.1186/s40561-024-00292-y
Facione, P. A. (2015). Critical thinking: What it is and why it counts. Measured Reasons and The California Academic Press.
Fathurrahman, M. A., Laksitowening, K. A., & Suwawi, D. D. J. (2022). Penerapan metode collaborative filtering untuk personalized learning content pada learning management system (LMS). JURIKOM (Jurnal Riset Komputer), 9(2), 257–264. https://doi.org/10.30865/jurikom.v9i2.3887
Fink, A., Born, S., Spoden, C., & Frey, A. (2018). A continuous calibration strategy for computerized adaptive testing. Psychological Test and Assessment Modeling, 60(3), 327–346.
Herlambang, A. D., & Rachmadi, A. (2024). Computational thinking skills theorization in the vocational high school computer programming subject context. Elinvo (Electronics, Informatics, and Vocational Education), 9(1), 64–75.
Hidayat, S. Z., Via, Y. V., & Maulana, H. (2025). Sistem rekomendasi program studi jenjang sarjana untuk siswa SMK berbasis self-assessment RIASEC dengan metode hybrid neural network dan collaborative filtering. JATI (Jurnal Mahasiswa Teknik Informatika), 9(3).
Istiyono, E., Setiawan, R., Dwandaru, W. S. B., & Megawati, I. (2020). Developing of computerized adaptive testing to measure physics higher order thinking skills of senior high school students and its feasibility of use. European Journal of Educational Research, 9(1), 91–101.
Juniarni, C., Akhyar, Sodikin, M. A., Rafmana, H., & Bashori. (2024). Transforming teaching and learning with artificial intelligence (AI): Personalized education, adaptive learning, and intelligent tutoring systems. Education Studies and Teaching Journal (EDUTECH), 1(3), 368–386.
Khusna, A. N., Delasano, K. P., & Saputra, D. C. E. (2021). Penerapan user-based collaborative filtering algorithm studi kasus sistem rekomendasi untuk menentukan gadget shield. MATRIK: Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer, 20(2), 293–304. https://doi.org/10.30812/matrik.v20i2.1124
Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. EBSE-2007-01.
Kousar, M., & Afzal, M. (2021). Role of critical thinking in programming: A systematic review. Journal of New Approaches in Educational Research, 10(1), 1–14.
Kurniawan, N. A., Hidayah, N., & Rahman, D. H. (2021). Analisis kemampuan berpikir kritis siswa SMK. Jurnal Pendidikan: Teori, Penelitian, dan Pengembangan, 6(3), 334–338. http://journal.um.ac.id/index.php/jptpp/
Li, J. (2022). A recommendation model for college English digital teaching resources using collaborative filtering and few-shot learning technology. Computational Intelligence and Neuroscience, 2022, Article 1233057. https://doi.org/10.1155/2022/1233057
Melisa, R., Ashadi, A., Triastuti, A., Hidayati, S., Ero, P. E. L., Marlini, C., Zefrin, Z., & Fuad, Z. A. (2025). Critical thinking in the age of AI: A systematic review of AI's effects on higher education. Educational Process: International Journal, 14, e2025031. https://doi.org/10.22521/edupij.2025.14.31
Milenia, K. (2022). Perancangan instrumen asesmen adaptif pada gaya belajar Kolb untuk meningkatkan kemampuan computational thinking siswa [Undergraduate thesis, Universitas Pendidikan Indonesia].
Mislevy, R. J., Haertel, G., Cheng, B. H., Ructtinger, L., DeBarger, A., Murray, E., Rose, D., Gravel, J., Colker, A. M., Rutstein, D., & Vendlinski, T. (2013). A 'conditional' sense of fairness in assessment. Educational Research and Evaluation, 19(2–3), 121–140. https://doi.org/10.1080/13803611.2013.767614
Morris, R., Perry, T., & Wardle, L. (2021). Formative assessment and feedback for learning in higher education: A systematic review. Review of Education, 9(3). https://doi.org/10.1002/rev3.3292
Muhammad, F. D. (2024). Rancang bangun sistem rekomendasi solusi pada aktivitas pembimbingan penyelidikan problem-based learning menggunakan collaborative filtering di mata pelajaran pemrograman berorientasi objek [Undergraduate thesis, Universitas Pendidikan Indonesia].
Nadya, R., Amalia, I., & Rachman, I. F. (2025). Analisis potensi dan tantangan dalam penggunaan AI di bidang pendidikan. Semantik: Jurnal Riset Ilmu Pendidikan, Bahasa dan Budaya, 3(2), 295–309. https://doi.org/10.61132/semantik.v3i2.1705
OECD. (2023). PISA 2022 assessment and analytical framework. OECD Publishing. https://doi.org/10.1787/dfe0bf9c-en
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjártsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., ... Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
Popham, W. J. (2022). Classroom assessment: What teachers need to know (8th ed.). Pearson.
Raj, N. S., & Renumol, V. G. (2021). A systematic literature review on adaptive content recommenders in personalized learning environments from 2015 to 2020. Journal of Computers in Education, 9(1), 113–148. https://doi.org/10.1007/s40692-021-00199-4
Ramgovind, P., & Pramjeeth, S. (2023). Students' perceptions of computerised adaptive testing in higher education. The Independent Journal of Teaching and Learning, 18(2).
Raudhatunnur, & Shaleh. (2025). Penyusunan instrumen berbasis computerized adaptive test (CAT) untuk mengukur perkembangan kognitif peserta didik pada mata pelajaran PPKn. Jurnal Ilmu Pendidikan dan Pembelajaran, 4(1), 73–82. https://doi.org/10.58706/jipp
Ricci, F., Rokach, L., & Shapira, B. (2015). Introduction to recommender systems handbook. In Recommender systems handbook (pp. 1–35). Springer US. https://doi.org/10.1007/978-0-387-85820-3_1
Rokhmah, T. I., Darsono, Sucipto, Restian, A., & Anshory AM, I. (2026). Developing critical thinking in digital learning environments: Challenges and opportunities. Nusantara: Jurnal Pendidikan Indonesia, 6(1).
Saifudin, I., & Widiyaningtyas, T. (2024). Systematic literature review on recommender system: Approach, problem, evaluation techniques, datasets. IEEE Access, 12. https://doi.org/10.1109/ACCESS.2024.3359274
Saputra, O. A., Nurfajriani, & Sudrajat, A. (2024). Analisis instrumen penilaian hasil belajar kimia untuk mengukur high order thinking skill (HOTS) berdasarkan permodelan Rasch. Didaktika: Jurnal Kependidikan, 13(2).
Sari, A. P., Munir, & Wahyudin. (2026). Systematic literature review: Integrasi adaptive learning environment dan differentiated instruction dalam meningkatkan computational thinking dan digital literacy siswa. Jurnal Basicedu, 10(1), 38–51. https://doi.org/10.31004/basicedu.v10i1.10815
Shemshack, A., Kinshuk, & Spector, J. M. (2021). A comprehensive analysis of personalized learning components. Journal of Computers in Education, 8(4), 485–503. https://doi.org/10.1007/s40692-021-00188-7
Silva, W., Spalenza, M., Bourguet, J. R., & de Oliveira, E. (2020). Recommendation filtering à la carte for intelligent tutoring systems. In L. Boratto, S. Faralli, M. Marras, & G. Stilo (Eds.), Bias and social aspects in search and recommendation (Vol. 1245, pp. 58–65). Springer. https://doi.org/10.1007/978-3-030-52485-2_6
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.
Vesin, B., Mangaroska, K., Akhuseyinoglu, K., & Giannakos, M. (2022). Adaptive assessment and content recommendation in online programming courses: On the use of Elo-rating. ACM Transactions on Computing Education, 22(3), 33:1–33:27. https://doi.org/10.1145/3511886
Wainer, H. (1990). Computerized adaptive testing: A primer. Lawrence Erlbaum Associates.
Xu, G., Jia, G., Shi, L., & Zhang, Z. (2021). Personalized course recommendation system fusing with knowledge graph and collaborative filtering. Computational Intelligence and Neuroscience, 2021, 1–8. https://doi.org/10.1155/2021/9590502
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel User-Based Collaborative Filtering for Adaptive Assessment of Students' Critical Thinking: A Systematic Literature Review
Copyright (c) 2026 Elsha Rahma Listyanti, Erna Piantari, H Eka Fitrajaya Rahman
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
