User Experience Analysis of a Multimodal Digital Application Integrating Multiple Intelligences for Young Learner
-
Rini Juliana Sipahutar
*
Universitas Negeri Medan, Medan,
Indonesia
- Natalia Silalahi Universitas Negeri Medan, Medan, Indonesia
- Nina Afria Damayanti Universitas Negeri Medan, Medan, Indonesia (*) Corresponding Author
Abstract
Despite the increasing adoption of multimodal educational applications for young learners, empirical research that explicitly examines young learner’s user experience from the perspective of cognitive diversity remains limited. Many existing studies emphasize learning outcomes or technical usability, while insufficient attention is given to affective engagement, interaction behavior, and experiential quality during young learner’s interaction with multimodal systems. This gap highlights the need for a structured analysis of how multimodal interaction grounded in the Multiple Intelligences (MI) framework shapes young learner’s user experience. This study examines young learner’s user experience while interacting with an existing multimodal educational application that incorporates the MI framework as a foundation for its interaction structure. The research explores how visual, auditory, and kinesthetic elements influence children’s engagement, affective responses, and interaction behaviors. A qualitative descriptive design was employed through systematic observation and semi-structured interviews involving five young learners aged 5–6 years (N = 5), along with accompanying educators. The study introduces an adapted user experience analysis framework tailored for young learner’s multimodal interaction contexts. Thematic analysis was conducted to identify interaction patterns and usability factors shaping the overall experience. The findings indicate that multimodal interaction enhances engagement, motivation, and accessibility, particularly for children with diverse intelligence profiles. Integrating Multiple Intelligences principles supports adaptive interaction pathways that improve satisfaction and sustained attention. This study contributes to the field of Human Computer Interaction (HCI) by providing empirical evidence on how cognitive diversity can inform the evaluation and design of multimodal interfaces for young learners.
Downloads
References
Buono, P., De Carolis, B., D’Errico, F., Macchiarulo, N., & Palestra, G. (2023). Assessing student engagement from facial behavior in on-line learning. Multimedia Tools and Applications, 82(9), 12859–12877. https://doi.org/10.1007/s11042-022-14048-8
Caiola, V., Cusumano, E., Motta, M., Piro, L., Gelsomini, M., Morra, D., Rizvi, M., & Matera, M. (2023). Designing integrated physical–digital systems for children–nature interaction. International Journal of Child-Computer Interaction, 36, 100582. https://doi.org/10.1016/j.ijcci.2023.100582
Dernikos, B. P., Lesko, N., McCall, S. D., & Niccolini, A. D. (2020). Mapping the Affective Turn in Education Theory, Research, and Pedagogies (First Edit, Issue January). Routledge. https://doi.org/10.4324/9781003004219-15
Dritsas, E., Trigka, M., Troussas, C., & Mylonas, P. (2025). Multimodal Interaction, Interfaces, and Communication: A Survey. Multimodal Technologies and Interaction, 9(1), 1–32. https://doi.org/10.3390/mti9010006
Gao, C., Uchitomi, H., & Miyake, Y. (2023). Influence of Multimodal Emotional Stimulations on Brain Activity: An Electroencephalographic Study. Sensors, 23(10). https://doi.org/10.3390/s23104801
Giannakos, M., & Cukurova, M. (2023). The role of learning theory in multimodal learning analytics. British Journal of Educational Technology, 54(5), 1246–1267. https://doi.org/10.1111/bjet.13320
Guerrero-Sosa, J. D. T., Romero, F. P., Menéndez-Domínguez, V. H., Serrano-Guerrero, J., Montoro-Montarroso, A., & Olivas, J. A. (2025). A Comprehensive Review of Multimodal Analysis in Education. Applied Sciences (Switzerland), 15(11), 1–37. https://doi.org/10.3390/app15115896
He, Z., Li, Z., Yang, F., Wang, L., Li, J., Zhou, C., & Pan, J. (2020). Advances in multimodal emotion recognition based on brain–computer interfaces. Brain Sciences, 10(10), 1–29. https://doi.org/10.3390/brainsci10100687
Hourcade, J. P. (2022). Child-Computer Interaction Second Edition. CreateSpace Independent Publishing Platform. https://homepage.cs.uiowa.edu/~hourcade/book/content.php
Hu, X., & Gao, J. (2024). Facial Expression Recognition Reveal Students’ Engagement in Online L2 Class: Correlations with Six Engagement Measurements. PLoS ONE, 20(10), 1–22. https://doi.org/https://pmc.ncbi.nlm.nih.gov/articles/PMC12543194/
Lalmas, M., O’Brien, H., & Yom-Tov, E. (2014). Measuring User Engagement. In Gary Marchionini (Ed.), Synthesis Lectures on Information Concepts, Retrieval, and Services (Vol. 6, Issue 4). Morgan &Claypool All. https://doi.org/10.2200/s00605ed1v01y201410icr038
Lee-Cultura, S., Sharma, K., & Giannakos, M. (2022). Children’s play and problem-solving in motion-based learning technologies using a multi-modal mixed methods approach. International Journal of Child-Computer Interaction, 31, 100355. https://doi.org/10.1016/j.ijcci.2021.100355
Lee, S. H., & Aspiranti, K. B. (2023). Using multimodal educational apps to increase the vocabulary of children with and without reading difficulties. International Journal of Child-Computer Interaction, 36, 100579. https://doi.org/10.1016/j.ijcci.2023.100579
Leong, W. Y. (2025). Designing for Diversity: Creating Inclusive Digital Learning Environments for Global Classrooms †. Engineering Proceedings, 103(1). https://doi.org/10.3390/engproc2025103017
Liu, H. L., Wang, T. H., Lin, H. C. K., Lai, C. F., & Huang, Y. M. (2022). The Influence of Affective Feedback Adaptive Learning System on Learning Engagement and Self-Directed Learning. Frontiers in Psychology, 13(April), 1–9. https://doi.org/10.3389/fpsyg.2022.858411
Liu, K., & Dr. Erna A. Lahoz. (2024). Impact of Learning Styles on Students’ Retention of Information. International Journal of Education and Humanities, 17(1), 207–212. https://doi.org/10.54097/0qpvve72
Lu, M., & Hu, Z. (2025). Leveraging Multimodal Information for Web Front-End Development Instruction : Analyzing Effects on Cognitive Behavior , Interaction , and Persistent Learning. Information, 16(9), 734. https://doi.org/https://doi.org/10.3390/info16090734
Luo, C. (2023). Research on The Design of Interactive Intelligent Toy Art experience Based on Multimodal Sensory. Applied Mathematics and Nonlinear Sciences, 9(1). https://doi.org/https://doi.org/10.2478/amns.2023.2.00338
Marques, L., Matsubara, P. G., Nakamura, W. T., Ferreira, B. M., Wiese, I. S., Gadelha, B. F., Zaina, L. M., Redmiles, D., & Conte, T. U. (2021). Understanding ux better: A new technique to go beyond emotion assessment. Sensors, 21(21), 1–26. https://doi.org/10.3390/s21217183
Mukuni, K., Tech, V., & Alfeleh, M. (2023). Strategies for Designing Inclusive Online Learning Environment. Association for Educational Communications and Technology, August. https://www.researchgate.net/publication/373238572
Pervez, F., Shoukat, M., Usama, M., Sandhu, M., Latif, S., & Qadir, J. (2024). Affective Computing and the Road to an Emotionally Intelligent Metaverse. IEEE Open Journal of the Computer Society, 5, 195–214. https://doi.org/10.1109/OJCS.2024.3389462
Suárez, V. J. C., Velasco, A. I. B., Roldán, S. H., Besteiro, S. R., Guardado, I. M., Rodríguez, A. M., & Aguilera, J. F. T. (2024). Digital Device Usage and Childhood Cognitive Development : Exploring Effects on Cognitive Abilities. Children, October. https://doi.org/10.3390/children11111299
Toader, C., Tataru, C. P., Florian, I. A., Covache-Busuioc, R. A., Bratu, B. G., Glavan, L. A., Bordeianu, A., Dumitrascu, D. I., & Ciurea, A. V. (2023). Cognitive Crescendo: How Music Shapes the Brain’s Structure and Function. Brain Sciences, 13(10), 1–25. https://doi.org/10.3390/brainsci13101390
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel User Experience Analysis of a Multimodal Digital Application Integrating Multiple Intelligences for Young Learner
Pages: 1628-1636
Copyright (c) 2026 Rini Juliana Sipahutar, Natalia Silalahi, Nina Afria Damayanti
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
