Android Based Indonesian Sign Language Hand Gesture Detection using Transfer Learning Method
-
Mohammad Iqbal Bachtiar
*
University of KH. Bahaudin Mudhary Madura, Sumenep,
Indonesia
- Rizki Anantama University of KH. Bahaudin Mudhary Madura, Sumenep, Indonesia
- Desi Anis Anggraini University of KH. Bahaudin Mudhary Madura, Sumenep, Indonesia
- Zeinor Rahman University of KH. Bahaudin Mudhary Madura, Sumenep, Indonesia
- Mohammad Ilham Bahri University of KH. Bahaudin Mudhary Madura, Sumenep, Indonesia (*) Corresponding Author
Abstract
People with hearing and speech impairments rely on sign language for daily communication. However, public understanding of the Indonesian Sign Language System (SIBI), particularly its alphabet hand gestures, remains limited, creating communication barriers between deaf individuals and the wider community. This study aims to develop an Android-based application for recognizing SIBI alphabet hand gestures using the Transfer Learning method implemented through Google Teachable Machine. A dataset consisting of 200 images for each SIBI alphabet gesture was collected and used to train the classification model. Several training scenarios were evaluated by varying the Epoch, Batch Size, and Learning Rate parameters to obtain the optimal model. The trained model was then converted into TensorFlow Lite and integrated into an Android application for real-time hand gesture recognition. Experimental results show that the best performance was achieved using Epoch 50, Batch Size 16, and Learning Rate 0.001, producing training and validation accuracy of 1.0 with an error rate close to 0.0. The developed application successfully recognized all tested static SIBI alphabet gestures, demonstrating reliable performance in practical implementation. This study contributes by providing a lightweight Android-based SIBI hand gesture recognition system and experimentally evaluating the influence of training hyperparameters on recognition performance. The proposed approach offers an efficient solution for supporting SIBI learning and improving communication accessibility for deaf or speech-impaired individuals.
Downloads
References
Anggraeni, M. E., Sarinastiti, W., & Wati, S. (2019). Indonesian Sign Language (SIBI) Vocabulary Learning Media Design Based on Augmented Reality for Hearing-Impaired Children. Jurnal EECCIS, 13(3), 139–144.
Anwar, A., Basuki, A., & Sigit, R. (2020). Hand Gesture Recognition For Indonesian Sign Language Interpreter System With Myo Armband Using Support Vector Machine. Klik - Kumpulan Jurnal Ilmu Komputer, 7(2), 164. https://doi.org/10.20527/klik.v7i2.320
Camgoz, N. C., Hadfield, S., Koller, O., & Bowden, R. (2017). SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. 2017 IEEE International Conference on Computer Vision (ICCV), 3075–3084. https://doi.org/10.1109/ICCV.2017.332
Den Hoed, J., & Fisher, S. E. (2020). Genetic pathways involved in human speech disorders. Current Opinion in Genetics & Development, 65, 103–111. https://doi.org/10.1016/j.gde.2020.05.012
DiMarzio, J. F. (2016). Beginning Android® Programming with Android Studio (1 ed.). Wiley. https://doi.org/10.1002/9781119419334
Eldeen, R. K., & H.G.Yousif, E. (2021). A Hand Gesture Recognition System for Deaf-Mute Individuals. 21(3).
Forchhammer, S., Abu-Ghazaleh, A., Metzler, G., Garbe, C., & Eigentler, T. (2022). Development of an Image Analysis-Based Prognosis Score Using Google’s Teachable Machine in Melanoma. Cancers, 14(9), 2243. https://doi.org/10.3390/cancers14092243
Gresse Von Wangenheim, C., Marques, L. S., & Hauck, J. C. R. (2020). Machine Learning for All – Introducing Machine Learning in K-12. https://doi.org/10.31235/osf.io/wj5ne
Harditya, A. (2020). Indonesian Sign Language (BISINDO) As Means to Visualize Basic Graphic Shapes Using Teachable Machine: Proceedings of the International Conference of Innovation in Media and Visual Design (IMDES 2020). International Conference of Innovation in Media and Visual Design (IMDES 2020), Tangerang, Indonesia. https://doi.org/10.2991/assehr.k.201202.045
Huang, Z., Siniscalchi, S. M., & Lee, C.-H. (2016). A unified approach to transfer learning of deep neural networks with applications to speaker adaptation in automatic speech recognition. Neurocomputing, 218, 448–459. https://doi.org/10.1016/j.neucom.2016.09.018
Kamruzzaman, M. M. (2020). Arabic Sign Language Recognition and Generating Arabic Speech Using Convolutional Neural Network. Wireless Communications and Mobile Computing, 2020, 1–9. https://doi.org/10.1155/2020/3685614
Kumar, L. A., Renuka, D. K., Rose, S. L., Shunmuga Priya, M. C., & Wartana, I. M. (2022). Deep learning based assistive technology on audio visual speech recognition for hearing impaired. International Journal of Cognitive Computing in Engineering, 3, 24–30. https://doi.org/10.1016/j.ijcce.2022.01.003
Kumar, M., Singh, S., Singh, D. P., & Rai, A. K. (2025). Hybrid CNN–LSTM Transfer Learning for Real-Time Hand Gesture Recognition. 2025 3rd International Conference on IoT, Communication and Automation Technology (ICICAT), 1–6. https://doi.org/10.1109/ICICAT68430.2025.11414484
Kurhekar, P., Phadtare, J., Sinha, S., & Shirsat, K. P. (2019). Real Time Sign Language Estimation System. 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), 654–658. https://doi.org/10.1109/ICOEI.2019.8862701
Li, J., Jiang, X., Fan, J., Geng, Y., Jia, F., & Dai, C. (2025). Deep end-to-end transfer learning for robust inter-subject and inter-day hand gesture recognition using surface EMG. Biomedical Signal Processing and Control, 100, 106892. https://doi.org/10.1016/j.bspc.2024.106892
Lian, Y., Lu, Z., Huang, X., Shangguan, Q., Yao, L., Huang, J., & Liu, Z. (2024). A Transfer Learning Strategy for Cross-Subject and Cross-Time Hand Gesture Recognition Based on A-Mode Ultrasound. IEEE Sensors Journal, 24(10), 17183–17192. https://doi.org/10.1109/JSEN.2024.3382040
Malahina, E. A. U., Hadjon, R. P., & Bisilisin, F. Y. (2022). Teachable Machine: Real-Time Attendance of Students Based on Open Source System. The IJICS (International Journal of Informatics and Computer Science), 6(3), 140. https://doi.org/10.30865/ijics.v6i3.4928
Prasad, P. Y., Prasad, D. D., & Malleswari, D. D. N. (2022). Implementation of Machine Learning Based Google Teachable Machine in Early Childhood Education. International Journal of Early Childhood Special Education, 14(03).
Rakun, E., Arymurthy, A. M., Stefanus, L. Y., Wicaksono, A. F., & Wisesa, I. W. W. (2018). Recognition of Sign Language System for Indonesian Language Using Long Short-Term Memory Neural Networks. Advanced Science Letters, 24(2), 999–1004. https://doi.org/10.1166/asl.2018.10675
Ribeiro, A., Ferreira, J. F., & Mendes, A. (2021). EcoAndroid: An Android Studio Plugin for Developing Energy-Efficient Java Mobile Applications. 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), 62–69. https://doi.org/10.1109/QRS54544.2021.00017
Vaidya, O., Gandhe, S., Sharma, A., Bhate, A., Bhosale, V., & Mahale, R. (2020). Design and Development of Hand Gesture based Communication Device for Deaf and Mute People. 2020 IEEE Bombay Section Signature Conference (IBSSC), 102–106. https://doi.org/10.1109/IBSSC51096.2020.9332208
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3(1), 9. https://doi.org/10.1186/s40537-016-0043-6
Zhang, Z., Liu, S., Wang, Y., Song, W., & Zhang, Y. (2024). Online cross session electromyographic hand gesture recognition using deep learning and transfer learning. Engineering Applications of Artificial Intelligence, 127, 107251. https://doi.org/10.1016/j.engappai.2023.107251
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Android Based Indonesian Sign Language Hand Gesture Detection using Transfer Learning Method
Pages: 354-365
Copyright (c) 2026 Mohammad Iqbal Bachtiar, Rizki Anantama, Desi Anis Anggraini, Zeinor Rahman, Mohammad Ilham Bahri
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
