Klasifikasi American Sign Language Menggunakan Convolutional Neural Network
Abstract
Communicating is a necessity for all groups or individual because each individual should communicate with their surroundings. Communicating can also make us get information so that it can be used as a reference to be able to adapt. Verbal language used by speaking out loud is a way of communicating with individuals, but not all individuals can communicate with it, especially there are some individuals who have hearing limitations. Because of these limitations, another program that can be used is through sign language. Language requirements are languages that are usually used by people with disabilities in terms of hearing or speaking and sign language also has a fairly well-known sign language standard, namely the American Sign Language (ASL) standard. Unlike languages in the world, sign language is also often of little interest to most people because people's interest in sign language is still lacking so that most people are unable to understand their language. Sign language has many types, one of which is sign language by using hands to form letters and numbers. In overcoming these problems, the solution is to create a system that can be used to recognize sign language, the system developed is a system that used machine learning technology. This study will propose an ASL classification approach through data preprocessing and a convolutional neural network model. The proposed model can classify ASL hand posture images to be translated into the alphabet. The result of this study is an model with accuracy of 99.8% obtained from the process of merging preprocessing data and the convolutional neural network model.
Downloads
References
K. K. Republik Indonesia, “Situasi Penyandang Disabilitas,” Pusdatin, 2014. [Online]. Available: https://pusdatin.kemkes.go.id/download.php?file=download/pusdatin/buletin/buletin-disabilitas.pdf. [Accessed: 21-Jun-2022].
Republika, “Sumbar Hanya Punya 8 Penerjemah Bahasa Isyarat,” Republika Online, 2019. [Online]. Available: https://www.republika.co.id/berita/pzjwqr335/sumbar-hanyapunya-8-penerjemah-bahasa-isyarat. [Accessed: 26-Jun-2022].
K. S. R. Indonesia, “Sistem Informasi Management Penyandang Disabilitas,” Kementerian Pendidikan dan Budaya Republik Indonesia, 2021. [Online]. Available: https://simpd.kemensos.go.id. [Accessed: 10-May-2022].
H. Kulhandjian, P. Sharma, M. Kulhandjian, and C. D’Amours, “Sign language gesture recognition using doppler radar and deep learning,” 2019 IEEE Globecom Work. GC Wkshps 2019 - Proc., 2019.
M. A. Jalal, R. Chen, R. K. Moore, and L. Mihaylova, “American Sign Language Posture Understanding with Deep Neural Networks,” 2018 21st Int. Conf. Inf. Fusion, FUSION 2018, pp. 573–579, 2018.
G. A. Rao, K. Syamala, P. V. V. Kishore, and A. S. C. S. Sastry, “Deep convolutional neural networks for sign language recognition,” 2018 Conf. Signal Process. Commun. Eng. Syst. SPACES 2018, vol. 2018-Janua, pp. 194–197, 2018.
A. Dasl, S. Gawde, K. Suratwala, and D. Kalbande, “Sign Language Recognition Using Deep Learning on Custom Processed Static Gesture Images,” 2018 Int. Conf. Smart City Emerg. Technol. ICSCET 2018, 2018.
S. Ameen and S. Vadera, “A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images,” Expert Syst., vol. 34, no. 3, 2017.
L. K. S. Tolentino, R. O. Serfa Juan, A. C. Thio-ac, M. A. B. Pamahoy, J. R. R. Forteza, and X. J. O. Garcia, “Static sign language recognition using deep learning,” Int. J. Mach. Learn. Comput., vol. 9, no. 6, pp. 821–827, 2019.
R. Daroya, D. Peralta, and P. Naval, “Alphabet Sign Language Image Classification Using Deep Learning,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2018-Octob, no. October, pp. 646–650, 2019.
K. Lata, M. Dave, and N. K.N., “Data Augmentation Using Generative Adversarial Network,” SSRN Electron. J., pp. 1–14, 2019.
N. Elsayed, A. S. Maida, and M. Bayoumi, “Empirical activation function effects on unsupervised convolutional LSTM learning,” Proc. - Int. Conf. Tools with Artif. Intell. ICTAI, vol. 2018-Novem, pp. 336–343, 2018.
F. Sultana, A. Sufian, and P. Dutta, “Advancements in image classification using convolutional neural network,” Proc. - 2018 4th IEEE Int. Conf. Res. Comput. Intell. Commun. Networks, ICRCICN 2018, pp. 122–129, 2018.
X. Luo, X. Qin, Z. Wu, F. Yang, M. Wang, and J. Shang, “Sediment Classification of Small-Size Seabed Acoustic Images Using Convolutional Neural Networks,” IEEE Access, vol. 7, pp. 98331–98339, 2019.
S. Z. M. Zaki, M. A. Zulkifley, M. Mohd Stofa, N. A. M. Kamari, and N. A. Mohamed, “Classification of tomato leaf diseases using mobilenet v2,” IAES Int. J. Artif. Intell., vol. 9, no. 2, pp. 290–296, 2020.
I. K. G. Darma Putra, I. P. D. Jayantha Putra, R. Fauzi, and D. Witarsyah, “Classification of Tomato Plants Diseases Using Convolutional Neural Network,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 10, no. 5, pp. 1821–1827, 2020.
A. Roihan, P. A. Sunarya, and A. S. Rafika, “Pemanfaatan Machine Learning dalam Berbagai Bidang: Review paper,” IJCIT (Indonesian J. Comput. Inf. Technol., vol. 5, no. 1, pp. 75–82, 2020.
A. Kurniadi, “Implementasi Convolutional Neural Network Untuk Klasifikasi Varietas Pada Citra Daun Sawi Menggunakan Keras,” DoubleClick J. Comput. Inf. Technol., vol. 4, no. 1, p. 25, 2020.
S. S. Yadav and S. M. Jadhav, “Deep convolutional neural network based medical image classification for disease diagnosis,” J. Big Data, vol. 6, no. 1, 2019.
B. Hanin and D. Rolnick, “How to start training: The effect of initialization and architecture,” Adv. Neural Inf. Process. Syst., vol. 2018-Decem, no. NeurIPS, pp. 571–581, 2018.
D. Normawati and S. A. Prayogi, “Implementasi Naïve Bayes Classifier Dan Confusion Matrix Pada Analisis Sentimen Berbasis Teks Pada Twitter,” J-SAKTI (Jurnal Sains Komput. dan Inform., vol. 5, no. 2, pp. 697–711, 2021.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Klasifikasi American Sign Language Menggunakan Convolutional Neural Network
Pages: 1376−1385
Copyright (c) 2022 Tino Israldi, Elin Haerani, Suwanto Sanjaya, Fadhilah Syafria

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).