Klasifikasi American Sign Language Menggunakan Convolutional Neural Network


  • Tino Israldi * Mail UIN Sultan Syarif Kasim, Riau, Indonesia
  • Elin Haerani UIN Sultan Syarif Kasim, Riau, Indonesia
  • Suwanto Sanjaya UIN Sultan Syarif Kasim, Riau, Indonesia https://orcid.org/0000-0003-2477-6439
  • Fadhilah Syafria UIN Sultan Syarif Kasim, Riau, Indonesia
  • (*) Corresponding Author
Keywords: American Sign Language; Convolutional Neural Network; Machine Learning

Abstract

Communicating is a necessity for all groups or individual because each individual should communicate with their surroundings. Communicating can also make us get information so that it can be used as a reference to be able to adapt. Verbal language used by speaking out loud is a way of communicating with individuals, but not all individuals can communicate with it, especially there are some individuals who have hearing limitations. Because of these limitations, another program that can be used is through sign language. Language requirements are languages that are usually used by people with disabilities in terms of hearing or speaking and sign language also has a fairly well-known sign language standard, namely the American Sign Language (ASL) standard. Unlike languages in the world, sign language is also often of little interest to most people because people's interest in sign language is still lacking so that most people are unable to understand their language. Sign language has many types, one of which is sign language by using hands to form letters and numbers. In overcoming these problems, the solution is to create a system that can be used to recognize sign language, the system developed is a system that used machine learning technology. This study will propose an ASL classification approach through data preprocessing and a convolutional neural network model. The proposed model can classify ASL hand posture images to be translated into the alphabet. The result of this study is an model with accuracy of 99.8% obtained from the process of merging preprocessing data and the convolutional neural network model.

Downloads

Download data is not yet available.

References

K. K. Republik Indonesia, “Situasi Penyandang Disabilitas,” Pusdatin, 2014. [Online]. Available: https://pusdatin.kemkes.go.id/download.php?file=download/pusdatin/buletin/buletin-disabilitas.pdf. [Accessed: 21-Jun-2022].

Republika, “Sumbar Hanya Punya 8 Penerjemah Bahasa Isyarat,” Republika Online, 2019. [Online]. Available: https://www.republika.co.id/berita/pzjwqr335/sumbar-hanyapunya-8-penerjemah-bahasa-isyarat. [Accessed: 26-Jun-2022].

K. S. R. Indonesia, “Sistem Informasi Management Penyandang Disabilitas,” Kementerian Pendidikan dan Budaya Republik Indonesia, 2021. [Online]. Available: https://simpd.kemensos.go.id. [Accessed: 10-May-2022].

H. Kulhandjian, P. Sharma, M. Kulhandjian, and C. D’Amours, “Sign language gesture recognition using doppler radar and deep learning,” 2019 IEEE Globecom Work. GC Wkshps 2019 - Proc., 2019.

M. A. Jalal, R. Chen, R. K. Moore, and L. Mihaylova, “American Sign Language Posture Understanding with Deep Neural Networks,” 2018 21st Int. Conf. Inf. Fusion, FUSION 2018, pp. 573–579, 2018.

G. A. Rao, K. Syamala, P. V. V. Kishore, and A. S. C. S. Sastry, “Deep convolutional neural networks for sign language recognition,” 2018 Conf. Signal Process. Commun. Eng. Syst. SPACES 2018, vol. 2018-Janua, pp. 194–197, 2018.

A. Dasl, S. Gawde, K. Suratwala, and D. Kalbande, “Sign Language Recognition Using Deep Learning on Custom Processed Static Gesture Images,” 2018 Int. Conf. Smart City Emerg. Technol. ICSCET 2018, 2018.

S. Ameen and S. Vadera, “A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images,” Expert Syst., vol. 34, no. 3, 2017.

L. K. S. Tolentino, R. O. Serfa Juan, A. C. Thio-ac, M. A. B. Pamahoy, J. R. R. Forteza, and X. J. O. Garcia, “Static sign language recognition using deep learning,” Int. J. Mach. Learn. Comput., vol. 9, no. 6, pp. 821–827, 2019.

R. Daroya, D. Peralta, and P. Naval, “Alphabet Sign Language Image Classification Using Deep Learning,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2018-Octob, no. October, pp. 646–650, 2019.

K. Lata, M. Dave, and N. K.N., “Data Augmentation Using Generative Adversarial Network,” SSRN Electron. J., pp. 1–14, 2019.

N. Elsayed, A. S. Maida, and M. Bayoumi, “Empirical activation function effects on unsupervised convolutional LSTM learning,” Proc. - Int. Conf. Tools with Artif. Intell. ICTAI, vol. 2018-Novem, pp. 336–343, 2018.

F. Sultana, A. Sufian, and P. Dutta, “Advancements in image classification using convolutional neural network,” Proc. - 2018 4th IEEE Int. Conf. Res. Comput. Intell. Commun. Networks, ICRCICN 2018, pp. 122–129, 2018.

X. Luo, X. Qin, Z. Wu, F. Yang, M. Wang, and J. Shang, “Sediment Classification of Small-Size Seabed Acoustic Images Using Convolutional Neural Networks,” IEEE Access, vol. 7, pp. 98331–98339, 2019.

S. Z. M. Zaki, M. A. Zulkifley, M. Mohd Stofa, N. A. M. Kamari, and N. A. Mohamed, “Classification of tomato leaf diseases using mobilenet v2,” IAES Int. J. Artif. Intell., vol. 9, no. 2, pp. 290–296, 2020.

I. K. G. Darma Putra, I. P. D. Jayantha Putra, R. Fauzi, and D. Witarsyah, “Classification of Tomato Plants Diseases Using Convolutional Neural Network,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 10, no. 5, pp. 1821–1827, 2020.

A. Roihan, P. A. Sunarya, and A. S. Rafika, “Pemanfaatan Machine Learning dalam Berbagai Bidang: Review paper,” IJCIT (Indonesian J. Comput. Inf. Technol., vol. 5, no. 1, pp. 75–82, 2020.

A. Kurniadi, “Implementasi Convolutional Neural Network Untuk Klasifikasi Varietas Pada Citra Daun Sawi Menggunakan Keras,” DoubleClick J. Comput. Inf. Technol., vol. 4, no. 1, p. 25, 2020.

S. S. Yadav and S. M. Jadhav, “Deep convolutional neural network based medical image classification for disease diagnosis,” J. Big Data, vol. 6, no. 1, 2019.

B. Hanin and D. Rolnick, “How to start training: The effect of initialization and architecture,” Adv. Neural Inf. Process. Syst., vol. 2018-Decem, no. NeurIPS, pp. 571–581, 2018.

D. Normawati and S. A. Prayogi, “Implementasi Naïve Bayes Classifier Dan Confusion Matrix Pada Analisis Sentimen Berbasis Teks Pada Twitter,” J-SAKTI (Jurnal Sains Komput. dan Inform., vol. 5, no. 2, pp. 697–711, 2021.


Bila bermanfaat silahkan share artikel ini

Berikan Komentar Anda terhadap artikel Klasifikasi American Sign Language Menggunakan Convolutional Neural Network

Dimensions Badge
Article History
Submitted: 2022-11-22
Published: 2022-12-26
Abstract View: 240 times
PDF Download: 230 times
How to Cite
Israldi, T., Haerani, E., Sanjaya, S., & Syafria, F. (2022). Klasifikasi American Sign Language Menggunakan Convolutional Neural Network. Building of Informatics, Technology and Science (BITS), 4(3), 1376−1385. https://doi.org/10.47065/bits.v4i3.2570
Issue
Section
Articles