Implementasi Pengenalan Ekspresi Wajah dengan Menggunakan Metode Convolutional Neural Network dan OpenCV Berbasis Webcam


  • Raifvaldhy Jounias Luppus Melatisudra Universitas Nurtanio, Bandung, Indonesia
  • Suharjanto Utomo * Mail Universitas Nurtanio, Bandung, Indonesia
  • Sri Sutjiningtyas Universitas Nurtanio, Bandung, Indonesia
  • Hernawati Hernawati Universitas Nurtanio, Bandung, Indonesia
  • (*) Corresponding Author
Keywords: Facial Expression; Convolutional Neural Network; OpenCV; Webcam; Classification Accuracy; Confusion Matrix; Psychology

Abstract

Facial expressions play an important role as a non-verbal language rich in emotional information, especially in psychological contexts. The main challenge in this research is to understand and analyse complex facial expressions, which are often difficult for psychologists to interpret. The implementation of webcam-based facial expression recognition leverages the computer's ability to visually recognise human emotions, supported by artificial intelligence and machine learning. Convolutional Neural Network (CNN) and OpenCV methods are used to detect and classify facial expressions directly. The CNN model is trained using a dataset with six expression classes (happy, sad, angry, surprised, neutral, afraid), with four convolution layers for multi-class classification. The implementation of facial expression recognition is successful, the system captures facial images from a webcam, detects faces in the frame, and classifies facial expressions directly on the screen window. The performance of training data against the trained model measured using Classification Accuracy shows an accuracy of 72.34% in training accuracy and 60.54% in validation accuracy. While the performance of the facial expression recognition system calculated using Confusion Matrix resulted in an accuracy of 70.55%. The calculation results show that the model is at the Fair Classification parameter level or able to classify facial expressions in humans with a fairly good level of accuracy, this research has great potential for application development in the field of psychology. However, further optimisation is needed by involving experts to ensure its effectiveness.

Downloads

Download data is not yet available.

References

S. Masrichah, “Ancaman dan Peluang Artificial Intelligence (AI),” Jurnal Pendidikan dan Sosial Humaniora., vol. 3, pp. 83 101, 2023.

M. Farwati, I. T. Salsabila, K. R. Navira, and T. Sutabri, “Analisa Pengaruh Teknologi Artificial Intelligence (AI) dalam Kehidupan Sehari-hari,” Universitas Darma Palembang., 2023.

D. M. Ulhaq, M. Zaidan, and D. Firdaus, "Pengenalan Ekspresi Wajah Secara Real-time Menggunakan Metode SSD Mobilenet Berbasis Android," Journal of Technology and Informatics (JoTI)., pp. 48-52, 2023.

Wikipedia. Ekspresi Wajah, Access Date July 2024, [Online].Available: https://id.wikipedia.org/wiki/Ekspresi_wajah.

A. Muttaqiin, H. Yuana, and M. Chulkamdi, "Implementasi Algoritma Convolutional Neural Network Untuk Pengenalan Ekspresi Wajah," Jurnal Riset Sistem Informasi dan Teknik Informatika (JURASIK)., pp. 772-792, 2023.

D. Setyadi, “Deteksi Ekspresi Wajah Menggunakan Metode CNN,” 2022.

M. Fauziah, “Perancangan dan Implementasi Sistem Deteksi Gerakan Kepala, Mata, dan Alis Berbasis Machine Learning,” Institut Teknologi Bandung., 2021.

A. N. Pulung, I. Fenriana, and R. Arijanto, "Implementasi Deep Learning Menggunakan Convolutional Neural Network (CNN) pada Ekspresi Manusia," Jurnal Algor., 2020.

S. A. Lioga, P. A. Hendrianto, and M. Florestiyanto, "Implementation of Convolutional Neural Network (CNN) in Facial Expression Recognition," Jurnal Informatika dan Teknologi Informasi., pp. 211-221, 2021.

S. Li dan W. Deng, "Deep Facial Expression Recognition: A Survey," IEEE Transactions on Affective Computing., vol.13, no. 3, pp. 1195-1215, 2020.

Y. Wang, Y. Zhang, and S. Zhang, "Facial expression recognition: A comprehensive survey," International Journal of Computer Applications., vol. 179, no. 1, pp. 1-6, 2018.

J. E. Widyaya and S. Budi, "Pengaruh Preprocessing Terhadap Klasifikasi Diabetic Retinopathy dengan Pendekatan Transfer Learning Convolutional Neural Network," Jurnal Teknik Informatika dan Sistem Informasi., vol. 7, no. 1, 2021.

R. Kumar, S. Agarwal, and A. Kumar, "Facial Expression Recognition: A Survey," International Journal of Computer Applications., vol. 181, no. 20, pp. 18-24, 2019.

Y. LeCun, Y. Bengio, and G. Haffner, Gradient-Based Learning Applied to Document Recognition," Proc. IEEE vol. 86, no. 11, pp. 2278-2324, 1998.

M. A. Leonardi and A. Y. Chandra, "Analisis Perbandingan CNN dan Vision Transformer untuk Klasifikasi Biji Kopi Hasil Sangrai," JURNAL MEDIA INFORMATIKA BUDIDARMA., vol. 8, no. 3, pp. 1398-1307, 2024.

D. Anggara, N. Suarna, and Y. A. Wijaya, "ANALISA PERBANDINGAN PERFORMA OPTIMIZER ADAM, SGD, DAN RMSPROP PADA MODEL H5," NERO (Networking Engineering Research Operation)., vol. 8, no. 1, pp. 53-64, 2023.

D. Pazriyah, “Penggunaan Raspberry Pi Dalam Mendeteksi Warna Melalui Webcam,” Politeknik Negeri Sriwijaya, 2017.

D. Leni, "Pemodelan Machine Learning untuk Memprediksi Tensile Strength Aluminium Menggunakan Algoritma Artificial Neural Network (ANN)," Jurnal Surya Teknika., vol. 10, no. 1, pp. 625-632, 2023.

R. S. Irawan, “APLIKASI PERANGKAT BERGERAK PENGKLASIFIKASI KESEGARAN DAGING MENGGUNAKAN METODE CONVOLUTIONAL NEURAL NETWORK (CNN) DENGAN ARSITEKTUR MOBILENETV2,” Sekolah Tinggi Informatika & Komputer Indonesia., 2024.

M. Zufar and B. Setiyono, "Convolutional Neural Networks untuk Pengenalan Wajah Secara Real-Time," Jurnal Sains dan Seni ITS., vol. 5, no. 2, pp. 128-862, 2016.

N. Ketkar, J. Moolayil, "Convolutional Neural Networks," in Deep Learning with Python: Learn Best Practices of Deep Learning Models with PyTorch., pp. 197-242, 2021.

M. Sun, Z. Song, X. Jiang, J. Pan, and Y. Pang, "Learning Pooling for Convolutional Neural Network," Neurocomputing.,vol. 224, pp. 96-104, 2017.

S. S. Basha, S. R. Dubey, V. Pulabaigari, and S. Mukherjee, "Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification," Neurocomputing., vol. 378, pp. 112-119, 2020.

D. Sinaga, "Jaringan Saraf Tiruan Infeksi Mata Dengan Menggunakan Metode Beraksitektur Multi Layer Perceptron,"Informasi dan Teknologi Ilmiah (INTI)., vol. 7, no. 2, pp. 189-192, 2020.


Bila bermanfaat silahkan share artikel ini

Berikan Komentar Anda terhadap artikel Implementasi Pengenalan Ekspresi Wajah dengan Menggunakan Metode Convolutional Neural Network dan OpenCV Berbasis Webcam

Dimensions Badge
Article History
Submitted: 2024-10-22
Published: 2024-11-30
Abstract View: 117 times
PDF Download: 59 times
Section
Articles