Analisis Spasio-Temporal Berbasis Data Video untuk Identifikasi Bangunan Melayu Menggunakan Metode Hybrid CNN-LSTM
- Ines Triseptiani Universitas Ahmad Dahlan, Yogyakarta, Indonesia
-
Sri Winiarti
*
Universitas Ahmad Dahlan, Yogyakarta,
Indonesia
(*) Corresponding Author
Abstract
Malay buildings have distinctive architectural characteristics and require technology-based identification systems to support cultural documentation and preservation. This study aims to develop an identification system for Malay and non-Malay buildings using video data extracted into frame-by-frame images. The use of video data in this study is not intended to analyze the physical movement of buildings, but to utilize visual variations caused by changes in camera angle, recording distance, lighting, object composition, and visible building elements. The proposed method is CNN-LSTM, where CNN extracts visual features from each frame, while LSTM learns inter-frame feature relationships as a sequence of visual information. To reduce redundant information between adjacent frames and minimize the risk of excessive similarity between training and testing data, the number of frames was limited to a maximum of 15 frames per video folder, and data splitting was performed by considering video source groups. The dataset consists of Riau Malay buildings, Kalimantan Malay buildings, and non-Malay buildings. The research stages include frame extraction, image resizing to 224×224 pixels, normalization, data augmentation, class labeling, train-test splitting, modeling, evaluation, GroupKFold validation, and web-based system implementation. The best testing scenario was obtained using an 80:20 data split, 80 maximum epochs, and a batch size of 16, achieving an accuracy of 0.9916 and a test loss of 0.0852. GroupKFold validation produced an average accuracy of 99.1% with a standard deviation of 0.5%. These results indicate that the model can recognize architectural visual patterns, such as roofs, windows, doors, ornaments, and overall building appearance, while the performance should still be interpreted within the scope of the dataset and evaluation scenario used in this study.
Downloads
References
W. Huang and L. Chen, “Digital design of architectural heritage protection based on 3D reconstruction technology,” Discov. Artif. Intell., vol. 5, p. Art. no. 248, 2025, doi: 10.1007/s44163-025-00504-5.
D. Zhang, “Artificial Intelligence for the Preservation and Transmission of Non-Material Cultural Heritage: Opportunities, Ethical Challenges, and Future Directions,” Appl. Comput. Eng., vol. 174, no. 1, pp. 214–220, 2025, doi: 10.54254/2755-2721/2025.PO25257.
W. Zhang, “Reimagining cultural heritage conservation through VR , metaverse , and digital twins : An AI and blockchain-based framework,” PLoS One, vol. 20, no. 11, p. Art. no. e0335943, 2025, doi: 10.1371/journal.pone.0335943.
J. Hutson, J. Weber, and A. Russo, “Digital Twins and Cultural Heritage Preservation : A Case Study of Best Practices and Reproducibility in Chiesa dei SS Apostoli e Biagio,” Art Des. Rev., vol. 11, no. 1, pp. 15–41, 2023, doi: 10.4236/adr.2023.111003.
F. Gîrbacia, “An Analysis of Research Trends for Using Artificial Intelligence in Cultural Heritage,” Electronics, vol. 13, no. 18, p. Art. no. 3738, 2024, doi: 10.3390/electronics13183738.
A. Heritage et al., “Research on Image Classification and Retrieval Using Deep Learning with Attention Mechanism on Diaspora Chinese Architectural Heritage in Jiangmen, China,” Buildings, vol. 13, no. 2, p. Art. no. 275, 2023, doi: 10.3390/buildings13020275.
M. Wallace, V. Poulopoulos, and A. Antoniou, “An Overview of Big Data Analytics for Cultural Heritage,” Big Data Cogn. Comput., vol. 7, no. 1, p. Art. no. 14, 2023, doi: 10.3390/bdcc7010014.
C. Frescoes, “The Application of ResNet-34 Model Integrating Transfer Learning in the Recognition and Classification of Overseas,” Electronics, vol. 12, no. 17, p. Art. no. 3677, 2023, doi: 10.3390/ electronics12173677.
A. M. Tasir, M. Nor, and A. Khalid, “OPEN A novel deep neural model for efficient and scalable historical place image classification,” Sci. Rep., vol. 15, p. Art. no. 42745, 2025, doi: 10.1038/s41598-025-26897-y 1.
M. Mishra and P. B. Lourenço, “Artificial intelligence-assisted visual inspection for cultural heritage : State-of-the-art review,” J. Cult. Herit., vol. 66, pp. 536–550, 2024, doi: 10.1016/j.culher.2024.01.005.
M. Alzahrani, M. Usman, S. K. Jarraya, S. Anwar, and T. Helmy, “Deep models for multi-view 3D object recognition: a review,” Artif. Intell. Rev., vol. 57, no. 12, p. Art. no. 323, 2024, doi: 10.1007/s10462-024-10941-w.
W. Wang, X. Wang, G. Chen, and H. Zhou, “Multi-view SoftPool attention convolutional networks for 3D model classification,” Front. Neurorobot., vol. 16, p. Art. no. 1029968, 2022, doi: 10.3389/fnbot.2022.1029968.
A. Pandey, P. Kumar, M. Manish, G. Ramraj, and G. Choudhary, “A Deep Learning-Based Hybrid CNN-LSTM Model for Location-Aware Web Service Recommendation,” Neural Process. Lett., vol. 56, no. 5, p. 248, 2024, doi: 10.1007/s11063-024-11687-w.
M. Mao and A. Lee, “Deep Learning Innovations in Video Classification : A Survey on Techniques and Dataset Evaluations,” Electronics, vol. 13, no. 14, pp. 1–30, 2024, doi: 10.3390/ electronics13142732.
X. An, S. Li, and T. Wu, “Modeling Nonlinear Aeroelastic Forces for Bridge Decks with Various Leading Edges Using LSTM Networks,” Appl. Sci., vol. 13, no. 10, p. 6005, 2023, doi: 10.3390/app13106005.
G. Gao et al., “CNN-Bi-LSTM : A Complex Environment-Oriented Cattle Behavior Classification Network Based on the Fusion of CNN,” Sensors, vol. 23, no. 18, p. 7714, 2023, doi: 10.3390/s23187714.
R. V. Bidwe et al., “Deep Learning Approaches for Video Compression : A Bibliometric Analysis,” Big Data Cogn. Comput., vol. 6, no. 2, p. 44, 2022, doi: 10.3390/ bdcc6020044.
B. A. Khan and J. Jung, “Deep Learning-Based Human Activity Recognition Using Dilated CNN and LSTM on Video Sequences of Various Actions Dataset,” Appl. Sci., vol. 15, no. 22, p. 12173, 2025, doi: 10.3390/ app152212173.
A. Rehman, S. B. Belhaouari, and A. Kabir, “On the Use of Deep Learning for Video Classification,” Appl. Sci., vol. 13, no. 12, p. Art. no. 2007, 2023, doi: 10.3390/ app13032007.
S. Tipper, H. F. Atlam, and H. S. Lallie, “An Investigation into the Utilisation of CNN with LSTM for Video Deepfake Detection,” Appl. Sci., vol. 14, no. 21, p. no. 9754, 2024, doi: 10.3390/app14219754.
J. Yoon and M. K. Choi, “Exploring Video Frame Redundancies for Efficient Data Sampling and Annotation in Instance Segmentation,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., vol. 2023-June, pp. 3308–3317, 2023, doi: 10.1109/CVPRW59228.2023.00333.
R. J. Babić, “Cultural heritage image classification using transfer learning for feature extraction: a comparison,” CEUR Workshop Proc., vol. 3266, 2022.[Online]. Available: https://ceur-ws.org/Vol-3266/paper4.pdf
J. Rodziewicz, A. Mielcarek, W. Janczukowicz, K. Bryszewski, and A. Jabło, “Technological Parameters of Rotating Electrochemical and Electrobiological Disk Contactors Depending on the Effluent Quality Requirements,” Appl. Sci., vol. 12, no. 11, p. 5503, 2022, doi: 10.3390/app12115503.
A. M. Halim, M. D. P, and F. Nhita, “Handling Imbalanced Data Sets Using SMOTE and ADASYN to Improve Classification Performance of Ecoli Data Sets,” Build. Informatics, Technol. Sci., vol. 5, no. 1, pp. 246–253, 2023, doi: 10.47065/bits.v5i1.3647.
A. Bensaoud and J. Kalita, “CNN-LSTM and Transfer Learning Models for Malware Classification based on Opcodes and API Calls,” Knowledge-Based Syst., 2023, doi: 10.1016/j.knosys.2024.111543.
D. Akinola, A. O. Oyedemi, and M. O. Ajinaja, “A Deep Learning - based Hybrid CNN - LSTM Model for Human Activity Recognition,” Int. J. Sci. Res. Comput. Sci. Eng., vol. 12, no. 6, pp. 66–74, 2024.
L. N. I. Afida, F. A. Bachtiar, and I. Cholissodin, “Klasifikasi Aktivitas Manusia Menggunakan Metode Long Short-Term Memory,” J. Teknol. Inf. dan Ilmu Komput., vol. 11, no. 2, pp. 357–368, 2024, doi: 10.25126/jtiik.20241127060.
B. Septian, C. Putra, and I. Tahyudin, “JITE ( Journal of Informatics and Telecommunication Engineering ) Performance Evaluation of CNN-LSTM and CNN-FNN Combinations for,” JITE (Journal Informatics Telecommun. Eng., vol. 8, no. 2, pp. 196–207, 2025, doi: 10.31289/jite.v8i2.13503.
A. Nur, H. Mubarok, E. Nur, F. Dewi, and R. Edwinda, “Implementation of Convolutional Neural Network and Long Short- Term Memory Algorithms in Human Activity Recognition Based on Visual Processing Video,” JOIV Int. J. Informatics Vis., vol. 7, no. June, pp. 494–501, 2023, doi: 10.30630/joiv.7.2.1504.
K. Siountri and C. N. Anagnostopoulos, “The Classification of Cultural Heritage Buildings in Athens Using Deep Learning Techniques,” Heritage, vol. 6, no. 4, pp. 3673–3705, 2023, doi: 10.3390/heritage6040195.
H. Pramono, S. Winiarti, and A. Fadlil, “Identifying Traditional Malay Building Architectural Styles Using Vision Transformer Architecture,” Int. J. Informatics Comput., vol. 7, no. 2, pp. 672–689, 2025, doi: 10.35842/ijicom.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Analisis Spasio-Temporal Berbasis Data Video untuk Identifikasi Bangunan Melayu Menggunakan Metode Hybrid CNN-LSTM
Pages: 318-329
Copyright (c) 2026 Ines Triseptiani, Sri Winiarti
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
