Integrasi Large Language Model dengan FAQ Retrieval dan Sinkronisasi Basis Pengetahuan Berbasis Web Scraping pada Chatbot Layanan Admisi

Abstract

University admissions services require timely and consistent answers, whereas manually maintained chatbot knowledge bases can quickly become outdated. This study develops an admissions chatbot integrating a Large Language Model, semantic FAQ retrieval, and knowledge base synchronization through web scraping. The system combines a Next.js interface, a Flask backend, Firebase Firestore, BeautifulSoup for static pages, Selenium for dynamic pages, and a generative model constrained by retrieved FAQ context. Development follows Design Science Research. Retrieval represents each FAQ as a 384-dimensional multilingual embedding, measures similarity using cosine similarity, and passes the top five FAQs to the model as answer context. Semantic retrieval is compared with a TF-IDF baseline, while answer quality is compared with an LLM-only configuration. Because operational data were unavailable, the evaluation used a controlled synthetic benchmark comprising 180 questions and 30 test URLs. In the simulation, semantic retrieval achieved 88.1% Hit@1, 93.8% Hit@3, and 0.909 MRR. FAQ-grounded generation improved simulated answer correctness from 72.2% for the LLM-only configuration to 91.1% and reduced unsupported answers from 17.8% to 5.0%. Direct functional testing in the deployed environment showed that four of seven scenarios passed. These findings indicate design feasibility, but field validation with real users remains necessary. The contribution of this study is an auditable orchestration design that connects official-source scraping, administrator curation, semantic retrieval, and the LLM in a single workflow, together with a separated evaluation framework for retrieval, answer quality, scraped-data quality, and system performance.