Aplikasi Web Question Answering Menggunakan Langchain OpenAI Tentang Peraturan Perundang-undangan Bidang Pendidikan
Abstract
In the rapid development of information technology over the past few years, the ease of accessing information has been one of the significant achievements. Artificial intelligence (AI) has emerged as a potential tool in bringing innovative solutions in various sectors of human life. This research aims to develop a web application capable of answering questions related to educational legislation using the LangChain framework and BERT model. The primary issue addressed is the complexity and volume of legal documents that are challenging for lay users to access and understand. The methodology involves converting legal documents from PDF to text, segmenting the text using LangChain, and evaluating system performance with BERTScore and ROUGE Score. The results indicate that BERTScore is superior in measuring the alignment between the system’s answers and reference answers, with some questions achieving a score of 100%. However, there are limitations, such as the manual effort required for document conversion and the substantial computational resources needed for text processing. This research significantly contributes to facilitating access and comprehension of educational legal documents and opens opportunities for further development with more advanced conversion techniques and AI models.
Downloads
References
L. Beurer-Kellner, M. Fischer, and M. Vechev, “Prompting Is Programming: A Query Language for Large Language Models,” Proc. ACM Program. Lang., vol. 7, no. June, pp. 186:2-186:3, 2023, doi: 10.1145/3591300.
S. Ott et al., “ThoughtSource: A central hub for large language model reasoning data,” Sci. Data, vol. 10, no. 1, pp. 1–13, 2023, doi: 10.1038/s41597-023-02433-3.
O. Topsakal and T. C. Akinci, “Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast,” Int. Conf. Appl. Eng. Nat. Sci., vol. 1, no. 1, pp. 1050–1056, 2023, doi: 10.59287/icaens.1127.
K. Pandya and B. V. Mahavidyalaya, “Automating Customer Service using LangChain,” Comput. Lang. (cs.CL); Comput. Soc. (cs.CY); Mach. Learn., vol. 1, pp. 28–31, 2023, doi: https://doi.org/10.48550/arXiv.2310.05421.
S. K. Nigam, S. K. Mishra, A. K. Mishra, N. Shallum, and A. Bhattacharya, “Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models,” Comput. Lang. (cs.CL); Artif. Intell., vol. 1–2, pp. 1–15, 2023, doi: https://doi.org/10.48550/arXiv.2309.14735.
R. Nakano et al., “WebGPT: Browser-assisted question-answering with human feedback,” Comput. Lang. (cs.CL); Artif. Intell. (cs.AI); Mach. Learn., vol. 1–3, pp. 2–32, 2021, doi: https://doi.org/10.48550/arXiv.2112.09332.
T. Lubiana et al., “Ten quick tips for harnessing the power of ChatGPT in computational biology,” PLOS Comput. Biol., vol. 19, no. 8, p. e1011319, Aug. 2023, doi: 10.1371/journal.pcbi.1011319.
A. Pesaru, T. S. Gill, and A. R. Tangella, “AI assistant for document management Using Lang Chain and Pinecone,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 05, no. 06, pp. 3980–3983, 2023, doi: 10.56726/irjmets42630.
L. Floridi and M. Chiriatti, “GPT-3: Its Nature, Scope, Limits, and Consequences,” Minds Mach., vol. 30, no. 4, pp. 681–694, 2020, doi: 10.1007/s11023-020-09548-1.
T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners,” Adv. Neural Inf. Process. Syst., vol. 35, no. NeurIPS, 2022.
H. K. Aroral, “Waterfall Process Operations in the Fast-paced World: Project Management Exploratory Analysis,” Int. J. Appl. Bus. Manag. Stud., vol. 6, no. 1, p. 2021, 2021.
D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: state of the art, current trends and challenges,” Multimed. Tools Appl., vol. 82, no. 3, pp. 3713–3744, 2023, doi: 10.1007/s11042-022-13428-4.
B. A. Andrei, A. C. Casu-Pop, S. C. Gheorghe, and C. A. Boiangiu, “a Study on Using Waterfall and Agile Methods in Software Project Management,” J. Inf. Syst. Oper. Manag., pp. 125–235, 2019.
I. Tri Julianto, D. Kurniadi, Y. Septiana, and A. Sutedi, “Alternative Text Pre-Processing using Chat GPT Open AI,” J. Nas. Pendidik. Tek. Inform., vol. 12, no. 1, pp. 67–77, 2023, doi: 10.23887/janapati.v12i1.59746.
G. Chowdhury, “Natural language processing . Annual Review of This is an author-produced version of a paper published in The Annual Review of Information Science and Technology ISSN 0066-4200 . This version has been peer-reviewed , but does not,” Annu. Rev. Inf. Sci. Technol., vol. 37, pp. 51–89, 2003.
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.
T. B. Brown et al., “Language models are few-shot learners,” Adv. Neural Inf. Process. Syst., vol. 2020-Decem, 2020.
Z. A. Yilmaz, S. Wang, W. Yang, H. Zhang, and J. Lin, “Birch: Applying BERT to document retrieval,” EMNLP-IJCNLP 2019 - 2019 Conf. Empir. Methods Nat. Lang. Process. 9th Int. Jt. Conf. Nat. Lang. Process. Proc. Syst. Demonstr., pp. 19–24, 2020.
W. Sakata, R. Tanaka, T. Shibata, and S. Kurohashi, “FAQ retrieval using query-question similarity and BERT-based query-answer relevance,” SIGIR 2019 - Proc. 42nd Int. ACM SIGIR Conf. Res. Dev. Inf. Retr., pp. 1113–1116, 2019, doi: 10.1145/3331184.3331326.
Y. Qiao, C. Xiong, Z. Liu, and Z. Liu, “Understanding the Behaviors of BERT in Ranking,” 2019, [Online]. Available: http://arxiv.org/abs/1904.07531.
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating Text Generation With Bert,” 8th Int. Conf. Learn. Represent. ICLR 2020, pp. 1–43, 2020.
J. Risch, T. Möller, J. Gutsch, and M. Pietsch, “Semantic Answer Similarity for Evaluating Question Answering Models,” Proc. 3rd Work. Mach. Read. Quest. Answering, MRQA 2021, pp. 149–157, 2021, doi: 10.18653/v1/2021.mrqa-1.15.
A. Chen, G. Stanovsky, S. Singh, and M. Gardner, “Evaluating question answering evaluation,” MRQA@EMNLP 2019 - Proc. 2nd Work. Mach. Read. Quest. Answering, pp. 119–124, 2019, doi: 10.18653/v1/d19-5817.
M. Barbella and G. Tortora, “Rouge Metric Evaluation for Text Summarization Techniques,” SSRN Electron. J., 2022, doi: 10.2139/ssrn.4120317.
G. Tsuchiya, “Postmortem Angiographic Studies on the Intercoronary Arterial Anastomoses.: Report I. Studies on Intercoronary Arterial Anastomoses in Adult Human Hearts and the Influence on the Anastomoses of Strictures of the Coronary Arteries.,” Jpn. Circ. J., vol. 34, no. 12, pp. 1213–1220, 1971, doi: 10.1253/jcj.34.1213.
W. Tay, A. Joshi, X. Zhang, S. Karimi, and S. Wan, “Red-faced ROUGE: Examining the Suitability of ROUGE for Opinion Summary Evaluation,” Proc. 17th Annu. Work. Australas. Lang. Technol. Assoc., pp. 52–60, 2019, [Online]. Available: https://www.aclweb.org/anthology/U19-1008.
Muchammad Catur Rizky, Rohman Hakim, Miftakhul Anam, Moch Nur Alim, and Wahyu Suhartatik, “Implementasi Undang-Undang Nomor 14 Tahun 2005 Tentang Guru dan Dosen terhadap Kesejahteraan Dosen Profesional di Universitas Sunan Giri Surabaya,” J. Kolaboratif Sains, vol. 5, no. 8, pp. 561–569, 2022, doi: 10.56338/jks.v5i8.2734.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Aplikasi Web Question Answering Menggunakan Langchain OpenAI Tentang Peraturan Perundang-undangan Bidang Pendidikan
Pages: 293-304
Copyright (c) 2024 Ikhsan Dwi Saputra, Nazruddin Safaat Harahap, Surya Agustian, Muhammad Fikry, Lola Oktavia

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).