Aplikasi Web Question Answering Menggunakan Langchain OpenAI Tentang Peraturan Perundang-undangan Bidang Pendidikan


  • Ikhsan Dwi Saputra Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
  • Nazruddin Safaat Harahap * Mail Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
  • Surya Agustian Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
  • Muhammad Fikry Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
  • Lola Oktavia Universitas Islam Negeri Sultan Syarif Kasim Riau, Pekanbaru, Indonesia
  • (*) Corresponding Author
Keywords: Question Answering; LangChain; BERTScore; ROUGE Score; Legal Documents; Education

Abstract

In the rapid development of information technology over the past few years, the ease of accessing information has been one of the significant achievements. Artificial intelligence (AI) has emerged as a potential tool in bringing innovative solutions in various sectors of human life. This research aims to develop a web application capable of answering questions related to educational legislation using the LangChain framework and BERT model. The primary issue addressed is the complexity and volume of legal documents that are challenging for lay users to access and understand. The methodology involves converting legal documents from PDF to text, segmenting the text using LangChain, and evaluating system performance with BERTScore and ROUGE Score. The results indicate that BERTScore is superior in measuring the alignment between the system’s answers and reference answers, with some questions achieving a score of 100%. However, there are limitations, such as the manual effort required for document conversion and the substantial computational resources needed for text processing. This research significantly contributes to facilitating access and comprehension of educational legal documents and opens opportunities for further development with more advanced conversion techniques and AI models.

Downloads

Download data is not yet available.

References

L. Beurer-Kellner, M. Fischer, and M. Vechev, “Prompting Is Programming: A Query Language for Large Language Models,” Proc. ACM Program. Lang., vol. 7, no. June, pp. 186:2-186:3, 2023, doi: 10.1145/3591300.

S. Ott et al., “ThoughtSource: A central hub for large language model reasoning data,” Sci. Data, vol. 10, no. 1, pp. 1–13, 2023, doi: 10.1038/s41597-023-02433-3.

O. Topsakal and T. C. Akinci, “Creating Large Language Model Applications Utilizing LangChain: A Primer on Developing LLM Apps Fast,” Int. Conf. Appl. Eng. Nat. Sci., vol. 1, no. 1, pp. 1050–1056, 2023, doi: 10.59287/icaens.1127.

K. Pandya and B. V. Mahavidyalaya, “Automating Customer Service using LangChain,” Comput. Lang. (cs.CL); Comput. Soc. (cs.CY); Mach. Learn., vol. 1, pp. 28–31, 2023, doi: https://doi.org/10.48550/arXiv.2310.05421.

S. K. Nigam, S. K. Mishra, A. K. Mishra, N. Shallum, and A. Bhattacharya, “Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models,” Comput. Lang. (cs.CL); Artif. Intell., vol. 1–2, pp. 1–15, 2023, doi: https://doi.org/10.48550/arXiv.2309.14735.

R. Nakano et al., “WebGPT: Browser-assisted question-answering with human feedback,” Comput. Lang. (cs.CL); Artif. Intell. (cs.AI); Mach. Learn., vol. 1–3, pp. 2–32, 2021, doi: https://doi.org/10.48550/arXiv.2112.09332.

T. Lubiana et al., “Ten quick tips for harnessing the power of ChatGPT in computational biology,” PLOS Comput. Biol., vol. 19, no. 8, p. e1011319, Aug. 2023, doi: 10.1371/journal.pcbi.1011319.

A. Pesaru, T. S. Gill, and A. R. Tangella, “AI assistant for document management Using Lang Chain and Pinecone,” Int. Res. J. Mod. Eng. Technol. Sci., vol. 05, no. 06, pp. 3980–3983, 2023, doi: 10.56726/irjmets42630.

L. Floridi and M. Chiriatti, “GPT-3: Its Nature, Scope, Limits, and Consequences,” Minds Mach., vol. 30, no. 4, pp. 681–694, 2020, doi: 10.1007/s11023-020-09548-1.

T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large Language Models are Zero-Shot Reasoners,” Adv. Neural Inf. Process. Syst., vol. 35, no. NeurIPS, 2022.

H. K. Aroral, “Waterfall Process Operations in the Fast-paced World: Project Management Exploratory Analysis,” Int. J. Appl. Bus. Manag. Stud., vol. 6, no. 1, p. 2021, 2021.

D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: state of the art, current trends and challenges,” Multimed. Tools Appl., vol. 82, no. 3, pp. 3713–3744, 2023, doi: 10.1007/s11042-022-13428-4.

B. A. Andrei, A. C. Casu-Pop, S. C. Gheorghe, and C. A. Boiangiu, “a Study on Using Waterfall and Agile Methods in Software Project Management,” J. Inf. Syst. Oper. Manag., pp. 125–235, 2019.

I. Tri Julianto, D. Kurniadi, Y. Septiana, and A. Sutedi, “Alternative Text Pre-Processing using Chat GPT Open AI,” J. Nas. Pendidik. Tek. Inform., vol. 12, no. 1, pp. 67–77, 2023, doi: 10.23887/janapati.v12i1.59746.

G. Chowdhury, “Natural language processing . Annual Review of This is an author-produced version of a paper published in The Annual Review of Information Science and Technology ISSN 0066-4200 . This version has been peer-reviewed , but does not,” Annu. Rev. Inf. Sci. Technol., vol. 37, pp. 51–89, 2003.

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.

T. B. Brown et al., “Language models are few-shot learners,” Adv. Neural Inf. Process. Syst., vol. 2020-Decem, 2020.

Z. A. Yilmaz, S. Wang, W. Yang, H. Zhang, and J. Lin, “Birch: Applying BERT to document retrieval,” EMNLP-IJCNLP 2019 - 2019 Conf. Empir. Methods Nat. Lang. Process. 9th Int. Jt. Conf. Nat. Lang. Process. Proc. Syst. Demonstr., pp. 19–24, 2020.

W. Sakata, R. Tanaka, T. Shibata, and S. Kurohashi, “FAQ retrieval using query-question similarity and BERT-based query-answer relevance,” SIGIR 2019 - Proc. 42nd Int. ACM SIGIR Conf. Res. Dev. Inf. Retr., pp. 1113–1116, 2019, doi: 10.1145/3331184.3331326.

Y. Qiao, C. Xiong, Z. Liu, and Z. Liu, “Understanding the Behaviors of BERT in Ranking,” 2019, [Online]. Available: http://arxiv.org/abs/1904.07531.

T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating Text Generation With Bert,” 8th Int. Conf. Learn. Represent. ICLR 2020, pp. 1–43, 2020.

J. Risch, T. Möller, J. Gutsch, and M. Pietsch, “Semantic Answer Similarity for Evaluating Question Answering Models,” Proc. 3rd Work. Mach. Read. Quest. Answering, MRQA 2021, pp. 149–157, 2021, doi: 10.18653/v1/2021.mrqa-1.15.

A. Chen, G. Stanovsky, S. Singh, and M. Gardner, “Evaluating question answering evaluation,” MRQA@EMNLP 2019 - Proc. 2nd Work. Mach. Read. Quest. Answering, pp. 119–124, 2019, doi: 10.18653/v1/d19-5817.

M. Barbella and G. Tortora, “Rouge Metric Evaluation for Text Summarization Techniques,” SSRN Electron. J., 2022, doi: 10.2139/ssrn.4120317.

G. Tsuchiya, “Postmortem Angiographic Studies on the Intercoronary Arterial Anastomoses.: Report I. Studies on Intercoronary Arterial Anastomoses in Adult Human Hearts and the Influence on the Anastomoses of Strictures of the Coronary Arteries.,” Jpn. Circ. J., vol. 34, no. 12, pp. 1213–1220, 1971, doi: 10.1253/jcj.34.1213.

W. Tay, A. Joshi, X. Zhang, S. Karimi, and S. Wan, “Red-faced ROUGE: Examining the Suitability of ROUGE for Opinion Summary Evaluation,” Proc. 17th Annu. Work. Australas. Lang. Technol. Assoc., pp. 52–60, 2019, [Online]. Available: https://www.aclweb.org/anthology/U19-1008.

Muchammad Catur Rizky, Rohman Hakim, Miftakhul Anam, Moch Nur Alim, and Wahyu Suhartatik, “Implementasi Undang-Undang Nomor 14 Tahun 2005 Tentang Guru dan Dosen terhadap Kesejahteraan Dosen Profesional di Universitas Sunan Giri Surabaya,” J. Kolaboratif Sains, vol. 5, no. 8, pp. 561–569, 2022, doi: 10.56338/jks.v5i8.2734.


Bila bermanfaat silahkan share artikel ini

Berikan Komentar Anda terhadap artikel Aplikasi Web Question Answering Menggunakan Langchain OpenAI Tentang Peraturan Perundang-undangan Bidang Pendidikan

Dimensions Badge
Article History
Submitted: 2024-11-01
Published: 2024-11-30
Abstract View: 69 times
PDF Download: 64 times
Section
Articles