Hasil Pencarian  ::  Simpan CSV :: Kembali

Abstrak :
Metadata statistik memiliki peran yang sangat penting bagi masyarakat. Dengan adanya metadata statistik, kita dapat mengetahui segala informasi mengenai semua kegiatan statistik yang dilakukan. Pada penelitian ini kami akan membangun sistem Closed Domain Question Answering (CDQA) mengenai metadata statistik (CDQA-Metadata Statistik). Sistem ini dibangun dengan menggunakan metode transfer learning pada data human question dan automatic question. Penggunaan metode transfer learning digunakan karena benchmark yang besar mengenai metadata statistik belum ada sama sekali. Pada penelitian ini kami akan menggunakan arsitektur retriever(BM25)-reader(IndoBERT) berbasis transfer learning. Ada tiga eksperimen utama yang kami lakukan. Hasil eksperimen pertama kami menunjukkan bahwa pada data human question model twostage fine-tuning (human) yang merupakan model dengan metode transfer learning secara statistik sangat signifikan mengguguli model non transfer learning dengan peningkatan exact match sebesar 53 kali lipat dan f1-score sebesar 9 kali lipat. Kemudian pada data automatic question, model two-stage fine-tuning (automatic) yang merupakan model dengan metode transfer learning secara statistik signifikan mengguguli model non transfer learning dengan peningkatan 80 kali lipat untuk exact match dan 13 kali lipat untuk f1-score. Hasil eksperimen kedua kami menujukkan bahwa sistem CDQAMetadata Statistik berbasis transfer learning secara statistik signifikan lebih baik pada data automatic question dibandingkan data human question. Hal ini mungkin disebabkan pada data automatic question memiliki term-of overlap yang lebih banyak dibandingkan data human question. Lalu pada hasil eksperimen ketiga menunjukkan bahwa pada data human question, penambahan data automatic question saat fine-tuning tidak dapat meningkatkan performa CDQA-Metadata Statistik. Begitu juga pada data automatic question, penambahan data human question saat fine-tuning ternyata tidak dapat meningkatkan performa CDQA-Metadata Statistik. ......Statistical metadata plays a very important role in society. With statistical metadata, we can find out all the information regarding all statistical activities carried out. In this research we will build a Closed Domain Question Answering system (CDQA) regarding statistical metadata (CDQA-Statistical Metadata). This system was built using the transfer learning method on human question and automatic question data. The use of the transfer learning method is used because large benchmarks regarding statistical metadata do not yet exist. In this research we will use a retriever (BM25)-reader (IndoBERT) architecture based on transfer learning. There were three main experiments we conducted. The results of our first experiment show that in human question data the two-stage fine-tuning (human) model, which is a model using the transfer learning method, is statistically very significantly superior to the non-transfer learning model with an increase in exact match of 53 times and f1-score of 9 times. Then in the automatic question data, the two-stage fine-tuning (automatic) model, which is a model using the transfer learning method, statistically significantly outperforms the non-transfer learning model with an increase of 80 times for exact match and 13 times for f1-score. The results of our second experiment show that CDQA-Metadata Statistik system based on transfer learning significantly as statistics get better performance in automatic question data than in human question data. This is because automatic question data have more term-of overlap than human question data. Then the results of the third experiment show that for human question data, the addition of the automatic question data during fine-tuning cannot improve the performance of CDQA-Metadata Statistics. Likewise for automatic question data, the addition of a human question data during fine-tuning apparently did not improve the performance of CDQA-Metadata Statistics.

Abstrak :
Sistem tanya jawab merupakan salah satu tugas dalam domain natural language processing (NLP) yang sederhananya bertugas untuk menjawab pertanyaan sesuai konteks yang pengguna berikan ke sistem tanya jawab tersebut. Sistem tanya jawab berbahasa Indonesia sebenarnya sudah ada, namun masih memiliki performa yang terbilang kurang baik. Penelitian ini bereksperimen untuk mencoba meningkatkan performa dari sistem tanya jawab berbahasa Indonesia dengan memanfaatkan natural language inference (NLI). Eksperimen untuk meningkatkan sistem tanya jawab berbahasa Indonesia, penulis menggunakan dua metode, yaitu: intermediate-task transfer learning dan task recasting sebagai verifikator. Dengan metode intermediate-task transfer learning, performa sistem tanya jawab berbahasa Indonesia meningkat, hingga skor F1-nya naik sekitar 5.69 dibandingkan tanpa menggunakan pemanfaatan NLI sama sekali, dan berhasil mendapatkan skor F1 tertinggi sebesar 85.14, namun, peningkatan performa dengan metode intermediate-task transfer learning cenderung tidak signifikan, kecuali pada beberapa kasus khusus model tertentu. Sedangkan dengan metode task recasting sebagai verifikator dengan parameter tipe filtering dan tipe perubahan format kalimat, performa sistem tanya jawab berbahasa Indonesia cenderung menurun, penurunan performa ini bervariasi signifikansinya. Pada penelitian ini juga dilakukan analisis karakteristik pasangan konteks-pertanyaan-jawaban seperti apa yang bisa dijawab dengan lebih baik oleh sistem tanya jawab dengan memanfaatkan NLI, dan didapatkan kesimpulan bahwa: performa sistem tanya jawab meningkat dibandingkan hasil baseline-nya pada berbagai karakteristik, antara lain: pada tipe pertanyaan apa, dimana, kapan, siapa, bagaimana, dan lainnya; kemudian pada panjang konteks ≤ 100 dan 101 ≤ 150; lalu pada panjang pertanyaan ≤ 5 dan 6 ≤ 10; kemudian pada panjang jawaban golden truth ≤ 5 dan 6 ≤ 10; lalu pada keseluruhan answer type selain law dan time; terakhir pada reasoning type WM, SSR, dan MSR. ...... The question-answering system is one of the tasks within the domain of natural language processing (NLP) that, in simple terms, aims to answer questions based on the context provided by the user to the question-answering system. While there is an existing Indonesian question-answering system, its performance is considered somewhat inadequate. This research conducts experiments to improve the performance of the Indonesian question answering system by utilizing natural language inference (NLI). In order to enhance the Indonesian question-answering system, the author employs two methods: intermediate task transfer learning and task recasting as verifiers. Using the intermediate-task transfer learning method, the performance of the Indonesian question-answering system improves significantly, with an increase of approximately 5.69 in F1 score compared to not utilizing NLI at all, achieving the highest F1 score of 85.14. However, the performance improvement with the intermediate-task transfer learning method tends to be non-significant, except in certain specific cases and particular models. On the other hand, employing the task recasting method as a verifier with filtering parameter type and sentence format change type leads to a decline in the performance of the Indonesian question-answering system, with the significance of this performance decrease varying. Additionally, this research conducts an analysis on the characteristics of context-question-answer pairs that can be better answered by the question-answering system utilizing NLI. The findings conclude that the question-answering system’s performance improves compared to its baseline across various characteristics, including different question types such as what, where, when, who, how, and others. Furthermore, it improves with context lengths ≤ 100 and 101 ≤ 150, question lengths ≤ 5 and 6 ≤ 10, as well as answer lengths (golden truth) ≤ 5 and 6 ≤ 10. Additionally, it performs better in overall answer types excluding law and time, and lastly, in reasoning types WM, SSR, and MSR.