Randy Goebel
ABSTRACT
We summarize the 10th Competition on Legal Information Extraction and Entailment. In this edition, the competition included four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law component includes an information retrieval task (Task 3), and an entailment/question answering task based on retrieved civil code statutes (Task 4). Participation was open to any group based on any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received results from 8 teams for Task 1 (22 runs) and seven teams for Task 2 (18 runs). On the statute law task, there were 9 different teams participating, most in more than one task. 6 teams submitted a total of 16 runs for Task 3, and 9 teams submitted a total of 26 runs for Task 4. We describe the variety of approaches, our official evaluation, and analysis of our data and submission results.
- Onur Bilgin, Logan Fields, Antonio Laverghetta Jr., Zaid Marji, Animesh Nighojkar, Stephen Steinle, and John Licato. 2023. AMHR Lab 2023 COLIEE Competition Approach. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE '2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Quan Minh Bui, Dinh-Truong Do, Nguyen-Khang Le, Dieu-Hien Nguyen, Khac-Vu-Hiep Nguyen, Trang Pham Ngoc Anh, and Minh Nguyen Le. 2023. JNLP COLIEE-2023: Data Argumentation and Large Language Model for Legal Case Retrieval and Entailment. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Michel Custeau and Diana Inkpen. 2023. Individual Models Can Perform Better than Agreement-Based Ensembles. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Rohan Debbarma, Pratik Prawar, Abhijnan Chakraborty, and Srikanta Bedathur. 2023. IITDLI: Legal Case Retrieval Based on Lexical Models. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Haitao Li, Weihang Su, Changyue Wang, Yueyue Wu, Qingyao Ai, and Yiqun Liu. 2023. THUIR@COLIEE 2023: Incorporating Structural Knowledge into Pretrained Language Models for Legal Case Retrieval. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. In arXiv preprint arXiv:1907.11692.Google Scholar
- Chau Nguyen and Minh-Le Nguyen. 2023. CAPTAIN at COLIEE 2023: Efficient Methods for Legal Information Retrieval and Entailment Tasks. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, Online, 708--718. https://doi.org/10.18653/v1/2020.findings-emnlp.63Google ScholarCross Ref
- Luisa Pereira Novaes, Daniela Vianna, and Altigran da Silva. 2023. A Topic-Based Approach for the Legal Case Retrieval Task. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Takaaki Onaga, Masaki Fujita, and Yoshinobu Kano. 2023. Japanese Legal Bar Problem Solver Focusing on Person Names. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE '2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Juliano Rabelo, Mi-Young Kim, and Randy Goebel. 2023. Transformer-based Legal Information Extraction. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Juliano Rabelo, Mi-Young Kim, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2021. COLIEE 2020: Methods for Legal Document Retrieval and Entailment. 196--210. https://doi.org/10.1007/978-3-030-79942-7_13Google ScholarDigital Library
- Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr. 3, 4 (apr 2009), 333--389. https://doi.org/10.1561/1500000019Google ScholarDigital Library
- Thi-Hai-Yen Vuong, Hai-Long Nguyen, Tan-Minh Nguyen, Hoang-Trung Nguyen, Thai-Binh Nguyen, and Ha-Thanh Nguyen. 2023. NOWJ at COLIEE 2023 - Multi-Task and Ensemble Approaches in Legal Information Processing. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- Masaharu Yoshioka. 2018. Analysis of COLIEE Information Retrieval Task Data. In New Frontiers in Artificial Intelligence, Sachiyo Arai, Kazuhiro Kojima, Koji Mineshima, Daisuke Bekki, Ken Satoh, and Yuiko Ohta (Eds.). Springer International Publishing, Cham, 5--19.Google Scholar
- Masaharu Yoshioka and Yasuhiro Aoki. 2023. HUKB at COLIEE 2023 Statute Law Task. In Workshop of the Tenth Competition on Legal Information Extraction/Entailment (COLIEE'2023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL).Google Scholar
- ChengXiang Zhai. 2008. Statistical language models for information retrieval. Synthesis lectures on human language technologies 1, 1 (2008), 1--141.Google Scholar
Index Terms
- Summary of the Competition on Legal Information, Extraction/Entailment (COLIEE) 2023
Recommendations
COLIEE-2018: Evaluation of the Competition on Legal Information Extraction and Entailment
New Frontiers in Artificial IntelligenceAbstractWe summarize the evaluation of the 5th Competition on Legal Information Extraction/Entailment 2018 (COLIEE-2018). The COLIEE-2018 tasks include two tasks in each of statute law and case law. The case law component includes an information retrieval ...
COLIEE 2022 Summary: Methods for Legal Document Retrieval and Entailment
New Frontiers in Artificial IntelligenceAbstractWe present a summary of the 9th Competition on Legal Information Extraction and Entailment (COLIEE 2022). The competition consists of four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), ...
COLIEE 2020: Legal Information Retrieval and Entailment with Legal Embeddings and Boosting
New Frontiers in Artificial IntelligenceAbstractIn this paper we investigate three different methods for several legal document retrieval and entailment tasks; namely, new low complexity pre-trained embeddings, specifically trained on documents in the legal domain, transformer models and ...
Comments