What AI says about LawBench

LawBench is a benchmarking suite that evaluates large language models’ legal knowledge and reasoning within the Chinese legal system, designed around three cognitive dimensions across 20 tasks. It provides a dataset of 20 tasks with 500 examples each, covering legal memory, understanding, and application (e.g., statute recall, QA, document proofreading, NER, reading comprehension, and argument mining) and introduces an abstention rate metric to measure refusals or misinterpretations. The project offers open data, a paper, and a public repository, and has benchmarked dozens of models (51) across multilingual and legal-specialized LLMs, serving as a standard for legal NLP evaluation.