What AI says about ScalingEval

ScaleEval is an agent-debate-assisted framework for scalable meta-evaluation of LLMs used as evaluators, leveraging multiple communicative LLM agents to enable multi-round discussions. It lets users submit LLM outputs, evaluation criteria, and scenarios to conduct meta-evaluation and identify the best evaluator. It is distributed as a Python package (pip install scaleeval) with example criteria such as metaeval_creativity, metaeval_helpfulness, metaeval_interpretability, and metaeval_reasoning.