Loading…
OpenJudge is an open-source evaluation framework for AI applications (such as agents and chatbots) designed to evaluate quality and drive continuous optimization. It provides a library of 50+ ready-to-use graders across agent, text, code, math, and multimodal tasks, with tools to generate rubrics (zero-shot or data-driven) and to train dedicated judge models, plus benchmark data and pytest validation. It integrates with popular observability and training platforms (LangSmith, Langfuse, VERL) and supports end-to-end evaluation from data collection to reward signaling for training.
AI named OpenJudge in March 2026.