Loading…
Inference Perf is a production-scale GenAI inference performance benchmarking tool that is agnostic of model servers. It measures latency, throughput, and goodput metrics to enable apples-to-apples comparison of inference deployments.
AI named inference-perf in March 2026.