How AI describes SmoothQuant

SmoothQuant is a training-free, accuracy-preserving post-training quantization solution that enables 8-bit weight and 8-bit activation (W8A8) quantization for large language models. It smooths activation outliers by migrating quantization difficulty from activations to weights, achieving up to 1.56x speedup and 2x memory reduction with negligible accuracy loss.