How AI describes OmniQuant

OmniQuant is a quantization technique for large language models that supports weight-only and weight-activation quantization formats such as W4A16, W3A16, W6A6, and W4A4. It provides pre-trained quantization parameters for model families including LLaMA, OPT, Falcon, and Mixtral, enabling efficient compression and inference on GPUs and mobile devices.