How AI describes FlexLLM

FlexLLM is a system that co-serves LLM inference and PEFT-based finetuning on shared GPUs at the token level. It uses static compilation optimizations and a hybrid token scheduler to reduce memory requirements by up to 80% while maintaining strict latency SLOs and improving finetuning throughput by up to 6.8×.