How AI describes InfLLM

InfLLM is a training-free memory-based method that enables large language models (LLMs) to process extremely long sequences (up to 1,024K tokens) while capturing long-distance dependencies. It stores distant contexts in memory units and retrieves relevant ones for attention computation, allowing LLMs pre-trained on short sequences to outperform baselines that are continually trained on long sequences.