Loading…
Data as of Oct 20, 2026Based on 7,081 AI responses
Reviewed by Dimitry Apollonsky
TensorRT Edge-LLM is NVIDIA's high-performance C++ inference runtime for running text, vision, audio, speech, and action models on edge platforms such as Jetson, DRIVE, and DGX Spark. It supports exporting Hugging Face checkpoints to ONNX for engine construction, with an experimental direct frontend that builds engines from checkpoints without ONNX. The framework provides an end-to-end edge LLM inference solution with ongoing updates, including support for new models and hardware and features like multi-turn KV-cache reuse and an experimental OpenAI-compatible server.
Hosted on GitHub
<1%
across TensorRT-Edge-LLM’s markets · since Jul 5
The market map
Edge AI Model Optimization ToolsMentioned in
Each engine’s share of its own answers across TensorRT-Edge-LLM’s markets · since Jul 5
Where TensorRT-Edge-LLM ranks in AI
