Runtime

TensorRT-LLM

NVIDIA optimized inference

Platforms

Linux

Why it matters

Runtime changes fit

Memory behavior, quantization support and throughput vary by runtime.

Evidence

Official/project docs

Inspect source ↗