Archived runtime
ExLlamaV2
quantized NVIDIA GPU inference
Runtime status: ARCHIVED. Official repository states that ExLlamaV2 is archived and development continues on ExLlamaV3. Continue with ExLlamaV3 →
Runtime changes fit
Memory behavior, quantization support and throughput vary by runtime.