Runtime
LocalAI
self-hosted OpenAI-compatible local inference
Runtime changes fit
Memory behavior, quantization support and throughput vary by runtime.
self-hosted OpenAI-compatible local inference
Memory behavior, quantization support and throughput vary by runtime.