Memory
Check VRAM, system RAM or unified memory against the complete runtime profile.
Private AI guide
Local AI by Hardware
Parameter count is not a hardware compatibility test. A useful local-AI recommendation must account for model format, quantization, context, KV cache, runtime overhead, CPU offload, modality and the workload you expect to run.
Decision context
Start with the boundary, not the brand.
Choosing a realistic local AI stack for an existing workstation, GPU server, CPU-only system or Apple Silicon machine.
Quantization
Model precision and quantization materially change memory requirements and performance trade-offs.
Context
Longer context increases memory use, including KV cache requirements.
Runtime & workload
Runtime overhead, CPU offload, modality and concurrency can change whether a configuration is practical.
Common failure modes
What not to assume
Inferring compatibility from parameter count alone.
Treating advertised maximum context as free from memory cost.
Ignoring runtime-specific overhead or assuming all quantizations behave identically.
Hard constraints win: mandatory privacy, platform, hardware or license requirements exclude incompatible configurations instead of merely lowering their rank.
Verified baselines
Reference configurations related to this guide
These configurations are evidence-backed launch references. They illustrate different privacy and hardware trade-offs; they are not universal recommendations.
Developer — Local Coding — RTX 4060 8 GB
Continue + Ollama + Qwen2.5-Coder-7B-Instruct Q4_K_M
GOOD_WITH_BOUNDED_CONTEXTfully-local
Consultant — Private RAG — Apple 32 GB
AnythingLLM + Ollama + Qwen3-8B
GOODfully-local
Developer — Agentic Coding — RTX 4090
OpenHands + LM Studio local server + Qwen3.6-35B-A3B
GOOD_QUANTIZEDlocal-network
Professional context
Apply the architecture to your work
Continue the decision
Use the evidence-backed catalog
Hardware profiles
Continue with STACKELIER data, reference configurations and deterministic fit logic.
Models directory
Continue with STACKELIER data, reference configurations and deterministic fit logic.
Runtimes directory
Continue with STACKELIER data, reference configurations and deterministic fit logic.
FAQ
Key questions
How does STACKELIER classify hardware fit?
Hardware profiles use states such as EXCELLENT, GOOD, LIMITED, POOR, INCOMPATIBLE and UNKNOWN based on the available evidence for the complete configuration.
Is VRAM the only important number?
No. RAM, unified memory, context, KV cache, runtime overhead, CPU offload, modality and concurrency can all affect practical compatibility.
Can the same model have different hardware fit with another runtime or quantization?
Yes. STACKELIER treats model, runtime, quantization and context as part of the hardware requirement profile rather than assigning one universal requirement to a model name.