NKB Gaming PC Builds
Light
NKB Gaming PC Builds
Deterministic calculation engine for local AI model deployment (Ollama, vLLM, llama.cpp). Simulate VRAM footprint, DeepSeek Multi-Head Latent Attention (MLA) vs GQA KV cache allocation, tokens/second generation speed, and required ATX 3.1 power supply excursions.
Running DeepSeek-R1 70B (Q4_K_M) locally requires 43.2 GB of VRAM with an 8k context window. A single 24GB or 32GB GPU will trigger an Out-of-Memory (CUDA OOM) failure; smooth inference at 18.5 tokens/sec mandates dual 24GB GPUs (e.g. 2ร RTX 3090 / 4090) or a single RTX 5090 32GB with CPU offload. Dual GPU peak electrical draw hits 820W continuous and 1,350W during 100ยตs transient current spikes, mandating a certified 1200W ATX 3.1 / PCIe 5.1 power supply.
Adjust model parameters, quantization, context, KV cache precision, and GPU hardware to test real-time fit.
Zero CPU RAM offloading. Maximum generation throughput.
Deterministic engineering benchmarks for top local AI model architectures. Verified against Ollama v0.5+ and llama.cpp b4100+ on NVIDIA CUDA 12.8 and AMD ROCm 6.2.
| Model Architecture | Quantization | Context | Required VRAM | Recommended GPU Setup | Tokens/Sec | Min. ATX 3.1 PSU |
|---|---|---|---|---|---|---|
| DeepSeek-R1 70B (Distilled) | Q4_K_M (4.50 bpw) | 8,192 (GQA) | 43.2 GB | Dual RTX 3090 24GB / Dual RTX 4090 | 18.5 t/s | 1200W ATX 3.1 (Dual 12V-2x6) |
| DeepSeek-R1 70B (Distilled) | Q8_0 (8.50 bpw) | 8,192 (GQA) | 76.5 GB | Triple RTX 3090 24GB / Dual RTX 6000 Ada | 11.2 t/s | 1600W ATX 3.1 |
| DeepSeek-R1 32B (Distilled) | Q4_K_M (4.50 bpw) | 16,384 (GQA) | 20.8 GB | Single RTX 3090 24GB / RTX 4090 / RTX 5090 | 38.0 t/s | 850W - 1000W ATX 3.1 |
| DeepSeek-R1 32B (Distilled) | Q8_0 (8.50 bpw) | 8,192 (GQA) | 34.6 GB | Single RTX 5090 32GB / Dual 16GB GPUs | 26.5 t/s | 1000W ATX 3.1 |
| DeepSeek-R1 14B (Distilled) | Q4_K_M (4.50 bpw) | 32,768 (GQA) | 10.4 GB | Single RTX 4070 12GB / RTX 5070 12GB | 54.0 t/s | 650W - 750W |
| DeepSeek-R1 14B (Distilled) | Q8_0 (8.50 bpw) | 16,384 (GQA) | 16.2 GB | Single RTX 4070 Ti Super 16GB / RTX 5080 | 42.5 t/s | 850W ATX 3.1 |
| DeepSeek-R1 7B / 8B (Distilled) | Q4_K_M (4.50 bpw) | 32,768 (GQA) | 5.8 GB | Single RTX 4060 8GB / RTX 3060 12GB | 72.0 t/s | 550W |
| DeepSeek-R1 7B / 8B (Distilled) | Q8_0 (8.50 bpw) | 32,768 (GQA) | 8.9 GB | Single RTX 3060 12GB / RTX 4070 12GB | 60.5 t/s | 650W |
| DeepSeek-R1 Full 671B (MoE) | Q4_K_M (4.50 bpw) | 32,768 (MLA) | 384.0 GB | 8x RTX 3090 24GB / Mac Studio 192GB x2 | 14.0 t/s | 2x 1600W ATX 3.1 (Multi-PSU) |
| Llama 3.3 70B Instruct | Q4_K_M (4.50 bpw) | 8,192 (GQA) | 42.8 GB | Dual RTX 3090 24GB / Dual RTX 4090 | 19.2 t/s | 1200W ATX 3.1 |
| Llama 3.3 70B Instruct | Q8_0 (8.50 bpw) | 8,192 (GQA) | 75.8 GB | Dual RTX 6000 Ada 48GB / 4x RTX 3090 | 12.0 t/s | 1600W ATX 3.1 |
| Qwen 2.5 Coder 32B | Q4_K_M (4.50 bpw) | 16,384 (GQA) | 21.2 GB | Single RTX 3090 24GB / RTX 4090 / RTX 5090 | 36.8 t/s | 850W - 1000W |
| Qwen 2.5 Coder 14B | Q8_0 (8.50 bpw) | 16,384 (GQA) | 16.5 GB | Single RTX 4070 Ti Super 16GB / RTX 5080 | 43.1 t/s | 850W ATX 3.1 |
| Mistral Large 2 123B | Q4_K_M (4.50 bpw) | 8,192 (GQA) | 74.2 GB | Triple RTX 3090 24GB / Dual RTX 6000 Ada | 10.5 t/s | 1600W ATX 3.1 |
| Command R+ 104B | Q4_K_M (4.50 bpw) | 8,192 (GQA) | 62.8 GB | Triple RTX 3090 24GB / Mac Studio 128GB | 12.8 t/s | 1500W ATX 3.1 |
Mandatory power foundation for dual RTX 3090 or RTX 5090 workstations. Absorbs 1,400W 100ยตs transient current spikes without triggering OCP shutdowns.
Ideal for single RTX 5090 32GB or RTX 4090 running DeepSeek-R1 32B with heavy CPU token preprocessing.
Essential system memory cushion for local LLM CPU offloading when model weights exceed dedicated VRAM buffers.
Enables dual high-power GPU configurations on motherboards with tight 3-slot spacing to ensure optimal cooling airflow.
Essential engineering answers on VRAM allocation, KV cache compression, ATX 3.1 power excursions, and inference speed.
No, not entirely in VRAM. At 4-bit quantization (Q4_K_M), the model weights alone consume 40.2 GB, plus 2โ3 GB for the KV cache at 8k context, totaling ~43.2 GB. A single 24GB GPU will encounter a CUDA Out-of-Memory (OOM) error unless offloaded to system RAM via llama.cpp or Ollama. CPU offloading drops generation speed from ~18.5 tokens/sec to approximately 1.5 tokens/sec.
DeepSeek-V3 and DeepSeek-R1 full models utilize Multi-Head Latent Attention (MLA). MLA compresses the key-value cache into a low-dimensional latent vector (512 dimensions) instead of caching full multi-head states. As a result, DeepSeek MLA requires roughly 85% less KV cache VRAM than Llama 3.3 GQA at the same context length, enabling 64k+ context windows with minimal memory overhead.
The undisputed price-to-performance sweet spot is dual refurbished or used NVIDIA GeForce RTX 3090 24GB cards (costing roughly $1,400 to $1,600 total). This provides a combined 48GB VRAM pool with 1,872 GB/s aggregate memory bandwidth, achieving 18โ20 tokens/sec in Ollama with zero CPU offloading.
During initial inference prompt ingestion and batch computation, dual high-end GPUs generate instantaneous 100-microsecond transient power spikes reaching 1,300W to 1,500W. Older ATX 2.x power supplies interpret these spikes as dead short-circuits and trip Over-Current Protection (OCP), causing sudden PC reboots. ATX 3.1 power supplies are engineered to sustain 200% power excursions safely.
The KV cache memory consumption follows the formula: 2 ร layers ร heads ร head_dim ร context_length ร bytes_per_element. In standard FP16 precision, an 8,192 context window adds ~2.6 GB of VRAM. Enabling 8-bit KV cache quantization (--cache-type-k q8_0 --cache-type-v q8_0 in llama.cpp) cuts KV cache memory consumption in half with negligible perplexity degradation.
During text generation (decoding phase at batch size 1), LLMs are strictly memory-bandwidth bound rather than compute-bound. The GPU must read every model weight from VRAM to generate a single token. Generation speed in tokens/second roughly equals GPU Memory Bandwidth (GB/s) divided by the Quantized Model Size (GB) multiplied by memory bus efficiency (~0.80).
