MAP THE COMPUTE BEHIND THE MODEL.

F1 Model weights

weights_GB = params × bytes_per_param ÷ 10⁹

70B at FP16 → 70 × 10⁹ × 2 B ≈ 140 GB. Mixture-of-experts models keep every expert resident, so total (not active) parameters count.

F2 KV cache

kv_GB = layers × 2 × kv_heads × head_dim × context × batch × kv_bytes ÷ 10⁹

Uses each preset's published architecture (grouped-query attention included). The cache is 16-bit for FP16/BF16/INT8/INT4 weights, 8-bit for FP8 and 32-bit for FP32. DeepSeek V3 uses its compressed MLA latent (576 values per token per layer). Custom models get an estimated shape with 8 KV heads.

F3 Inference total

vram = weights + kv_cache + 0.10 × weights

The 10% runtime overhead stands in for the CUDA context, kernel workspaces and allocator fragmentation.

F4 Fine-tuning (LoRA / QLoRA)

vram = frozen_weights + 0.005 × params × 16 B + activations, + 10%

Adapters train ~0.5% of parameters (≈ rank 16 on all linear layers) with FP32 weights, gradients and Adam states. INT8/INT4 bases correspond to QLoRA-style setups.

F5 Full training

vram = params × (w + g + 12) B + activations, + 10%

Mixed-precision Adam: 16-bit weights and gradients plus FP32 master weights and two moments (16 B/param for BF16). No ZeRO/FSDP sharding is assumed. INT8/INT4 training is estimated as BF16.

F6 Activations

act = s·b·h·(2·L + 34) + s·b·vocab·4 bytes

Assumes gradient checkpointing and fused attention: each layer keeps its 16-bit input and one layer is recomputed in full, plus FP32 logits.

F7 GPU recommendation

count = round_up(vram ÷ (gpu_vram × 0.90)) → 1, 2, 4, 8, then 8-GPU nodes

A GPU is usable up to 90% of its memory, keeping ≥10% headroom. The smallest count wins, then the least total VRAM, then the higher bandwidth. Consumer and workstation cards are limited to one 8-GPU node.

F8 Decode ceiling

tok/s ≤ (count × bandwidth) ÷ active_weight_bytes

A single-stream, memory-bandwidth-bound upper limit. Real throughput is lower and depends on kernels, parallelism and batching.