Back to devhandbook.io
Free dev tool

GPU VRAM Calculator

Which quantized model actually fits in your card — and how fast will it run? Pick your GPU and a model to find out.

Not listed? Custom:
How this works. Each quantization level has a known size in bytes-per-parameter (GGUF): FP16 ≈ 2.0, Q8 ≈ 1.06, Q6 ≈ 0.82, Q5 ≈ 0.71, Q4 ≈ 0.61, Q3 ≈ 0.49, Q2 ≈ 0.37. We multiply that by the model's parameter count, reserve ~2 GB for OS/display/context on a discrete GPU, and check fit against your VRAM. Tokens/sec is a memory-bandwidth heuristic (generation is bandwidth-bound): t/s ≈ bandwidth ÷ model size in VRAM. These are estimates, not guarantees — actual numbers shift with context length, batch size, and drivers.

Quantization size reference (bytes per parameter)

Quant Bytes/param 70B model ≈ Quality vs FP16
FP16 / BF162.00~140 GBReference (lossless)
Q8_01.06~74 GBNear-lossless
Q6_K0.82~57 GBExcellent
Q5_K_M0.71~50 GBVery good
Q4_K_M0.61~43 GBGood — the sweet spot
Q3_K_M0.49~34 GBNoticeable degradation
Q2_K0.37~26 GBSignificant loss — last resort