Tool
LLM VRAM estimator: weights, KV cache and overhead
Choose a catalogued model to use its real attention layout from config.json, or enter your own. The result is an estimate in three parts, because a single number would hide where the memory goes.
Estimated memory
55.2 GiB
Rough fit
Compared with the total only. Usable memory is lower than the card size (driver, display, other processes), and runtimes can offload layers to system RAM at a speed cost.
Formula with your numbers
weights = 70.55B × 4.8944 bits / 8 = 40.2 GiB
KV cache = 2 × layers × kv_heads × head_dim × 2 bytes × tokens × 1 sequence(s)
= 327,680 bytes per token in full-attention layers
overhead = 10% of weights and cacheAttention layout from huihui-ai/Llama-3.3-70B-Instruct-abliterated config.json.
What the estimate includes
Weights are parameters times bits per weight. The bits for each GGUF type are llama.cpp's own measurements for Llama-3.1-8B, from its quantize documentation; other models land close to these but not on them, because different tensors are quantized differently. For an exact figure, add up the file sizes listed for the quantization you download.
KV cache stores keys and values for every token in context, for every attention layer: 2 × layers × KV heads × head dimension × bytes per value, per token. Models with grouped-query attention have few KV heads, so their cache is far smaller than their hidden size suggests. The presets read the real layout from each model's config.json:
- Sliding-window layers (Gemma, gpt-oss) cache at most their window, so long contexts cost much less than the simple formula suggests.
- Linear-attention layers (Qwen3.8's Gated DeltaNet) keep a small fixed state instead of a growing cache; only its 16 full-attention layers are counted.
- Latent or compressed attention (GLM-4.7-Flash, GLM-5.3-Flash) stores a compressed cache that this formula would overestimate, so the estimator leaves KV out and says so instead of guessing.
Overhead covers the runtime's working buffers and activations. It varies with the runtime, batch size and settings; 10% is a starting assumption, not a measurement. Mixture-of-experts models need memory for every expert even though each token uses a few, so use total parameters, not active ones.
Related: GGUF quantization explained, models you can run locally, and the context window planner.