Running language models and image generators on your own PC has gone mainstream, and the first question builders ask is how much memory it takes. The honest answer is that model size, quantization, and whether you lean on your GPU all change the math dramatically. Here is a practical framework so you can size a build without overspending.
Two kinds of memory matter
Local AI leans on two pools: system RAM (your DDR4 or DDR5 sticks) and GPU VRAM. If a model fits entirely in VRAM, it runs fast. If it spills into system RAM, it still works but slows down. That is why AI builders care about both numbers, and why 32GB of system RAM plus a healthy GPU is a common sweet spot.
How quantization shrinks the requirement
Models are described by parameter count (7B, 13B, 70B) and precision. A raw 16-bit model needs about 2 bytes per parameter, but 4-bit quantization cuts that to roughly half a byte, making large models fit on consumer hardware with only a modest quality trade-off. The table shows rough memory needs for a quantized 4-bit language model.
| Model size | ~VRAM (4-bit) | Min system RAM | Comfortable RAM |
|---|---|---|---|
| 7B | ~5-6 GB | 16 GB | 32 GB |
| 13B | ~9-10 GB | 16 GB | 32 GB |
| 34B | ~20-22 GB | 32 GB | 48-64 GB |
| 70B | ~40-45 GB | 64 GB | 64-128 GB |
CPU-only versus GPU-accelerated
If you have no capable GPU, models run on the CPU using system RAM alone. This works but is far slower, and now the RAM figure has to hold the entire model plus overhead. A 13B model in 4-bit needs roughly 10GB just for weights, so 16GB total is tight once Windows and apps are loaded, and 32GB is the practical minimum for a smooth experience.
Memory bandwidth is the hidden factor
For CPU inference, bandwidth often matters more than capacity. Dual-channel DDR5-6000 moves data far faster than single-channel DDR4-3200, and token generation speed scales with it. Always populate two channels, and favor faster kits if you plan to run models on the CPU.
Practical build tiers
- Dabbler: 32GB DDR5 + 8-12GB VRAM GPU runs 7B-13B models comfortably.
- Enthusiast: 64GB DDR5 + 16-24GB VRAM handles 34B models and light fine-tuning.
- Power user: 128GB + 24GB VRAM (or multiple GPUs) for 70B-class models and larger context windows.
Context length and overhead add up
The model weights are only part of the memory picture. Running inference also consumes memory for the context window, the chunk of conversation or prompt the model holds in mind, and larger context windows can add several gigabytes on top of the base model. Longer contexts, multiple simultaneous requests, and higher-precision math all push the requirement up. That is why a model that technically fits in your VRAM can still run out of memory once you feed it a long document. When sizing a build, leave headroom above the raw model size, typically a few extra gigabytes, so you can use generous context windows and keep other applications open without forcing the model to spill onto slower storage.
FAQ
Is 16GB of RAM enough for local AI?
It can run small 7B models, especially if most of the work happens in GPU VRAM, but 16GB fills quickly once the operating system, browser, and model overhead are added. For anything beyond casual experimentation, 32GB is the realistic floor.
Does more system RAM speed up a GPU-based model?
Not directly, if the model already fits in VRAM. Extra system RAM helps when a model is too large for the GPU and must offload layers to the CPU, or when you run large context windows and multiple applications at once.
Bottom line
For most people getting started, 32GB of dual-channel memory paired with a GPU that has 8-16GB of VRAM covers 7B to 13B models nicely. Step up to 64GB for 34B models and 128GB for 70B-class work. Quantization is your best friend, and if you run on the CPU, prioritize memory bandwidth and always fill both channels.
