DeepSeek-R1 Local Inference Guide: From 7B to 671B (What Actually Runs)

DeepSeek-R1 is one of the most capable open-source reasoning models ever released. The 7B version runs on a gaming laptop. The 671B requires a server farm. This guide covers what each size is actually useful for — with exact commands to get them running today.

For a complete hardware comparison to find the best GPU for running DeepSeek-R1 locally, see LLMHardware.io's GPU comparison tool.

The DeepSeek-R1 Family

Model	VRAM (Q4)	Target Hardware	Best Use Case
R1-Distill-7B	5 GB	Any 8GB GPU / MacBook	Daily use, fast responses
R1-Distill-14B	10 GB	RTX 3080 / Mac Mini M4	Strong reasoning budget hardware
R1-Distill-32B	20 GB	RTX 4090 / Mac Studio	Best quality single-GPU ★
R1-70B	42 GB	2x RTX 4090 / Mac Ultra	Near-frontier reasoning
R1-671B (full)	~380 GB	Multi-GPU cluster	Frontier-level (server only)

Setup: R1-Distill-32B (The Sweet Spot)

# Via Ollama (easiest)
ollama pull deepseek-r1:32b
ollama run deepseek-r1:32b

# Via llama.cpp (5-10% faster)
huggingface-cli download bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF \
  --include "*Q4_K_M*" --local-dir ./models/deepseek-r1-32b

./build/bin/llama-cli \
  -m ./models/deepseek-r1-32b/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf \
  --n-gpu-layers 999 --ctx-size 32768 --temp 0.6

Understanding Thinking Mode

R1 models generate explicit reasoning chains inside <think>...</think> tags before answering. This is the key to their performance on hard problems:

import ollama

response = ollama.generate(
    model='deepseek-r1:32b',
    prompt='What is the time complexity of Dijkstra with a Fibonacci heap?',
    options={'temperature': 0.6, 'top_p': 0.95}
)

text = response['response']
if '</think>' in text:
    thinking = text.split('</think>')[0].replace('<think>', '').strip()
    answer = text.split('</think>')[1].strip()
    print("Reasoning:", thinking[:300])
    print("Answer:", answer)

Benchmark: R1 vs Competition (32B class)

Model	AIME Math	HumanEval	MMLU
R1-Distill-32B	72.6%	88%	87%
Qwen3.6-32B	74.1%	90%	89%
Kimi K2.5-32B	70.3%	91%	88%

DeepSeek-R1-32B remains one of the top locally-runnable reasoning models despite being over a year old. If you want a complete hardware build guide for running it, check LLMHardware.io's recommended builds.