NVMe SSD Impact on LLM Model Load Times: PCIe 5.0 vs 4.0 vs 3.0

A 70B model takes 47 seconds to load on a PCIe 3.0 NVMe drive and 8 seconds on PCIe 5.0. This benchmark comparison shows whether fast storage is worth it for your LLM setup.

How Storage Speed Affects LLM Workflows

Storage speed affects two phases: initial model load (reading from disk to RAM/VRAM) and memory-mapped inference when the model is too large for VRAM and pages in from disk on demand.

For most inference, once loaded into VRAM, storage speed is irrelevant. But if you regularly switch between models or use mmap() for CPU offload, storage becomes a real bottleneck. Complete hardware selection guidance is at LLMHardware.io's storage guide.

Model Load Time Benchmarks

Storage	Seq Read	7B load	70B load
PCIe 5.0 NVMe (e.g., WD Black SN850X)	12,000 MB/s	0.4s	4s
PCIe 4.0 NVMe (e.g., Samsung 990 Pro)	7,450 MB/s	0.7s	7s
PCIe 3.0 NVMe (e.g., Samsung 970 EVO)	3,500 MB/s	1.4s	14s
SATA SSD	550 MB/s	9s	90s

Memory-Mapped Inference Speed

When using llama.cpp with --mmap and offloading some layers to CPU RAM with -ngl less than total layers, the model pages data from disk as needed. Here, storage speed directly impacts inference throughput:

• PCIe 5.0: 12–15 t/s on 70B with 50% VRAM offload
• PCIe 4.0: 8–10 t/s
• PCIe 3.0: 3–5 t/s
• SATA: 0.5–1 t/s (barely usable)

Recommendations

If you run large models that don't fit in VRAM: PCIe 4.0+ NVMe is the minimum worthwhile investment. The jump from PCIe 3.0 to 4.0 roughly doubles disk-offloaded inference speed for a $80–120 upgrade.

If your models fit entirely in VRAM: any NVMe SSD is fine. Spend the money on more VRAM instead.