LLM Power Consumption Guide: What Your AI Habit Really Costs

Running local LLMs has real electricity costs that add up over time. An RTX 4090 drawing 450W during inference, running 8 hours a day, costs roughly $50-60/month depending on your electricity rate. Understanding power consumption helps you choose hardware that balances capability with operating costs.

Rule of thumb: GPU TDP during LLM inference runs at 70-100% of rated TDP. Budget accordingly. The LLMHardware.io power calculator shows annual electricity costs for every major GPU at your local rate.

Power Draw by GPU Class

Budget GPUs (RTX 4060, Arc B580): 60-120W during inference. Mid-range (RTX 3080 10GB, RX 6800 XT): 180-250W. High-end (RTX 4090): 300-450W. Apple Silicon is exceptional: an M4 Max Mac Studio with 128GB unified memory draws only 35-50W while running 70B models that would require a $3000+ dual-GPU NVIDIA setup drawing 600W+.

Efficiency: Tokens Per Watt

The better metric for long-running deployments is tokens per watt. Apple M3/M4 chips consistently lead here, producing 2-4 tokens/watt on 7B models. The RTX 4090 produces roughly 0.8-1.2 tokens/watt on the same models. For a home server running Mistral 7B all day, an M2/M3 Mac Mini uses ~80% less power than an RTX 3080 build for similar output speeds.

Reducing Power Without Losing Performance

On NVIDIA GPUs, set power limits via nvidia-smi -pl [watts]. An RTX 3090 at 250W instead of 350W loses only ~8% inference speed while cutting power draw by 30%. Enable PCIe Gen 3 instead of Gen 4 if your workload doesn't benefit from bandwidth. Run smaller quantized models when response speed matters less than throughput.