DeepSeek-V3: Reading the $5.6M Training Bill Closely
What the DeepSeek-V3 paper actually claims about training cost and benchmarks, what the famous $5.6M leaves out, and why the enterprise story is inference cost, not training cost.
When DeepSeek-V3 shipped at the end of December 2024, the number that travelled fastest was the price tag: roughly $5.6 million to train a frontier-class model. Close behind came the claim that it matches GPT-4o. Both statements are true in a narrow sense and misleading in a broader one, so it is worth reading what the paper (arXiv:2412.19437) actually says.
What the $5.6M covers
The paper reports 2.788 million H800 GPU-hours for the final training run, costed at $2 per GPU-hour. That is where $5.576M comes from. The load-bearing word is final. The figure excludes the architecture research, the ablations, the failed runs, and everything learned while building DeepSeek-V2. Read it as the price of the last step, not the price of the staircase.
With that caveat, it is still a striking number, and the paper is unusually open about where it came from. Two levers did most of the work. The first is mixture-of-experts sparsity: 671 billion parameters in total, 37 billion active per token, so the model stores a lot and computes a little on each forward pass. The second is FP8 mixed-precision training, which cuts the cost of most of the compute without, in their runs, the accuracy tax that usually comes with it. Add 14.8 trillion training tokens and careful pipeline engineering, and the bill lands where it landed.
What "GPT-4o level" means in the paper's own table
The paper benchmarks against GPT-4o and Claude 3.5 Sonnet, not the older GPT-4 that most headlines said. Its own numbers:
| Benchmark | DeepSeek-V3 | GPT-4o |
|---|---|---|
| HumanEval | 82.6 | 90.2 |
| MBPP | 80.6 | 84.6 |
| MATH-500 | 90.2 | 74.6 |
Read plainly, that is not "matches GPT-4o". It is "beats GPT-4o at maths, loses at code". Taken across the whole table the model is genuinely competitive with the best models of its moment, which is remarkable for the budget. But anyone quoting a single headline number from it is selling something.
One hedge worth keeping: these are the vendor's own numbers, on the vendor's own choice of benchmarks. We have not seen an independent reproduction that changes the picture much, but a paper's table is a claim, not a measurement.
The enterprise lesson is on the inference side
Almost nobody is going to train a V3-class model, so the training bill is not really the enterprise story. The story is inference. A 671B-parameter MoE that activates 37B per token serves at roughly the compute cost of a 37B dense model while carrying 671B of parameters in memory. If you run models for a living, sparsity is the lever that decides your GPU bill, and V3 made that argument in public, with receipts, more forcefully than anyone had.
That is the part of this story we would steal. Not the price tag: the leverage.
Source: DeepSeek-V3 Technical Report, arXiv:2412.19437 (December 2024). Benchmark figures above are the paper's own.