Deepseek’s 7 key advances in LLMs.

DeepSeek deserves a closer look for a specific reason. The team claims to have produced a model up to 20 times cheaper than OpenAI, while maintaining comparable performance on several benchmarks. Understanding the gap requires examining the training choices, the role of RL and the work done on distilled models.

1. The return of direct RL

DeepSeek puts Reinforcement Learning (RL) back at the centre of training, with a less conventional approach. The team applies RL directly to the base model, without starting with expensive supervised training or conventional fine-tuning beforehand.

👉 As a result, training uses fewer resources and progresses faster. Iterations can improve the model without starting from scratch each time. It is an approach that is easier to understand operationally, because it reduces some of the burdensome steps involved in training an LLM.

2. Closely linked RL and Chain-of-Thought

DeepSeek also combines RL with reasoning chains, known as Chain-of-Thought (CoT). The model learns to work through reasoning step by step, which matters for complex tasks such as mathematics, coding or scientific problems.

👉 As a concrete example, on the MATH-500 benchmark, DeepSeek-R1 achieves 97,3 %, a score that exceeds most other models.

3. Smaller models through distillation

DeepSeek has worked on distilling the capabilities of large models into lighter versions. The team takes what works in the main model and transfers it to smaller models.

👉 As a result, distilled versions such as DeepSeek-R1-Distill-Qwen-7B outperform some larger models while consuming fewer resources.

👉 For applications constrained by cost or computing capacity, the benefit is direct. A lighter model can run in more settings, with a lower bill and fewer infrastructure constraints.

4. More readable responses

DeepSeek also places considerable emphasis on the readability of responses. During a Cold Start phase, the team fine-tunes the model on clean, structured data. The goal is to obtain clear, better-organised responses, without mixing languages or producing confusing content.

👉 This aspect is then reinforced through rewards during RL. The model is steered towards responses closer to what humans prefer to read, rather than producing only technically correct output.

5. Self-evolving reasoning and aha moments

During training, DeepSeek observes interesting behaviours. The model learns to extend its reasoning, correct its own errors and then try other strategies to solve a problem.

👉 The team calls these behaviours aha moments. They show that RL can push a model to evolve autonomously, without direct human intervention at every correction step.

6. Performance close to OpenAI’s models

DeepSeek-R1, at a much lower cost, competes with leading OpenAI models such as o1-1217 across several categories.

• Mathematics and scientific reasoning, with similar or higher scores.
• Coding, with very strong performance in competitions such as Codeforces.
• Writing and creative tasks, with good results on benchmarks such as AlpacaEval and ArenaHard.

What stands out here is the relationship between the claimed cost and measured performance. DeepSeek does not rely on a single score. The model performs well across several groups of tasks, making the comparison with OpenAI’s models more interesting.

7. A multistage training pipeline

To achieve these results, DeepSeek uses a structured training pipeline.

Cold Start, with initial fine-tuning on clean, structured data.

Targeted RL, to improve logical reasoning capabilities.

Rejection Sampling, collecting new data while discarding bad responses.

Global RL, to align the model with human preferences across a wide variety of scenarios.

👉 This method improves performance step by step, while maintaining a more efficient and less costly approach.

DeepSeek shows that a model can cost far less while remaining competitive with OpenAI’s best models. Direct RL, distillation into smaller models and pipeline optimisation explain much of the gap. The technical details are documented in the paper published on arXiv.