Pre-training and unsupervised learning enhance the model's broad-based intelligence and improve its generalization capabilities, complementing reasoning abilities that can be more variable.

Reasoning is expected to be a core capability of future models, and the two paradigms of unsupervised pre-training and specialized reasoning complement each other well.

Achieving AGI requires not just pre-training but also the ability to reason and utilize additional computational resources effectively to improve outcomes.

Pre-training is inherently broad and diverse, making it challenging to achieve the same breadth when using reinforcement learning in specific environments.

In 2025, reasoning will become the defining focus in AI development, as pre-training reaches diminishing returns.

The DeepSeek paper detailed that they used reasoning-heavy but standard post-training techniques after large-scale reasoning training.

The diminishing returns in pre-training mean that significant increases in intelligence will require exponentially more compute resources, making it harder to achieve breakthroughs.

While coding is a primary focus, we see a transfer of skills from coding models to enhancing reasoning and agentic behavior in other applications.

Structuring the reasoning in prompts helps the model perform better, even if it's not strictly reasoning as we define it.