Even small models can boost their performance in tasks like math with a small amount of reinforcement learning training.

Reinforcement learning methods have not yet become mainstream in AI because we need to address the challenge of translating human goals into measurable metrics.

Reinforcement learning from verifiable rewards has significantly improved AI capabilities, especially in software engineering, where tasks can be verified easily through tests and outputs.

We've shown that we can reach high levels of intellectual complexity in tasks, but we haven't yet demonstrated long-running agentic performance in AI systems.

The challenge of achieving truth with large language models is a contentious topic in the industry, especially in domains where definitive answers exist, like math and coding.

Our focus is on optimizing for reasoning and behavior of models rather than just their scores on traditional benchmarks like MMLU.

We started building Deep Research over a year ago, driven by progress in reinforcement learning and reasoning models, initially focusing on math, science, and coding tasks.

While RLHF systems are powerful, they lack the ability to go beyond human knowledge, as they depend on human feedback which may not recognize new ideas or better actions.

AI systems like DeepMind's AlphaProof are trained using reinforcement learning on both successful and failed formal proofs, demonstrating the potential for AI to assist in solving high-level mathematical problems.