The potential of 10 trillion parameter models could unlock new capabilities, similar to how nuclear power and the Fourier transform revolutionized technology and science.

Creating a giant model like GPT-4.5 requires a lot of people, time, and computational resources.

Hassabis argues that while smaller, specialized models can be effective, the development of large, general models is essential for achieving significant advancements in AI capabilities. He also mentions that larger models are essential for training the smaller, more efficient models that we use extensively.

DeepSeek's V3 model utilizes a mixture of experts architecture, activating only 37 billion out of 671 billion parameters for each token prediction, significantly reducing computation compared to models like Llama 3, which activates all 405 billion parameters.

The ability to distill models from larger ones will be valuable, as companies will likely have AIs tailored for every function they need.

With the advent of 10 trillion parameter models, we could unlock unprecedented scientific discoveries, potentially leading to breakthroughs like room temperature fusion and flying cars.

In the future, we will see a proliferation of different AI models, with companies using the largest models for performance optimization.

I believe we are nearing the upper limits of large language models, as the future will likely involve many smaller, specialized models rather than a few large ones. We're at the upper end of large language models, and they're rapidly becoming a commodity as they mature.

Larger models have more parameters, which are the internal values adjusted during training to enhance prediction accuracy, and they are trained on significantly more data, often measured in tokens.