Tools

Search

Explore

Videos Channels Figures

Atmrix

About

Tools

Search

Explore

Videos Channels Figures

Atmrix

About

GT

Garry Tan

01/24/25

@ Y Combinator

Training larger models requires substantial computing power, which translates to more GPUs running for extended periods and consuming increasing amounts of energy.

Video

YC

How Scaling Laws Will Determine AI's Future | YC Decoded

@ Y Combinator

How Scaling Laws Will Determine AI's Future | YC Decoded

01/24/25

Related Takeaways

LF

Lex Fridman Cast

02/03/25

@ Lex Fridman

The largest AI models, like GPT-4, require massive GPU clusters, with power consumption reaching unprecedented levels, highlighting the increasing scale of AI infrastructure.

LF

Lex Fridman Cast

02/03/25

@ Lex Fridman

DeepSeek's training run with 2,000 GPUs demonstrates that while they can still train models effectively, strong export controls could limit their ability to scale further.

YC

Y Combinator Cast

11/01/24

@ Y Combinator

The scaling law in AI suggests that as models grow larger, they will require significantly more computational resources, making them more capital intensive to develop.

O

OpenAI Cast

04/11/25

@ OpenAI

Training large models is challenging because new issues arise that weren't anticipated at the project's inception, requiring ongoing adjustments and problem-solving.

YC

Y Combinator Cast

11/01/24

@ Y Combinator

AI advancements are increasing GPU requirements, as the computational needs for inference are growing significantly.

JD

Jeff Dean

05/12/25

@ Sequoia Capital

The scaling of compute resources will vary significantly depending on the problem, with some requiring up to 10,000 times more compute than others, which allows for more capable models.

JD

Jeff Dean

05/12/25

@ Sequoia Capital

The future of computing will involve running incredibly large neural networks with high performance and low power consumption, requiring different solutions for training and inference workloads.

YC

Y Combinator Cast

01/24/25

@ Y Combinator

By allowing models to think longer, large language models can leverage more compute on the fly, enhancing their intelligence for complex problems.

YC

Y Combinator Cast

01/24/25

@ Y Combinator

Instead of just scaling up model size during training, researchers are likely to focus on scaling the compute available for the model's Chain of Thought.