Tools

Search

Explore

Videos Channels Figures

Atmrix

About

Tools

Search

Explore

Videos Channels Figures

Atmrix

About

JD

Jeff Dean

08/08/17

@ Y Combinator

The frequency of retraining models varies by domain; stable domains like vision require less frequent updates compared to rapidly changing domains like ad prediction.

Video

YC

Jeff Dean’s Lecture for YC AI

@ Y Combinator

08/08/17

Related Takeaways

YC

Y Combinator Cast

09/22/23

@ Y Combinator

Fine-tuning AI models on new types of data is crucial to maintain accuracy, especially as trends and terminology evolve rapidly.

O

OpenAI Cast

04/11/25

@ OpenAI

Training large models is challenging because new issues arise that weren't anticipated at the project's inception, requiring ongoing adjustments and problem-solving.

YC

Y Combinator Cast

02/09/24

@ Y Combinator

The trend of purpose-trained models that are smaller and customized for specific domains is gaining traction, as they can outperform general models in targeted applications.

YC

Y Combinator Cast

09/22/23

@ Y Combinator

Solutions may change over time due to evolving data and model quality, necessitating ongoing iteration and adaptation.

LF

Lex Fridman Cast

02/03/25

@ Lex Fridman

Even small models can boost their performance in tasks like math with a small amount of reinforcement learning training.

SB

Sergey Brin

05/24/25

@ Google for Developers

The shift in model training is moving from a focus on pretraining to a greater emphasis on post-training, which is becoming a more substantial part of the overall training process.

GT

Garry Tan

01/24/25

@ Y Combinator

Training larger models requires substantial computing power, which translates to more GPUs running for extended periods and consuming increasing amounts of energy.

JB

Joe Benton

03/18/25

@ Anthropic

Control evaluations must be rerun with each new model to ensure that monitoring systems remain effective against potential threats.

JH

Jensen Huang

07/30/24

@ NVIDIA

Using larger AI models to teach smaller models allows for the creation of specialized models that fit specific operating domains or cost requirements.