The effectiveness of AI models can vary significantly based on how they interpret and apply rubrics, with some being rigid and others more adaptable to exceptions.

OpenAI's recent paper shows that models can effectively grade medical answers based on specific criteria, indicating their potential in other domains.

Generative models might be able to create plots that are difficult for humans to generate but still make sense to readers.

The distinction between how models claim to perform tasks and how they actually do them can reveal significant insights into their capabilities.

In medical diagnostics, models can demonstrate impressive reasoning capabilities, mapping symptoms to correct medical cases effectively.

Models can struggle with tasks requiring expertise and taste, such as evaluating art, which presents a significant challenge for AI.

AI models are valuable because they can interpolate between vague questions and provide tailored responses, even when data is messy.

AI models are valuable because they can interpolate between vague questions and provide tailored responses, even when data is messy.

The surprising interpretability of thinking models allows us to understand how they reach their conclusions, which was not expected in earlier AI developments. The fact that AI models can articulate their reasoning provides a level of comfort regarding safety, even though it's not infinite comfort.