The challenge is to make our models good enough and easy to use so that even non-expert musicians can find them genuinely musically engaging, focusing on generating interesting sounds rather than just melodies.

I have models that continuously process different bodies of text and music to see what interesting outputs emerge, sometimes resulting in messy garbles and sometimes in intriguing sounds.

We're rethinking our music sequence generation work, moving beyond simple models to care about expressive timing, dynamics, and polyphony, aiming to create high-quality music generation tools.

We've learned a lot through the process of training this model, discovering what scales well and what doesn't.

Models will get better at understanding and creating images, audio, and especially video, which will dramatically enhance how we interact with data.

To make music interesting, we need to incorporate hierarchical patterning and conditional models that can generate variations, such as chord changes, to create more engaging compositions.

We haven't yet created a model that can generate an entire song because we haven't perfected the conditional aspects of the LSTM yet, which is crucial for handling long-term musical structures.

Our focus is on optimizing for reasoning and behavior of models rather than just their scores on traditional benchmarks like MMLU.

Models can struggle with tasks requiring expertise and taste, such as evaluating art, which presents a significant challenge for AI.