Tools

Search

Explore

Videos Channels Figures

Atmrix

About

Tools

Search

Explore

Videos Channels Figures

Atmrix

About

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

I believe our brain doesn't actually retrieve information; instead, it fine-tunes and generates the next token autoregressively.

Video

CJ

The Theory That Shatters Language Itself

@ Curt Jaimungal

The Theory That Shatters Language Itself

06/12/25

Related Takeaways

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

The autoregressive process in language generation means that the brain is not just producing outputs but also generating the next input for itself.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

I propose that cognition itself is autoregressive, similar to language, and that our brains have the machinery to support this process.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

Memory consolidation happens through fine-tuning the weights of our neural network over time, rather than being a database of facts.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

Cognition doesn't work like a retrieval system; when we try to remember something, we're not searching a space but generating from our past experiences.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

The brain's memory is not a database; it consists of fixed weights that represent potentialities, guiding the generation of outputs based on inputs.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

Language is an autoregressive model, and this autoregressive nature is fundamental to how we generate and understand language.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

The tip of the tongue phenomenon occurs because our brain is running an autogenerative process, making it feel like we can almost produce the word, but we can't complete the generation.

PB

Paul Buchheit

01/25/25

@ Y Combinator

The last decade has shown that we found the right objective function for AI, which is simply to predict the next token, leading to a more beneficial form of intelligence.

EB

Elan Barenholtz

06/12/25

@ Curt Jaimungal

I don't believe in the traditional concept of working memory; instead, I see it as continuous activation influenced by context, not limited to a short duration.