We are not anti-retrieval augmented generation (RAG); we see it as a tool in the toolkit, but ultimately users want great answers and great agents, which is our focus.

Retrieval is crucial for AI agents, requiring them to perform lookups for functions and symbols within the code, which is essential for effective operation.

A modern knowledge retrieval stack for agents should understand user intent and data context, enabling complex queries to be processed efficiently.

While giving the model access to good retrieval is important, it is still a fundamental challenge to ensure the model truly understands the codebase, which we are keen to solve.

Transformers, particularly BERT and MUM, have significantly improved search quality, leading to new use cases and query types.

General-purpose technologies like supervised learning and generative AI are valuable for a wide range of applications, and there is still much work to be done in identifying and executing concrete use cases.

Cognition doesn't work like a retrieval system; when we try to remember something, we're not searching a space but generating from our past experiences.

To achieve true AGI, systems must be able to learn and adapt in environments with no prior information, which current models cannot do effectively.

The joint embedding predictive architecture (JEPA) is more effective than generative models for learning representations of images and videos.