In collaboration with the Street View team, we developed a model to identify text in images by predicting which pixels contain text, using training data where text was marked.

The Maps team utilized a similar model to assess solar energy potential on rooftops by identifying rooftops in images, demonstrating the versatility of our models.

Architecturally, there's a significant similarity among various AI models, including video diffusion models like Veo and text language models, which share a lot of core elements.

We're using drones to analyze the built environment, focusing on processing lines rather than points, which is traditionally more challenging in computer vision.

Further research confirmed that scaling laws apply to various models, including those for text-to-image and image-to-text tasks, while in early 2020, these laws for large language models (LLMs) were largely unknown outside of OpenAI, except for one researcher who focused on the scaling hypothesis, suggesting that increasing the size of data and compute would lead to the emergence of intelligence.

We've learned a lot through the process of training this model, discovering what scales well and what doesn't.

The foundation models today represent the world in text, and we are expanding to include images and video to create a more comprehensive understanding of our environment.

Claude can analyze images and respond with text, and now it has been trained to recognize specific actions on a computer screen, such as clicking buttons or typing text based on what it sees.

We partner with the largest satellite companies to source high-quality imagery and collect lidar data, which is crucial for improving our algorithms and expanding our model globally.