Self-supervised learning aims to reduce human supervision by allowing machines to learn from unannotated data, which has been successful in natural language processing and is increasingly applied in computer vision.

Supervised learning involves training neural networks with examples, where the network learns by being provided with labeled data, such as images of cats and dogs with their corresponding labels.

The last decade focused on large-scale supervised learning, where increasing data and model size led to improved performance.

The dream of self-supervised learning is for AI systems to learn from vast amounts of unstructured data, similar to how children learn from minimal examples after being exposed to extensive information.

Neural networks are now effective because we can train them to solve various supervised learning problems, mapping examples to labels and outperforming other methods.

We are exploring automating the solution of machine learning problems through a concept called 'learned to learn', aiming to reduce reliance on human machine learning experts.

In self-driving cars, supervised learning can help label the positions of other vehicles based on sensor readings.

This decade is characterized by the addition of generative AI tools, which build on the foundations of supervised learning.

In AlphaZero, we found that removing human data allowed the system to learn more effectively, achieving higher performance than models that relied on human knowledge.