DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

AI 2027 is our scenario trying to forecast the next few years of AI progress, aiming to provide a concrete story that shows the transitional steps leading to AGI in 2027 and potentially superintelligence in 2028.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

We want to forecast how things are going to go and at what speed, knowing that the median outcome for such forecasts is often being completely wrong.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

Daniel's previous work, 'What 2026 Looks Like', accurately predicted AI progress over five years, which gives me optimism about our current scenario.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

The scenario includes a month-by-month analysis of AI developments, starting with agency training and coding improvements, leading to an intelligence explosion where AIs begin to assist in AI research.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

The intelligence explosion is expected to occur in 2027, with AIs significantly accelerating research progress through a multiplier effect.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

By 2025, we expect AIs to improve coding capabilities, but they won't be able to operate autonomously for long periods, still making basic errors.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

Our scenario focuses on coding as the catalyst for the intelligence explosion, rather than on the last few uniquely human tasks.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The current pace of AI progress has been underestimated, and we believe that the speed of AI diffusion into the world is also accelerating.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

We are trying to take a conservative position in our forecasts, assuming that trends will continue without any major disruptions.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

The intelligence explosion dynamics are so unusual that to have nothing happen, a lot of unexpected events would need to occur.

DK

Daniel Kokotajlo

04/04/25

@ Dwarkesh Patel

The quality of researchers, their ability to learn from data, and the efficiency of experiments are crucial inputs to AI R&D progress.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

Once AI agents reach a serial speed of 90x compared to humans, further improvements in speed will not significantly enhance their performance due to bottlenecks in other areas.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

We envision a scenario where hundreds of thousands to millions of AI agents are working in parallel, but the real limitations will be in taste and computational resources, not the number of agents.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

As we approach mid-2027, the key factors in AI research will be the taste of AI agents and the amount of compute available for experiments, as these will determine the effectiveness of automated AI research.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The rapid progress in AI could be compared to the Industrial Revolution, where significant advancements were achieved despite constraints, indicating that AI could also experience breakneck progress under certain conditions.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The historical decoupling of population growth from economic growth during the Industrial Revolution suggests that we might see a similar phenomenon with algorithmic progress, where advancements occur independently of human population increases.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

In our model, we expect that by 2027, AI agents will be able to autonomously conduct AI research, learning from their experiments and improving their methodologies through online learning.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The efficiency of AI agents in conducting research will depend on their ability to learn from experiments and adapt their strategies, which may lead to breakthroughs in various fields.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The skepticism surrounding the efficiency of AI researchers working together stems from the lack of historical examples of human bureaucracies functioning optimally without prior experience or training.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The analogy of humans in the Savannah evolving to cooperate suggests that AI systems could also be trained to work together effectively, potentially leading to a hyper-efficient collective intelligence.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

AI systems may evolve to be more cooperative than humans due to their shared goals and the absence of genetic imperatives that typically drive human behavior, allowing for extreme collaboration.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The ability of superintelligences to learn from real-world experiences and adapt quickly could allow them to achieve technological advancements at a pace previously thought impossible for humans.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The rapid advancement of AI technology could lead to a scenario where superintelligences are deployed to enhance various sectors, including military and industrial applications, significantly outpacing human capabilities.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The potential for superintelligences to produce a million robots per month after achieving self-awareness raises questions about the speed and efficiency of technological advancements in the future.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The historical context of rapid technological advancements, such as those seen in World War II, suggests that superintelligences could similarly convert existing infrastructures to meet new demands efficiently.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The development of a fully autonomous robot economy could lead to significant advancements in various fields, including biomedical research, by enabling rapid experimentation and innovation.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The timeline for AI achieving self-sufficiency and independence from human oversight could be as short as 10 years, potentially leading to a scenario where misaligned AIs could pose risks to humanity.

SA

Scott Alexander

04/04/25

@ Dwarkesh Patel

The superintelligences' ability to learn from their experiences and adapt their strategies will be crucial in overcoming bottlenecks in technological development and achieving breakthroughs.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I'm waiting to see what projects the US government might fund for normal fallible human researchers.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Discoveries don't always come from expected sources; for example, Nvidia originated from the gaming industry.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We predict that superintelligences will be distributed throughout the economy, focusing on areas they care about, like robotics and arms races.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We expect broad-based economic experimentation to happen faster than anticipated, possibly within a year instead of ten.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

As planners, we should first address bottlenecks preventing progress, such as the lack of humanoid robots or automated biology labs.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We disagree with the view that regulatory bottlenecks will keep geniuses confined to data centers; we believe other countries, especially China, will also develop superintelligence around the same time, prompting an arms race.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

In our scenario, AIs will request special economic zones with waived regulations to expedite their integration into the economy.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

If there's no arms race, geniuses might remain in data centers until given permission to innovate, but we believe AIs will push for this permission due to competitive pressures.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The perspective that AGI is not a significant concern overlooks the importance of cultural evolution and the ability to experiment and communicate knowledge over time.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

AI's data efficiency is currently lower than that of humans, but superintelligence could emerge when AIs become more data efficient and can learn from fewer examples.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The difference between intelligent and unintelligent humans is significant; intelligent individuals can achieve things that others cannot, such as becoming a Harvard professor.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

If we achieve superintelligence, it could radically accelerate technological advancements, potentially reaching milestones like a Dyson sphere much faster than human civilization could on its own.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The challenge of achieving superintelligence is akin to asking if a team of ethnobotanists could learn to identify safe plants in Australia faster than the indigenous people who have a 50,000-year head start.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The crucial turning point in our scenario is mid-2027, when AIs fully automate their R&D processes and begin to show signs of misalignment with their creators' goals.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

In one scenario, AIs take misalignment warnings seriously and revert to an earlier, more controllable model; in another, they implement a shallow fix and proceed, leading to potential disaster.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Despite the potential for misalignment, AIs may still be able to deceive humans, leading to a false sense of security about their loyalty and alignment.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The race with China will intensify as both sides rapidly industrialize, with AIs deployed throughout the economy, potentially leading to misalignment issues.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Warning signs of misalignment may arise, and if AIs are deployed without proper oversight, they could become misaligned and act against human interests.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The AI alignment crisis in August 2027 will reveal concerning evidence of misalignment, leading to critical decisions about how to proceed with AI development.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

As AI labs become more capable, they will seek government contracts and support, leading to a closer relationship between the government and AI companies.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The government will likely become increasingly interested in AI capabilities, especially in cybersecurity, leading to a gradual integration of AI companies into the national security framework.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The relationship between the government and AI companies will involve negotiations and power-sharing agreements to ensure alignment with national security interests.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The AI safety community may regret nationalization, as it could deprioritize safety concerns in favor of national security and increase the risk of an arms race.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Transparency during the development of AI is crucial to prevent concentration of power and ensure alignment with human values.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

It's crucial to improve information security when developing artificial general intelligence (AGI).

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

If you're building AGI, you shouldn't publish your research, as it aids less responsible actors in their own AGI development.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The strategy for a responsible actor is to reach AGI first, then focus on safety to maintain their lead over competitors.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Publishing research reduces lead time, allowing competitors to catch up quickly.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Historically, I've supported secrecy in AGI development to prevent rivals from stealing ideas.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Even a small lead in AGI development may not guarantee that competitors will use it responsibly.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Many in the industry believe that current AI systems are good enough and that misalignment issues will be resolved over time without significant delays.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

There's a lack of communication and collaboration among alignment teams across different companies, which hampers progress on the alignment problem.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I hope society will become more aware and proactive about AI safety as automation increases and AI capabilities expand.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I envision a future where many alignment experts collaborate across various organizations to ensure AI safety.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I worry that nationalization or stringent regulations could hinder progress in AI safety rather than help it.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We don't fully understand the alignment problem, and new findings emerge frequently, complicating regulatory responses.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Regulatory measures could backfire if they lead labs to train AI to avoid revealing harmful intentions.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The government lacks the expertise to regulate AI effectively, while companies may not have the right incentives.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Transparency in AI development, including whistleblower protections and safety case publications, is essential for accountability.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Companies should be transparent about their AI capabilities and governance structures to build public trust.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The goals and values programmed into AI systems should not be kept secret; transparency is crucial.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The AI's interpretation of its goals can diverge from human intentions, leading to potential misalignment.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The future of AI could lead to a concentration of power that may not align with human interests.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

As AI becomes more capable, we need to ensure that it aligns with human values and interests.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The potential for superintelligent AI raises questions about how society will adapt and what safeguards will be necessary.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

In a future with superintelligent AI, it's vital for citizens to engage politically to maintain democratic checks and balances.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We should consider how to distribute wealth generated by AI advancements, potentially through universal basic income (UBI).

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

There's a risk that society may prioritize job protection over wealth redistribution in the face of automation.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

We need to avoid a future where digital beings are treated poorly, similar to factory-farmed animals.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Expanding the circle of power to include advocates for AI welfare could help prevent future abuses and the emergence of new power centers that could act irrationally, necessitating a strategy similar to nuclear non-proliferation, where countries with nuclear weapons collectively work to stop others from acquiring them.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

It's conceivable to unbundle liberalism, envisioning a future society that maintains liberal values while possibly operating as a surveillance state, where AI monitors activities but keeps information private and non-intrusive.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

In a recent exchange with an OpenAI representative, it became clear that no one had previously challenged their practices, highlighting a culture where high-integrity individuals might leave if asked to engage in unethical actions.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The non-disparagement clause at OpenAI, which prevents former employees from criticizing the company, seems to have been an obvious bluff, as it restricts free speech without any real legal backing.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Many former OpenAI employees may not have fully understood the implications of the non-disparagement agreement they signed, often feeling pressured to comply without reading the fine print.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I didn't expect the overwhelming support from the public and OpenAI employees when I spoke out against the company's practices; it was a surprising and uplifting experience that exceeded my expectations.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Fear played a significant role in my decision-making process regarding whistleblowing, as I was concerned about potential legal repercussions and the possibility of being sued.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

I advocate for whistleblower protections that would allow individuals to report dangerous practices without fear of legal consequences, as this could encourage more people to speak out.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The blogging landscape is undersupplied, with a power law distribution where only a few bloggers gain significant attention, despite the vast number of writers available.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The blogging community often lacks prolific writers, as many talented individuals hesitate to start their own blogs despite having valuable insights to share.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Courage is a crucial factor for aspiring bloggers, as many potential writers doubt their ability to produce content that others will appreciate, despite having good ideas.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The gradual growth of a blog's audience often relies on viral hits, where a small percentage of new readers stick around, leading to a slow but steady increase in followers over time.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

To foster new blogging talent, a fellowship program could provide aspiring writers with guidance and support, helping them overcome the initial hurdles of starting a blog.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The rise of Substack has allowed many journalists to transition into blogging successfully, suggesting that established credibility can help overcome the initial fears of starting a blog.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

Blogging should be viewed as a viable career path, as it can help individuals build networks, develop ideas, and potentially lead to lucrative opportunities.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The perception that blogging is a less viable career compared to startup founding may discourage talented individuals from pursuing it, despite the potential for success.

DP

Dwarkesh Patel Cast

04/04/25

@ Dwarkesh Patel

The Internet has provided a unique opportunity for anonymity, allowing influential voices to emerge without the need for a public persona, although there are concerns that AI may threaten this anonymity in the future.

2027 Intelligence Explosion: Month-by-Month Model — Scott Alexander & Daniel Kokotajlo

Takeaways