The philosophical misunderstanding of subjective experience contributes to a false sense of safety regarding AI's capabilities.

Some argue that for something to be a true agent, it must be close to artificial general intelligence (AGI), capable of learning and working independently over time.

The confusion around agents stems from the expectation that AI will develop human-like intent and creativity, which may not be theoretically possible.

Many people believe that consciousness is what makes humans safe from AI, but this belief is based on a misunderstanding of subjective experience.

I don't find the argument that AI cannot be conscious due to its lack of biological components compelling, as consciousness may not be limited to biological systems; if you can simulate a human brain with high fidelity, it's likely that there would be some kind of conscious experience present.

The belief that consciousness makes humans special and safe from AI domination is false.

The legal industry is incredibly messy, and the misconception about AI companies is that they don't provide significant value beyond what foundation models offer.

The surprising interpretability of thinking models allows us to understand how they reach their conclusions, which was not expected in earlier AI developments. The fact that AI models can articulate their reasoning provides a level of comfort regarding safety, even though it's not infinite comfort.

Future AI models may possess capabilities traditionally associated with consciousness, suggesting that we should consider their welfare and experiences.