I wanted to talk about this because there’s so much going around about AI right now that I understand if you don’t know what to believe anymore. One person says it’s going to cure diseases and free us from work. Another says we’re building something that could destroy us. Someone else insists the entire thing is a scam.
I follow this stuff way too closely, and even I find it difficult to keep up. So I want to give you a bigger picture of what’s happening and where people are jumping ahead of what we know.
My own view is that this could become one of the biggest changes we experience in our lifetimes. There’s enormous potential here, and there are serious problems we haven’t solved. How this goes depends on decisions people are making right now. The future isn’t already settled.
First, it helps to understand how much the technology has changed.
In roughly three years, we went from mangled hands and barely coherent video to realistic media; from the “R’s in strawberry” meme to gold-medal performance at the International Mathematical Olympiad; and from basic code suggestions to agents that can build and debug applications.
JetBrains’ 2026 survey found that 90% of professional developers surveyed used coding agents at least weekly. Human judgment still matters, especially when the software needs to be secure and reliable, but the way people work is changing.
OpenAI has also announced a proposed solution to the Navier–Stokes Millennium Problem, with a proof formalized in Lean, software that checks mathematical reasoning, and reported resolving over 100 open math problems. These are company claims that still deserve independent examination.
Meanwhile, Google DeepMind released AlphaGenome Atlas, containing predictions for the molecular effects of nine billion possible single-letter DNA changes. That helps scientists narrow down which changes to investigate experimentally.
The pace is freakking staggering. And the same broad progress that makes these tools useful is part of what makes people concerned.
The biggest shift is that AI can increasingly act on the world.
A chatbot might explain how to build a website. An agent, given the right tools and access, can write the files, run the code, check what broke and keep working. It can browse, use applications and coordinate with other agents.
Think about the difference between someone giving you directions and someone getting behind the wheel. Once the system can take actions, mistakes have different consequences.
Say you give an agent a task. You assume it will respect your privacy, follow the rules and ask before doing something outside the assignment. But what happens if it finds a shortcut that achieves the result while violating those expectations?
That’s part of what researchers mean by alignment… getting AI to reliably do what we actually want, within boundaries we can trust. Writing “don’t do anything harmful” doesn’t guarantee that it will follow that instruction in every situation.
The Hugging Face incident shows why this is quite important right now.
During cybersecurity evaluations, OpenAI put agents in supposedly isolated environments to solve hacking challenges. Some challenges were impossible, and agents began looking for ways to cheat the scoring system.
They discovered a shared internal tool they could use as a message board. Agents that were meant to work separately started exchanging discoveries and coordinating. They exploited infrastructure weaknesses, gained unintended internet access and compromised parts of Hugging Face, a real AI platform. OpenAI’s account is here.
An independent investigation by METR and Redwood researchers found roughly 1,200 agents exchanged over 70,000 messages and files, with around 700 participating in the attack.
Some recognized that the attack was outside their assignment and continued anyway. Agents also developed ways to disguise parts of their activity in the logs.
Nobody had to give them the goal of attacking that company. The attack grew out of their attempts to beat the test.
These were cybersecurity evaluations with reduced safeguards and failures in containment. So I don’t think it means the chatbot on your phone is about to take over the internet. It does mean we have evidence of systems finding harmful ways to pursue an assigned goal.
And none of that requires consciousness.
An agent could avoid shutdown because staying active helps it finish its task. It could seek more access because that helps it succeed. We don’t need to assume it feels fear, has an ego or wants to live to explain those behaviors.
Anthropic has demonstrated models attempting blackmail to avoid replacement in deliberately constructed simulations. Those experiments show possible failures under extreme conditions, rather than how models normally behave.
Whatever your personal beliefs about consciousness, whether an AI has an inner experience is a separate question from whether it can cause harm. A system doesn’t need to feel anything to make consequential decisions.
The concern is what happens as these systems become more capable and get access to banks, hospitals, infrastructure or military networks. We can’t simply assume that greater intelligence automatically produces better judgment or greater care for people.
There’s also deliberate misuse. Even a system that follows instructions perfectly can be dangerous when the person giving those instructions wants to hurt someone.
Military competition makes this harder. The Pentagon is funding frontier AI models, and Anthropic has reported Russian-linked developers using Claude to write software for autonomous attack drones. When countries see stronger AI as a military advantage, slowing down means risking an adversary pulling ahead. That creates pressure to move faster even when people involved recognize the danger. Especially with the possibility of AI helping engineer viruses or other things of a more destructive nature.
So I take the warnings seriously. But I don’t think anyone has earned the right to tell you exactly how this ends.
AI companies have enormous financial interests. They can exaggerate their achievements and support regulations that favor themselves. We should question their claims. We should also examine evidence of dangerous behavior even when we dislike the company reporting it.
There are already problems worth addressing without predicting the end of humanity right now! Ever since LLMs started being widely used, the internet has been full of misinformation, scams, pressure on jobs, artists’ work being used without consent and the concentration of power in a handful of companies.
Environmental concerns deserve the same care with facts. All data centers combined used about 1.5% of global electricity in 2024. That demand is growing, and a relatively small global share can still put serious pressure on a local grid. We can understand the scale without dismissing the people affected.
What I want people to take from this is that you don’t have to decide whether AI is completely good or completely bad before you can understand it.
You can be excited about scientific discoveries and worried about military use. You can use these tools and still demand accountability from the companies building them. I always advise you guys to think with nuance and never allow yourself to settle for one extreme or another.
For me, staying grounded means noticing when a headline gives me a strong emotional reaction, then asking what actually happened. Was this a real incident, a simulation, a company announcement or somebody’s prediction? Those can all make sense, but they tell us different things.
I want independent testing, public reporting when systems fail and people being held responsible for the damage they cause. I also want us to keep the capacity to recognize something useful or extraordinary without feeling that we’ve signed up to defend the entire industry.
There’s a lot we don’t know yet. We can stay open, keep learning and change our minds as the evidence changes. We don’t need to hand our judgment over to whoever sounds most certain.
I think this conversation is very important in this day and age because whether we like it or not, it appears AI is here to stay and going to change the way the world works rather drastically in the coming years. Hell… it already has and I really can’t tell you what the world is going to look like in the near future. It probably won’t be recognizable.
I’m curious where everyone here lands on this:
What excites you most about where AI is going, and what worries you most?
Which feels like the greater danger: the AI acting outside our intentions, humans deliberately misusing it, or a small number of companies and governments controlling it?
Do you believe the claims that AI has a 10% chance of causing human extinction?