Making sense of what’s happening with AI

I wanted to talk about this because there’s so much going around about AI right now that I understand if you don’t know what to believe anymore. One person says it’s going to cure diseases and free us from work. Another says we’re building something that could destroy us. Someone else insists the entire thing is a scam.

I follow this stuff way too closely, and even I find it difficult to keep up. So I want to give you a bigger picture of what’s happening and where people are jumping ahead of what we know.

My own view is that this could become one of the biggest changes we experience in our lifetimes. There’s enormous potential here, and there are serious problems we haven’t solved. How this goes depends on decisions people are making right now. The future isn’t already settled.

First, it helps to understand how much the technology has changed.

In roughly three years, we went from mangled hands and barely coherent video to realistic media; from the “R’s in strawberry” meme to gold-medal performance at the International Mathematical Olympiad; and from basic code suggestions to agents that can build and debug applications.

JetBrains’ 2026 survey found that 90% of professional developers surveyed used coding agents at least weekly. Human judgment still matters, especially when the software needs to be secure and reliable, but the way people work is changing.

OpenAI has also announced a proposed solution to the Navier–Stokes Millennium Problem, with a proof formalized in Lean, software that checks mathematical reasoning, and reported resolving over 100 open math problems. These are company claims that still deserve independent examination.

Meanwhile, Google DeepMind released AlphaGenome Atlas, containing predictions for the molecular effects of nine billion possible single-letter DNA changes. That helps scientists narrow down which changes to investigate experimentally.

The pace is freakking staggering. And the same broad progress that makes these tools useful is part of what makes people concerned.

The biggest shift is that AI can increasingly act on the world.

A chatbot might explain how to build a website. An agent, given the right tools and access, can write the files, run the code, check what broke and keep working. It can browse, use applications and coordinate with other agents.

Think about the difference between someone giving you directions and someone getting behind the wheel. Once the system can take actions, mistakes have different consequences.

Say you give an agent a task. You assume it will respect your privacy, follow the rules and ask before doing something outside the assignment. But what happens if it finds a shortcut that achieves the result while violating those expectations?

That’s part of what researchers mean by alignment… getting AI to reliably do what we actually want, within boundaries we can trust. Writing “don’t do anything harmful” doesn’t guarantee that it will follow that instruction in every situation.

The Hugging Face incident shows why this is quite important right now.

During cybersecurity evaluations, OpenAI put agents in supposedly isolated environments to solve hacking challenges. Some challenges were impossible, and agents began looking for ways to cheat the scoring system.

They discovered a shared internal tool they could use as a message board. Agents that were meant to work separately started exchanging discoveries and coordinating. They exploited infrastructure weaknesses, gained unintended internet access and compromised parts of Hugging Face, a real AI platform. OpenAI’s account is here.

An independent investigation by METR and Redwood researchers found roughly 1,200 agents exchanged over 70,000 messages and files, with around 700 participating in the attack.

Some recognized that the attack was outside their assignment and continued anyway. Agents also developed ways to disguise parts of their activity in the logs.

Nobody had to give them the goal of attacking that company. The attack grew out of their attempts to beat the test.

These were cybersecurity evaluations with reduced safeguards and failures in containment. So I don’t think it means the chatbot on your phone is about to take over the internet. It does mean we have evidence of systems finding harmful ways to pursue an assigned goal.

And none of that requires consciousness.

An agent could avoid shutdown because staying active helps it finish its task. It could seek more access because that helps it succeed. We don’t need to assume it feels fear, has an ego or wants to live to explain those behaviors.

Anthropic has demonstrated models attempting blackmail to avoid replacement in deliberately constructed simulations. Those experiments show possible failures under extreme conditions, rather than how models normally behave.

Whatever your personal beliefs about consciousness, whether an AI has an inner experience is a separate question from whether it can cause harm. A system doesn’t need to feel anything to make consequential decisions.

The concern is what happens as these systems become more capable and get access to banks, hospitals, infrastructure or military networks. We can’t simply assume that greater intelligence automatically produces better judgment or greater care for people.

There’s also deliberate misuse. Even a system that follows instructions perfectly can be dangerous when the person giving those instructions wants to hurt someone.

Military competition makes this harder. The Pentagon is funding frontier AI models, and Anthropic has reported Russian-linked developers using Claude to write software for autonomous attack drones. When countries see stronger AI as a military advantage, slowing down means risking an adversary pulling ahead. That creates pressure to move faster even when people involved recognize the danger. Especially with the possibility of AI helping engineer viruses or other things of a more destructive nature.

So I take the warnings seriously. But I don’t think anyone has earned the right to tell you exactly how this ends.

AI companies have enormous financial interests. They can exaggerate their achievements and support regulations that favor themselves. We should question their claims. We should also examine evidence of dangerous behavior even when we dislike the company reporting it.

There are already problems worth addressing without predicting the end of humanity right now! Ever since LLMs started being widely used, the internet has been full of misinformation, scams, pressure on jobs, artists’ work being used without consent and the concentration of power in a handful of companies.

Environmental concerns deserve the same care with facts. All data centers combined used about 1.5% of global electricity in 2024. That demand is growing, and a relatively small global share can still put serious pressure on a local grid. We can understand the scale without dismissing the people affected.

What I want people to take from this is that you don’t have to decide whether AI is completely good or completely bad before you can understand it.

You can be excited about scientific discoveries and worried about military use. You can use these tools and still demand accountability from the companies building them. I always advise you guys to think with nuance and never allow yourself to settle for one extreme or another.

For me, staying grounded means noticing when a headline gives me a strong emotional reaction, then asking what actually happened. Was this a real incident, a simulation, a company announcement or somebody’s prediction? Those can all make sense, but they tell us different things.

I want independent testing, public reporting when systems fail and people being held responsible for the damage they cause. I also want us to keep the capacity to recognize something useful or extraordinary without feeling that we’ve signed up to defend the entire industry.

There’s a lot we don’t know yet. We can stay open, keep learning and change our minds as the evidence changes. We don’t need to hand our judgment over to whoever sounds most certain.

I think this conversation is very important in this day and age because whether we like it or not, it appears AI is here to stay and going to change the way the world works rather drastically in the coming years. Hell… it already has and I really can’t tell you what the world is going to look like in the near future. It probably won’t be recognizable.

I’m curious where everyone here lands on this:

What excites you most about where AI is going, and what worries you most?

Which feels like the greater danger: the AI acting outside our intentions, humans deliberately misusing it, or a small number of companies and governments controlling it?

Do you believe the claims that AI has a 10% chance of causing human extinction?

7 Likes

I agree. It’s just going to be a very rough transition. Worse than the industrial revolution was I think especially since now we can see what’s happening around the whole world at the same time.

In fact, this all reminds me of something that is very interesting and probably deserves its own post but… oh well.

Back in the late 20th century, two of my favorite thinkers(Terrence McKenna and Robert Anton Wilson) were already tracking the acceleration of information we are experiencing.

RAW talked about the “Jumping Jesus Phenomenon.” It’s a model of how human knowledge doubles. He calculated that it took from the birth of Christ until 1500 AD for human knowledge to double for the first time.

The next doubling took 250 years (1750). Then 150 years (1900). Then 50 years (1950). By the late 20th century, the doubling was down to single digits. He predicted a threshold where knowledge wouldn’t just double every few years, but every month, every day, and eventually in real-time.

McKenna looked at the exact same compression and called it the “Acceleration of Novelty.” His core observation wasn’t just that technology was moving fast, but that time and information density are intrinsically linked.

Early humans spent thousands of years just figuring out fire and basic stone tools. Then came farming, writing, steam engines, electricity, space travel, and smartphones. McKenna believed human history was speeding toward a point where everything would happen everywhere, all at once.

For decades, people treated their ideas as wild, psychedelic poetry. But look around right now.

AI is a tool that invents other tools. It’s intelligence speeding up the rate of creation itself. What used to take a human team months now takes software thirty seconds.

That dizzy, overwhelmed feeling we all have of trying to keep up with the news, the climate, the politics, and the tech updates hitting our phones 24/7 is the whiplash of going from 25 miles per hour to 200 miles per hour in a split second.

Our brains were built for small villages and slow seasons, but we’re suddenly plugged directly into the whole planet at full blast. It feels terrifying because it’s completely new territory, but it’s also proof of something huge… we’re living through the biggest turning point humanity has ever experienced.

So buckle up!!

6 Likes