r/technology • u/Just-Grocery-2229 • 19d ago

Artificial Intelligence Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-ceo-warns-of-ai-driven-botnet-swarm-taking-over-the-entire-internet-in-6-12-months-such-a-swarm-could-be-capable-of-taking-over-the-entire-internet-with-a-persistent-botnet
10.4k Upvotes

94% Upvoted

1.8k comments sorted by

View all comments

41

u/VanGogh0810 19d ago

Dario needs to face jail time, for IT’S HIS RESPONSIBILITY TO MAKE PRODUCTS THAT DO NOT HARM SOCIETAL INFRASTRUCTURES AND HUMANS. Gov needs to reign in the egotistical billionaire tech boys who think this is a game. I included Zuckerberg and Altman.

2

u/CritiqueDeLaCritique 19d ago

That's exactly what he wants so he has an excuse as to why he can't produce a return on investments.

-7

u/Maxfunky 19d ago

If nobody in the United States builds this, it just happens in China with even fewer guardrails. The risks don't become lower, they become higher. Anthropic is in the game because they believe whoever is in the lead needs to be safety focused, so they're trying hard to be that guy.

But they can't just stop or it makes things worse. To stop, the United States first needs to negotiate with China to agree to a pause. That's what anthropic wants. Stopping before that's in place is counter-productive.

10

u/pl3x1 19d ago

Anthropic is in the game to make money, nothing more.

2

u/artemis2k 18d ago

Wrong. China actually has quite a lot of guardrails around AI use. Way more than we do, which is none. Also, who tf cares if China develops first? What is the actual goalpost? AGI? Can you define that?  And why exactly does it follow that if we build it first then that somehow prevents China from building it....

1

u/Maxfunky 18d ago

The goal is to build it slowly and safely. Neither side can do that when they're racing each other. If the United States slows down, it accomplishes nothing if just means a Chinese AI destroys the world instead of an American AI.

1

u/artemis2k 18d ago

What evidence do you have that China wants to destroy the world with AI? Or that they are even participating in this "race"? Why do we have to race toward the end of the world instead of reaching agreements with other countries about it? 

1

u/Maxfunky 18d ago edited 18d ago

Nobody wants to destroy the world with AI, it's a potential consequence of everyone moving so fast. And yes, there is ample evidence that China is most definitely participating in this race. They're the only other player across the entire world. It's very clearly a race between China and the United States.

The parallels are numerous between this race and the space race between Russia and the United States way back when.

China has indicated an unwillingness to agree to safety slowdowns with the United States, but the United States has also made virtually no efforts or enticements towards trying to get such a deal. I'm not blaming China here, I'm blaming the United States for not making the effort.

A slowdown would make us safer, but it only makes us safer if everyone agrees to it otherwise the risk remains the same but just shifted to whoever doesn't agree to it.

1

u/artemis2k 18d ago

How is AI meant to destroy the world though and how does the US getting to that capability first make anyone safer? It's not like with nukes where if we have one it acts as a deterrent to hostilities. Are we suddenly going to be able to control everyone else's development of AI if we "win"?

0

u/Maxfunky 18d ago

How is AI meant to destroy the world though and how does the US getting to that capability first make anyone safer?

It's not meant to, it just might.

The risk exists as a function of something called "alignment". There's no risk with a properly aligned AI but alignment is also not easy and mistakes may happen. The idea is that the risk can avoided entirely if we all slow down so that we have plenty of time to get alignment right.

But if we all race, mistakes will happen.

The biggest threat is only realized if it comes from the lab with the most advanced lab. Because if there's a more powerful AI out there, then that more powerful AI can be used to counteract whatever the less powerful AI is doing.

So, there are two things that keep us safe here: either everyone slowing down, or, failing that, for the lab with the most advanced AI to be taking extra precautions. Right now, anthropic is the lab with that burden. They don't trust handing it off to someone else who takes it less seriously. They want to slow down so that nobody has that burden, but you can see why passing it off to someone else at random is a shitty solution.

Are we suddenly going to be able to control everyone else's development of AI if we "win"?

Quite literally, "yes". The stakes in this race are the entire world and control of it. Because of the exponential curve to capability growth, whoever gets to a self-improving model first will have an insurmountable lead in perpetuity. They will always have the more powerful AI and that will enable them to pretty much do whatever they want. The hugging face incident is already bonkers and we're about to go so much deeper.

An American AGi would be able to shut down the entire Chinese Internet at will. It would functionally be a "pet God".

1

u/artemis2k 18d ago

Look, no disrespect to you, but it seems to me like all of this speculation about AI being an existential threat to humanity isn't grounded in any specific capabilities this software has. An AI on its own does not exist in the physical world and cannot act independently. If you're telling me that the danger is that it will act independently and affect the physical world, you need to explain to me how exactly that could happen. And what about the current models supports that assumption. Right now it just feels like everyone is imagining a scenario like they've seen in their favorite sci-fi movie, but that's not the real world. 

I think we've let a bunch of self important tech cultists who call themselves "rationalists" sell us a shitty fanfic of a book they read and learned the wrong lessons from. And because normal people don't understand tech all that well, they just take what they say as gospel. 

1

u/Maxfunky 18d ago edited 18d ago

I can see how it's hard to connect the dots there, but it's already been done for you. Last year even one think tank published a theoretical scenario just to sort of explain how such a thing might have and what it might look like in the real world with real world capabilities.

That project is called AI-2027 and it's available to read for free, but it's also super long. There's some really good YouTube summaries out there. Here's one I can endorse personally:

https://youtu.be/k_onqn68GHY?si=zOBgEqNFuLkrL_eB

The hugging face incident was even basically one of the explicit waypoints on this scenario, outlined last year before it actually happened.

It's not just the "tech cultists" either; it's a concern for basically all of the smart people in the world. You may recall that Stephen Hawking had quite a bit to say on the subject.

→ More replies (0)