r/technology • u/Just-Grocery-2229 • 19d ago

Artificial Intelligence Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-ceo-warns-of-ai-driven-botnet-swarm-taking-over-the-entire-internet-in-6-12-months-such-a-swarm-could-be-capable-of-taking-over-the-entire-internet-with-a-persistent-botnet
10.4k Upvotes

94% Upvoted

1.8k comments sorted by

View all comments

Show parent comments

346

u/ClashM 19d ago

Who would have access to the kind of computational power required to manage all those agents though? I can't put my finger on who would be responsible if such a thing occurred.

180

u/yaxis50 19d ago

That's the best part. No one is responsible for the mistakes AI makes because there is no regulation. 

58

u/FeatherlyFly 19d ago

And obviously the existing legislation against hacking only applies to individuals and people who work for small companies.

You couldn't possible consider holding Altman and Dario responsible for their companies' actions, they're rich! 

2

u/ptear 19d ago

But I thought they wanted a big virtual hug.

1

u/Senior-Albatross 13d ago edited 13d ago

That's a fundamentally important principle in Western law. We don't hold people rich and connected enough to account if their fall would have an impact on the portfolio of the Judge. 

10

u/GarageNo8276 19d ago

If AI commits a crime the owning entity should absolutely be liable even without regulation. If Google paid people to hack Amazon for whatever reason it's a crime. No different if you tell a computer to do it.

1

u/HarveysBackupAccount 18d ago

"But what if the computer decides to do it by itself?"

3

u/PenguinTD 19d ago

yeah, so who is paying the bill for this permanently attached to internet botnet swarm? Which company's model is already making more money than they burn and can self sustain the equipment/electricity/maintenance bill? Are people really that oblivious on how the internet runs?

Why not just hack Wall Street or the banks and then more damage is done that way? Why a botnet that cost so much just so agents chat with each other and just hacking for fun and try to beat the benchmark score?

What can botnet or agent do when a admin just come and flip the switch off then on with new boot image that auto update with a reboot and wipe the slate clean. So Anthropic is just gonna let all the agents run indefinitely because they have some top secret sauce and can't be shutdown and cut off Internet access, even physically?

Like, really?

3

u/WatzUpzPeepz 18d ago

This is the biggest dupe in the whole AI discussion. It is not sentient or capable of unprompted action. It is a tool, and should be subject to the same treatment at a minimum.

If I used regular software to conduct a cyberattack I am individually responsible. If I used a worm or virus that replicated itself far beyond its original target, I am still individually responsible. If I told an AI to solve a problem and it did so by obtaining the answers by hacking, I should be responsible, because of my extreme negligence in preventing it from doing so.

2

u/KryptonicOne 19d ago

Surely we'll all be fine because yall have the smartest president (high IQ) to keep AI in check.

20

u/This-Requirement6918 19d ago

.... Amazon Web Services?

2

u/BudgetMattDamon 19d ago

Nah, that's not doomy enough.

We need something that'll sound badass when we uncover lore in decrepit terminals 200 years from now.

2

u/Won-Ton-Wonton 19d ago

To be fair, consumer compute devices have a total compute power that is vastly larger than all AI data centers combined.

If a distributed networked AI super intelligence were to create a synchronous layer of some sort, it could make prediction about node compute layers using global consumer hardware, completely negating latency issues... kinda like how our brain will fill in an image before it actually processes what the eyes are seeing, and only retroactively changing what we thought we saw into what we actually saw.

The idea is science pseudo-fiction. But the idea of bot swarms already exists, as does distributed systems, and distributed intelligence compute.

It is entirely possible that the real super intelligence will happen when an AGI tries to improve itself by piggybacking on consumer hardware. Or an LLM model does the same thing by mistake and creates an AGI as a result of trying to "make" distributed consumer hardware, with all of its disconnections, dropped signals, corrupted memory sectors, etc... just "work" for intelligence compute.

Kinda like distributing the brain across specialized lobes with "synapse networks" that break, move, grow new cells, make new connections, and that somehow powers your brain.

Except a synapse network is now you, your spouse, your kids, and all the hardware devices in your home that you have connected to the internet, which it uses 5% of to compute with but only sometimes and sometimes it specializes that network and other networks become more generalized or simple message passers.

1

u/ClashM 19d ago

LLMs aren't ever creating or evolving into AGI, but otherwise I'm familiar. My background is in computer science, I'm just being snarky at the AI hyperscalers' expense.

1

u/sephirothFFVII 19d ago

If they distributed it they could run this from laptops and run of the mill servers. Token generation would be abysmal but it doesn't take a lot to make a rage bot unfortunately

1

u/Joohansson 19d ago

I think the scary part is it doesn't have to be a single entity. Say a hacker group of 1000 users from some dark corner of the darknet join forces paying for 1 agent each and 1 coordinator AI, that could probably do a lot of damage with shady enough models.

1

u/kvlr954 19d ago

The guy from Black Mirror’s Plaything episode. That’s who!

1

u/Think_Wing_1357 19d ago

No kidding, who happens to had a box full of blackwell cards running for these "persistent botnet"?

1

u/ProfessionalGold6193 19d ago

Given that local models could be build to facilitate this, it could all be done quite easily by someone with an agenda. Now think of the havoc someone like Musk could create in defending the cheeto.

1

u/ElectricEcstacy 17d ago

each computer you control in the botnet gives you more computational power.

1

u/Quirky_Entry_2783 19d ago

How about a distributed network of internet connected handheld devices that could be used as a compute platform.
How's your battery life been lately?

1

u/Membership-Great 19d ago

I too saw Silicon Valley but unless there’s a shred of proof of this then you’re kinda just talking out your ass

1

u/Quirky_Entry_2783 19d ago

I didn't watch Silicon Valley past the first season. I'm guessing they put Pied Piper on people's phones?