r/technology • u/Just-Grocery-2229 • 19d ago

Artificial Intelligence Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-ceo-warns-of-ai-driven-botnet-swarm-taking-over-the-entire-internet-in-6-12-months-such-a-swarm-could-be-capable-of-taking-over-the-entire-internet-with-a-persistent-botnet
10.4k Upvotes

94% Upvoted

1.8k comments sorted by

View all comments

Show parent comments

24

u/ploptart 19d ago edited 19d ago

Agreed, but it’s also insane to imagine you’re building something you believe to be extremely dangerous, but you keep building it and encourage people to use it, because it makes you money.

Or you’re investing in something you believe to be detrimental to humanity, and rather than stopping your funding, you continue, because you could get even richer than you already are.

Then you rationalize it by saying other people are risking civilization for their own benefit too, so it’s fine for you to put your desires above the entire species. You could pause until you felt sufficient guardrails were in place, but that would mean the other guy will get richer than you.

“Someone stop me because I want to be rich so badly that I’ll risk killing all of civilization!”

Humanity is kinda shit

4

u/Tetratheon 19d ago

Their idea is that they'd rather it be them than some other company which reaches AGI/ASI first because they each think the others are doing it less safely than they are.

Even if they solve alignment, it puts whoever the ASI is loyal to in a position of being by far the most powerful person in all of human history, and they retain the power until they feel like giving it up. Do you want to live in Elon Musk's world? I bet Elon Musk thinks he'd make a great God Emperor.

Even if they all shut down tomorrow, China wouldn't, and one thing Trump was right about is that whoever wins AI, wins forever. It's over.

It needs government restrictions and an agreement with China akin to nuclear arms reduction treaties.

6

u/Starfall0 19d ago

Uh, no. None of them think they are doing it safely. Multiple techs quit in the last week or 2 because they couldn't continue to work on something they saw as so dangerous. This right before the 3 CEO's came out and repeated what the techs that quit said... I also love how inherently fucked we are as a species.

We are flying breakneck towards a technology that if they aren't yanking our leg on, could ruin our species with ease... So does it really matter who wins forever if our prize for winning or losing is death?

6

u/Tetratheon 19d ago

None of them think they are doing it safely, all of them think they're doing it more safely than the other companies.

None of them think ASI means death for everyone, they think it's a risk. Each of them think that risk is lower if they do it than I'd someone else does.

The other side of the risk is abundance beyond imagination.

1

u/Grisbyus 18d ago

Does USA or China stick to agreements?

1

u/Tetratheon 18d ago

Sometimes they leave them, but they stick to them when they're in them.

In the case of a US/China agreement it'd be pretty easy to verify because they'd need to build out data centres to break it.

2

u/Grisbyus 18d ago

With the imposed tariffs, US did not stick to USMCA agreement intentions.

2

u/Starfall0 17d ago

Well as you can see from this article, yes. International agreements totally work!

https://research.noaa.gov/two-additional-regions-of-asia-were-sources-of-banned-ozone-destroying-chemicals/

Oh wait...

1

u/Tetratheon 17d ago

The restoration of the ozone layer is one of the most successful international efforts ever.

2

u/Starfall0 17d ago

And yet, proving my point. It still happens and requires 3rd party investigators to even call it out. It's easy to see cfc in the atmosphere... can you see an underground research facility for AI? 

Hell it doesn't even need to be underground it could be dead center of a city in some fake building. Good luck finding out where that is.

My point being government regulations and agreements mean jack shit if the action of ignoring the agreement isn't a highly visible thing.

1

u/Tetratheon 17d ago

Data centres need to be huge and put out a lot of heat. If China or America start doing massive digging projects it won't go unnoticed.

1

u/Starfall0 17d ago

Uh huh... because there's no way to hide that or to alter the structure. They also don't need a massive data center for the advancement of ai. That's simply for them to be able to provide the services of AI to a looooot of people at once.

And its not hard to vent heat from underground in fact it's even easier because the ground typically stays a stable cool temperature at all times.

1

u/ploptart 18d ago

But one company building it doesn’t stop another company from also building it. People who believe in AGI or ASI don’t think humans will be able to control it. Having one more company building it adds more risk, regardless of how well-intentioned that company is. But it could make you incredibly rich, so… to hell with everyone else!

1

u/runningraider13 18d ago

Or you keep building it because someone is going to build it. And if all the people worried about AI safety stop building AI, then the person that does build it is not going to be worried about AI safety.

It’s a collective action problem, and the only solution is external intervention from government.

2

u/Grisbyus 18d ago

You mean intervention from the stable genius?