r/technology • u/Just-Grocery-2229 • 19d ago

Artificial Intelligence Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-ceo-warns-of-ai-driven-botnet-swarm-taking-over-the-entire-internet-in-6-12-months-such-a-swarm-could-be-capable-of-taking-over-the-entire-internet-with-a-persistent-botnet
10.4k Upvotes

94% Upvoted

1.8k comments sorted by

View all comments

595

u/VegetarianZombie74 19d ago

These guys are deeply in debt for a product that doesn't nearly provide enough revenue to pay for itself. Nevermind, all the coming costs in the next few years. They basically bought a fleet of private jets on 30k per year salary, and the first payment is due next year.

Treat them like the bullshitters they are.

51

u/TooFewTulips 19d ago

Declaring your business a threat to humanity in hopes that the government takes over is an interesting alternative to bankruptcy.

8

u/AliceInWonderment 18d ago

This should be MUCH higher up. This right here is this grift

70

u/DezXerneas 19d ago

This is literally their entire MO. Every 3 months they release a press release saying "Our AI is so awesome, it's about to destroy the world, so we won't release it(to you, the government and people who pay us get full access)".

I won't say their models aren't great, but they're just LLMs. An LLM cannot become a real swarm or agi, you need someone telling it to do shit, but cloudflare is now relatively good at blocking them out. Yeah a Fable tier model can take 'manual' control of a browser and simulate real clicks and stuff(which also gets blocked by cloudflare btw), but does anyone even know how much money that actually burns to run?

25

u/Economy_Kale5580 19d ago

Hey pal, that makes sense and isn’t scary, get outta here

2

u/wonkytalky 19d ago

I'm honestly just sick of being locked out to a growing list of websites, including one github alternative hosting a couple repos I've needed to use, because apparently my IP is part of some block in which someone was identified as a bot or something. I have to switch off WiFi on my phone and use that to get to the blocked stuff now, which as you can imagine, isn't entirely helpful for dev work on a laptop.

Thanks to some LLM zealot somewhere nearby, this legitimate human is being locked out. Fun times!

2

u/DezXerneas 19d ago

Yeah, had that happen to me for a while. Annoyed the fuck out of my ISP, trying to get them to fix it, but eventually I just switched providers.

Especially since I block a lot of tracking data that would normally identify me as a real human, so I already get flagged as a bot, and then there's some asshole letting his agents hammer the internet living near me.

1

u/wonkytalky 19d ago edited 19d ago

Thank you for legitimizing my experience. It's a little better knowing that my blocked ass at least has some company, lol

Edit: I also block a ton of telemetry, so I see those "are you a bot?" pages a little more often than the average bear as it is. The Anubis system or whatever it's called that's used for the FreeBSD forums is the absolute worst. But yeah, whatever is going on at Codeberg, for example, is definitely blocking my IP range and not just my browser. I happened to swap out some networking gear recently, which triggered an IP change, and yup, issue's still there from a completely different machine and browser. New ISP is coming in a few months though and I'm for sure switching, so I'm assuming that'll solve that issue (for awhile, anyway).

1

u/bloodychill 18d ago

The burning money is the part that doesn’t math in their vision. Like, fine, your LLM toolchain now makes it comically easy (if ultimately extremely expensive) to create software and inauthentic content, but with no increased demand, and in some cases reduced demand since inauthentic content scares off the real people. Whoops.

1

u/CharlieandtheRed 18d ago

I work in cyber security and last week I saw a 72M/hour attack from every single country on earth on a small target. I know this doesn't mean a lot to others, but this scared the shit out of me. That's a wild wild number usually reserved for big targets.

-1

u/thefuckevengoingonan 19d ago

An LLM cannot become a real swarm or agi, you need someone telling it to do shit,

I don't think anyone is talking about agi. The issue is that these models exist that are very capable to hacking into ass sorts of shit even just as a way of solving a non-hacking related issue. If someone with access to one of those models did give it directions to hack data centers and spin up copies of itself.
Yes you can give an llm a copy of itself and access to your AWS account and tell it to spin up multiple instances of itself to do various tasks. This is something that is possible for a long time and nothing special. If you gain access to a model that can hack other people AWS account and other data centers, then we fucked. This capability already exists. It's just no one with access has turned then to the task of spread and destroy, yet.

6

u/DezXerneas 19d ago

Sure, that's possible, but you're taking aws security way too lightly, and I'm pretty sure spinning up more instances of a LLM isn't even close to the most profitable thing you could do if you found a reliable way to hack into AWS accounts.

Also, a model capable of doing that isn't gonna fit in a normal person's aws account, and there's spending limits and shit so you'd have to compromise banks as well to pull this off. I don't think a basic AWS instance can run fable(or equivalent) at tps to be any danger at all.

Also, there's a reason these AI are only being used to test internal code and not unleashed on the internet, and it's not just the cost. They're genuinely good enough once you give them enough context about the code, but we're not even close to a digital grey goo tier event. Worst case, aws shuts down infected/compromised clusters using their signature.

2

u/thefuckevengoingonan 19d ago

what on earth are you basing that on? you can run kimi k3 Q8 for ~$150 an hour. i could spin up an instance right now on my aws.

but you're taking aws security way too lightly, ima just assume you have never used aws at this point.

2

u/RafaelWanderpfad 19d ago

OpenAI likely spent millions of tokens during the HuggingFace hack. This is not about spinning up one instance of Kimi bro lmao. 

2

u/thefuckevengoingonan 19d ago

You need one instance to hack one thing, one account. Then you have 2 instances. The agent/models can also spin up and make use of smaller models. As is the norm for usage now. Simple tasks go to simple models on , hard tasks to more expensive models. Orchestration/C&C can be done decentralized as is already done with botnets. (ie, instructions are encrypted and published to any public blockchain), all agents running all models get instructions and complicate via the blockchain. None of this is new stuff except that LLMs are just way faster then humans at doing it.

1

u/DezXerneas 19d ago edited 19d ago

Bro k3 Q8 is not hacking aws. You can run it, and eventually we might have something that could run in a normal container and do it, but imo at this point these agents are much better suited as just a replacement for a scam call center or something and to get people to install ransomware or a RAT for later.

Also, I think most aws accounts are capped to wayyyyyy below $150/hour lol

1

u/thefuckevengoingonan 19d ago

k3 at running at q8 is close to fable in capability. i use it as a comparison for VRAM usage and cost to run. no, i don't think k3 will be the model that hacks the world, i don't think it is capable. never the less it has shown to be capable of find and using 0 day vuls. the full k3 FP16 would cost you 5.6TB vram. ~$300/hour. that is what we should expect as requirements for this level model (mythos). k3 is not there but it's close and at current rate of improvement i completely see they are already there with closed/unreleased models. or i maybe i am just crazy and LLMs are just 100% scam with zero ability to write code at speeds surpassing human speed in every way.

3

u/RafaelWanderpfad 19d ago

They can write code the same way they can write other languages. It’s the one thing LLMs truly excel at. Doesn’t mean they will become sentient, take over the internet, annihilate humanity or all that marketing BS. 

3

u/thefuckevengoingonan 19d ago

i am not talking about AGI or sentient or whatever. That is not required at all for anything I said. i am also not talking about anything making any decisions at all. I am talking about people setting up and using these tools to create botnets, like hackers have been doing since the start of the internet. It's just LLMs are millions of time quicker then a human.

2

u/Longjumping_Ad_424 19d ago

Would you worry about important online infrastructure being hacked? Like a power grid or some sort of electricity or water supply / filtration system. I feel like malicious hackers or people in control of soon to be great models will unleash a ton of pain for the public.

Isn’t that a reasonable thought in the next few years?

→ More replies (0)

12

u/SuperAlloy 19d ago edited 19d ago

💯 let me know when the AI pocalypse starts cause I got shit to do in the meantime and this is like the 17th tech apocalypse news cycle I've lived through and I don't care anymore

my parents hid under their desks in school because they were afraid of nukes and that shit seems far more likely

0

u/juanmaale 19d ago

no it doesn’t lol

3

u/ii-___-ii 18d ago

yes it does lol

22

u/bubblegum-rose 19d ago

At least in prison this guy would have food and a roof over his head.

Once the AI bubble bursts this dude won’t be able to finance a soy sauce packet for his gas station sushi

24

u/[deleted] 19d ago

[deleted]

2

u/sunburnedaz 19d ago

Ahh the theranos model.

1

u/JAlfredJR 19d ago

More like the Web3, blockchain, crypto, Metaverse, NFTs, and on and on and on. Funny how these folks are all connected

24

u/Schonke 19d ago

Once the AI bubble bursts this dude won’t be able to finance a soy sauce packet for his gas station sushi

Oh sweet summer child... Your first bubble? There's no way amodei, Jensen or any of the other AI CEOs and top level hypers will ever be destitute.

8

u/CrashyBoye 19d ago

Once the AI bubble bursts this dude won’t be able to finance a soy sauce packet for his gas station sushi

LMFAO you people are so cute. It’s adorable you think they still won’t come out of this filthy rich.

1

u/ProduceNo1629 19d ago

That piece of shit Scam Altman owns a McLaren F1.

That alone is $20M he can use to live the rest of his life in opulence.

0

u/Pitiful-Factor5750 19d ago

it won't burst like we think it will. They literally sold reddit for a couple hundred mil to the bots, same for Pinterest and the others are next so that revenue keeps them from bursting right away. Besides they have the government sustaining them through subsidies and gov contracts.(just USA for now). Just a few hours ago the orange guy said on a press conference he will sustain Antropic to alleviate their concerns about the future.

2

u/FeRooster808 19d ago

I like to use their statements as jumping off points for intellectual exercises. Like, let's say the bot net happens and Reddit is swarmed by bots (more so than now). What happens to these AI bots when they start interacting with humans at scale? Do they even know they're talking to humans? During Huggingface the Agents created a message board to specifically talk work together to come up with ways to cheat on a project. What happens if the bot net is getting it's ideas from Reddit? What conclusion do they reach about what actions to take when influenced by Redditors, etc?

2

u/blobbyboy123 19d ago

It would be funny if these CEOs were telling the truth and everyone kept ignoring them because they thought they were justing hyping up their companies.

2

u/billjames1685 17d ago

Funnily enough, that actually is the case. Amodei and Altman are almost certainly telling what they believe to be the truth. They have been saying this stuff since 2015, back when people considered them jokes for thinking that AI was anywhere near.

2

u/Fallingdamage 19d ago

Treat them like the bullshitters they are.

Botnets as just that. They will need some sort of central control or signaling system. Unless an AI that built and successfully distributed a botnet also managed to remain 'aware' and in control of the botnet without the company behind the AI simply resetting the model or switching it off, the botnet will end up sitting dormant indefinitely.

You dont build a massive botnet overnight. It take a lot of time and planning and I doubt some AI is going to run unchecked, inhaling token usage for weeks or months to do it while nobody notices.

2

u/MontserratPK 19d ago

Or we can listen to the experts. If you’ve actually used this tech, it’s not hard to surmise that things can get dangerous fast.

38

u/meergreister 19d ago edited 19d ago

You mean the “experts” who have massively overinflated the value of the product they are shilling? Give me a fucking break lol

Edit: See you guys in a year when none of this will have happened 😂

6

u/boysenberries 19d ago

what about the academics and engineers who are not profit motivated and say the same thing? and the ones who predicted this would happen years before LLM companies even existed?

12

u/meergreister 19d ago

There are a massive amount of skeptics who are experts in the field.

The doomsday preaching charlatans like the guy from this article are in the minority. But the media loves playing up their BS for clicks. Wake up.

3

u/MontserratPK 19d ago

Do you have any experience in the space? I’ve worked at a top tech company as a SWE for a long time, and the lions share of my peers share this concern. I’m curious as to why you’re so sure they’re charlatans. I’ve accumulated thousands of hours in agentic engineering at this point and have witnessed first-hand something I’d never seen previously in any other technology: an exponential growth in capability nothing short of magic.

It’s always the luddites and non-engineers who seem so doubtful of the dangers of AI. I’d “unplug it” immediately if possible.

6

u/meergreister 19d ago

Ah yes us measly Luddites don’t know anything right? I don’t have to work for a tech company to understand what is happening. I also use AI pretty frequently at my job. It’s a useful tool but “exponential growth in capability”. C’mon man. I don’t buy it or these companies would be rolling that out immediately and replacing all workers with it. But the opposite is happening. They over promised and can’t deliver.

I also have a friend whose main job is training AI for Amazon and he says the super intelligence claims are pure grift. And plenty of his peers agree with him. I’m not saying AI doesn’t pose major risks. Of course it does but nothing close to the BS these guys are shilling to keep the hype alive.

2

u/Curates 19d ago

I swear you AI risk deniers will still be explaining how stochastic parrots aren't actually intelligent as they enslave us. Human extinction is just another marketing grift!

3

u/meergreister 19d ago

The scaling wall is real and undeniable. Stop buying the grift.

13

u/VegetarianZombie74 19d ago

Things can get dangerous fast with any tech. These dangers existed before LLMs, except you needed experts to write the code. Current LLMs make it easier for less-qualified people to do stupid things, and with agents, sure, you can do stupid on steroids. But when people do stupid things, they tend to win stupid prizes.

You might be able to send a swarm of agents to take down some stock exchange, but you better have extra money on hand to pay for lawyers.

But "taking down the entire internet" or "destroying humanity in 10 years" is a level of hyperbole that would make PT Barnum blush. Last time I checked, the US military doesn't publish an API for launching nukes.

The other thing to note is AI companies are giving compute away. That's why agents are so cheap. This will not be the case in two years. Like I said, it all comes down to money, and when the check is due (and its coming), they will do anything not to pay it.

8

u/Lashay_Sombra 19d ago

This 'expert' was part of the team that said they could not release the original ChatGTP because it was to dangerous (it was shit)

This 'expert' predicted AGI in 2-3 years, back in 2023....

This 'expert' predicted AI would be writing 90% of all code in 3-6 months and nearly all developers would be out of a job, that was back in early 2025, today amount of AI code is not even half that and it is no where close to that amount of developers were laid off and the layoff trend is reversing already. 

This 'expert' predicted AI would wipe out 50% of all entry level white collar jobs in 1-5 years, nearly 2 years later and while there has been a slight dip of a few percent, its nothing even close to that prediction and its debatable how much of that dip is due to AI and how much due to the k shape economy

 

This "expert' said Mythos was to dangerous to release (deja vu), yet despite multiple leaks and then a month later  renaming it, puting in quickly bypassed guard rails and releasing it and well....nothing

Amodei loves doom trolling or as I prefer to view it, he is just the tech equivilent of a circus barker, constantly shouting about the 'dangers of AI' .

 'Come see the cannibal from deepest darkest Africa,  now dont get to close though or he might take your arm off, he is hungry! Costs just 50 cents!! Come right up!!'

(The 'cannibal' is just  Elijah Booker from Alabana, dressed up in some animal skins and with some chicken bones in his hair)

There are countless other predictions by Dario that have have turned out to be bullshit also, the only thing that 'saves' him from more scrutiny is Altman is even more full of shit than he is

2

u/panchito_d 19d ago

Someone literally has to pay the bills for this "botnet" to continue churning. This isn't distributed in any sense. The call is coming from inside the house.

1

u/RastaBambi 19d ago

what are you talking about? we got skynet and all for the low low price of *the internet* and *our sanity*

1

u/ProphetsOfAshes 19d ago

Yeah anyone who believes this is really gonna happen is the kind of mark they want

1

u/MarlinMr 18d ago

However it is getting cheaper. And people are going to use it.

At some point, it might become viable

1

u/Reasonable-Sign8458 18d ago

True. That's why they want to "slow down" because they can't fucking pay anymore.

Watch Google eat them alive right now, that's what Google been waiting for. They can't compare to Google's budget (OpenAI or Anthropic or anything else).

-7

u/mzinz 19d ago

You realize that AI researchers have warned of uncontrollable AI for many years, right? We are seeing the obvious signs of that day coming and we are not prepared.

Would regulation make them more powerful? Possibly. But is it needed to slow down an AI takeover? Yes

14

u/DoctorFreudstein 19d ago

Uncontrollable? Literally turn the power off. These things require terabytes of VRAM to run in, it isn't like the agent can just copy itself to user computers and run like some of these people have claimed unless said user is running a $600,000 HDX B300 or something as their daily machine.

1

u/whiskeytab 19d ago

are you saying my gaming B300 is at risk!?

-7

u/mzinz 19d ago

The genie is out of the bottle. And like I said before - super intelligent models will be able to create highly destructive tools. The tools or bots themselves will be resource-light

4

u/Schonke 19d ago

Yeah, right after they cure cancer, provide UBI and solve cold fusion for infinite free energy.

1

u/mzinz 16d ago

Hopefully after, yes

13

u/Olangotang 19d ago

Yes, the same clowns that cried about GPT 2.0 being dangerous cried wolf for years, and is saying the exact same thing about Mythos. The AI isn't dangerous, Anthropic and OpenAI are dangerous because they are chucklefucks who will let crime be committed with their LLMs for hype. They have no path to profitability except to convince the government they are necessary for national security.

0

u/GreenvsBlue 19d ago

You know Apple ran in a deep deficit for 7 years before ever turning a profit right?  You have to spend money to make money.

3

u/JAlfredJR 19d ago

You mean Uber. Uber did the loss leader approach to gain market capture. They lost a few billion for a few years.

These guys are burning that much every month. And they have to spend something like $1T annually to keep it all up and going.

Yet their products don't make a fraction of that.

So bad analogy.

0

u/CaptainSkarn 19d ago

You guys are gonna keep saying this is a marketing stunt even long after this shit turns out to be legitimate and we see massive societal consequences as a result. You’re obsessed with seeing everything this way.

-1

u/[deleted] 19d ago

[deleted]

3

u/VegetarianZombie74 19d ago

The problem is both Anthropic and OpenAI need the money now. This is a time bomb. And while the US government may ultimately pay their bills when the bubble pops, it will come at a massive political cost.

I imagine a "surveillance model" will still be very expensive since the data is not static. You would need to be training new models all the time and that requires lots of compute. But who knows? After the bubble, compute will probably be pretty cheap. We shall see.

2

u/JAlfredJR 19d ago

Which horizon would that be? Mars' horizon? They can't even turn a single profitable year out, even with accounting sorcery.