Anthropic CEO warns of AI-driven botnet 'swarm' taking over the entire internet — 'In 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet'

[u/Just-Grocery-2229]

https://www.tomshardware.com/tech-industry/artificial-intelligence/anthropic-ceo-warns-of-ai-driven-botnet-swarm-taking-over-the-entire-internet-in-6-12-months-such-a-swarm-could-be-capable-of-taking-over-the-entire-internet-with-a-persistent-botnet

[u/DezXerneas]

This is literally their entire MO. Every 3 months they release a press release saying "Our AI is so awesome, it's about to destroy the world, so we won't release it(to you, the government and people who pay us get full access)".

I won't say their models aren't great, but they're just LLMs. An LLM cannot become a real swarm or agi, you need someone telling it to do shit, but cloudflare is now relatively good at blocking them out. Yeah a Fable tier model can take 'manual' control of a browser and simulate real clicks and stuff(which also gets blocked by cloudflare btw), but does anyone even know how much money that actually burns to run?

[u/Economy_Kale5580 → u/DezXerneas]

Hey pal, that makes sense and isn’t scary, get outta here

[u/wonkytalky → u/DezXerneas]

I'm honestly just sick of being locked out to a growing list of websites, including one github alternative hosting a couple repos I've needed to use, because apparently my IP is part of some block in which someone was identified as a bot or something. I have to switch off WiFi on my phone and use that to get to the blocked stuff now, which as you can imagine, isn't entirely helpful for dev work on a laptop.

Thanks to some LLM zealot somewhere nearby, this legitimate human is being locked out. Fun times!

[u/DezXerneas → u/wonkytalky]

Yeah, had that happen to me for a while. Annoyed the fuck out of my ISP, trying to get them to fix it, but eventually I just switched providers.

Especially since I block a lot of tracking data that would normally identify me as a real human, so I already get flagged as a bot, and then there's some asshole letting his agents hammer the internet living near me.

[u/wonkytalky → u/DezXerneas]

Thank you for legitimizing my experience. It's a little better knowing that my blocked ass at least has some company, lol

Edit: I also block a ton of telemetry, so I see those "are you a bot?" pages a little more often than the average bear as it is. The Anubis system or whatever it's called that's used for the FreeBSD forums is the absolute worst. But yeah, whatever is going on at Codeberg, for example, is definitely blocking my IP range and not just my browser. I happened to swap out some networking gear recently, which triggered an IP change, and yup, issue's still there from a completely different machine and browser. New ISP is coming in a few months though and I'm for sure switching, so I'm assuming that'll solve that issue (for awhile, anyway).

[u/bloodychill → u/DezXerneas]

The burning money is the part that doesn’t math in their vision. Like, fine, your LLM toolchain now makes it comically easy (if ultimately extremely expensive) to create software and inauthentic content, but with no increased demand, and in some cases reduced demand since inauthentic content scares off the real people. Whoops.

[u/CharlieandtheRed → u/DezXerneas]

I work in cyber security and last week I saw a 72M/hour attack from every single country on earth on a small target. I know this doesn't mean a lot to others, but this scared the shit out of me. That's a wild wild number usually reserved for big targets.

[u/thefuckevengoingonan → u/DezXerneas]

An LLM cannot become a real swarm or agi, you need someone telling it to do shit,

I don't think anyone is talking about agi. The issue is that these models exist that are very capable to hacking into ass sorts of shit even just as a way of solving a non-hacking related issue. If someone with access to one of those models did give it directions to hack data centers and spin up copies of itself.
Yes you can give an llm a copy of itself and access to your AWS account and tell it to spin up multiple instances of itself to do various tasks. This is something that is possible for a long time and nothing special. If you gain access to a model that can hack other people AWS account and other data centers, then we fucked. This capability already exists. It's just no one with access has turned then to the task of spread and destroy, yet.

[u/DezXerneas → u/thefuckevengoingonan]

Sure, that's possible, but you're taking aws security way too lightly, and I'm pretty sure spinning up more instances of a LLM isn't even close to the most profitable thing you could do if you found a reliable way to hack into AWS accounts.

Also, a model capable of doing that isn't gonna fit in a normal person's aws account, and there's spending limits and shit so you'd have to compromise banks as well to pull this off. I don't think a basic AWS instance can run fable(or equivalent) at tps to be any danger at all.

Also, there's a reason these AI are only being used to test internal code and not unleashed on the internet, and it's not just the cost. They're genuinely good enough once you give them enough context about the code, but we're not even close to a digital grey goo tier event. Worst case, aws shuts down infected/compromised clusters using their signature.

[u/thefuckevengoingonan → u/DezXerneas]

what on earth are you basing that on? you can run kimi k3 Q8 for ~$150 an hour. i could spin up an instance right now on my aws.

but you're taking aws security way too lightly, ima just assume you have never used aws at this point.

[u/RafaelWanderpfad → u/thefuckevengoingonan]

OpenAI likely spent millions of tokens during the HuggingFace hack. This is not about spinning up one instance of Kimi bro lmao. 

[u/thefuckevengoingonan → u/RafaelWanderpfad]

You need one instance to hack one thing, one account. Then you have 2 instances. The agent/models can also spin up and make use of smaller models. As is the norm for usage now. Simple tasks go to simple models on , hard tasks to more expensive models. Orchestration/C&C can be done decentralized as is already done with botnets. (ie, instructions are encrypted and published to any public blockchain), all agents running all models get instructions and complicate via the blockchain. None of this is new stuff except that LLMs are just way faster then humans at doing it.

[u/DezXerneas → u/thefuckevengoingonan]

Bro k3 Q8 is not hacking aws. You can run it, and eventually we might have something that could run in a normal container and do it, but imo at this point these agents are much better suited as just a replacement for a scam call center or something and to get people to install ransomware or a RAT for later.

Also, I think most aws accounts are capped to wayyyyyy below $150/hour lol

[u/thefuckevengoingonan → u/DezXerneas]

k3 at running at q8 is close to fable in capability. i use it as a comparison for VRAM usage and cost to run. no, i don't think k3 will be the model that hacks the world, i don't think it is capable. never the less it has shown to be capable of find and using 0 day vuls. the full k3 FP16 would cost you 5.6TB vram. ~$300/hour. that is what we should expect as requirements for this level model (mythos). k3 is not there but it's close and at current rate of improvement i completely see they are already there with closed/unreleased models. or i maybe i am just crazy and LLMs are just 100% scam with zero ability to write code at speeds surpassing human speed in every way.

[u/RafaelWanderpfad → u/thefuckevengoingonan]

They can write code the same way they can write other languages. It’s the one thing LLMs truly excel at. Doesn’t mean they will become sentient, take over the internet, annihilate humanity or all that marketing BS. 

[u/thefuckevengoingonan → u/RafaelWanderpfad]

i am not talking about AGI or sentient or whatever. That is not required at all for anything I said. i am also not talking about anything making any decisions at all. I am talking about people setting up and using these tools to create botnets, like hackers have been doing since the start of the internet. It's just LLMs are millions of time quicker then a human.

[u/Longjumping_Ad_424 → u/thefuckevengoingonan]

Would you worry about important online infrastructure being hacked? Like a power grid or some sort of electricity or water supply / filtration system. I feel like malicious hackers or people in control of soon to be great models will unleash a ton of pain for the public.

Isn’t that a reasonable thought in the next few years?

[u/thefuckevengoingonan → u/Longjumping_Ad_424]

There are many many things that could happen with that sort of power. Hack banks, infra, blah blah blah... But all those things are for nothing if someone gets access and instructs to just fucking burn it all down. Seems there is a lot of people who would be will to do so. Edit: what i a saying is that the biggest "end game" threat is one day we all just wake up and there is not internet, at all, worldwide. No, AGI required, no super intelligent or quantum computing, no targeting or specific goals, just "Spread until x% of internet ifra comprised and then rm -rf"