r/technology • u/Miserable_Phase_2519 • 14h ago

Artificial Intelligence Florida woman used Claude as a diary, then Anthropic reported an entry to police

https://www.techspot.com/news/114091-florida-woman-used-claude-diary-anthropic-reported-shoot.html
13.5k Upvotes

97% Upvoted

1.4k comments sorted by

View all comments

Show parent comments

339

u/SimoneNonvelodico 14h ago

I mean I think it's wrong that such lawsuits exist and aren't dismissed out of hand. I swear if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime. Companies shouldn't spy on their customers, not even for the greater good!

238

u/MournfulAccount 14h ago

The court case is happening because OpenAI was aware of the Tumbler Ridge shooter's plans and did nothing. Which is something they admitted to.

93

u/riyehn 12h ago

To be specific, OpenAI has openly admitted their company leadership was briefed on the shooter's plans and made the decision not to report it. We're way past the question of bots flagging chats to outsourced service agents here. This was a fully-informed decision made by humans at the very top.

11

u/harmoniaatlast 8h ago

Why would they not report it????

1

u/tonyrocks922 2h ago

Anyone at OpenAI who had a conscience went over to Anthropic. Only the shitheads are left.

-3

u/Remarkable_Cause1384 7h ago

read the article.

200

u/PunctuationsOptional 13h ago

The point is they shouldn't be aware.

The software shouldn't be reporting back a customer's private data and the company shouldn't be storing it

234

u/Marcoscb 13h ago

The point is they shouldn't be aware.

But they are. So either they aren't allowed to know what you write and are then not responsible for it, or they are and they should be responsible. The point is that they don't get both the benefits of knowing everything you write and not being responsible for anything.

61

u/TrulyOutrageous42 13h ago

Absolutely, I think the overarching argument is that they should NOT be aware of what we're writing when it explicitly states it's not using the data to train models. That rather clearly implies it's not reading or saving or sending it beyond the instance, but it is.

29

u/THEBAESGOD 12h ago

When you use our services for individuals, such as ChatGPT and Codex, we may use your content to train our models. You can choose whether your conversations help improve our models. To opt out, turn off Improve the model for everyone under Settings > Data controls in ChatGPT, or select Do not train on my content in our Privacy Portal.

Claude is Opt-In but it says it’ll save your data and use it for training if “ Your conversations are flagged for safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Usage Policy…”

2

u/Piranata 11h ago

It should be more direct, like the error you receive in Linux when you try to use sudo when you're not in the sudoer list: "This topic violates our Usage Policy, this incident will be reported".

1

u/IrregularPackage 7h ago

oh, so it’s not opt-in at all.

4

u/MovieTrawler 11h ago

Yep, they've opened Pandora' Box by reading, analyzing, storing and selling our private data. You can't do that and then cherry pick what you see.

1

u/slfnflctd 10h ago

This right here.

But we have been long past this discussion for at least a dozen years. Ever since Snowden and the collective yawn which followed, it has been over. Really since the 'patriot act', but that's what laid the groundwork for the inescapable surveillance we're under now.

There are so many incentives for data gathering at this point, it's nuts... the scales have been tipped against users for a long time. It is vanishingly rare to find an organization which provides full transparency of exactly how they handle your data, along with a clear pathway to total obfuscation/deletion. The increasing reach of law enforcement is most of the reason why.

I agree with your last sentence so much, I just don't know what can be done about it, especially with the whole Section 230 thing. It deserves more discussion, though, for sure.

3

u/ShareGlittering1502 11h ago

What do you think Reddit meta Microsoft Google etc etc are doing?

It’s their data. We are the product.

15

u/REPL_COM 13h ago

How else will they continue to steal data to train the next model with… /s (but also, yes they do do this and it should be illegal)

35

u/immaownyou 13h ago

Idk, I think its comparable to therapists being required to report dangerous activity even if patient confidentiality is supposed to exist. We can't just keep basing how we react to new technology on how we used old tech

25

u/ashgs872tbhjs 13h ago

No, we should absolutely not treat fancy text generators like they're therapists. That's worse. You get how that's worse, right?

21

u/juliasct 13h ago

Yeah, we shouldn't, bc we should hold therapists to a higher standard (of privacy and everything). I think we can agree privacy with consumer AIs is less important than with a literal therapist.

15

u/Omikron 13h ago

Yeah they should be held to a lower standard, if you're telling AI you plan a mass killing it should 100% report you to authorities.

1

u/Megneous 12h ago

Anything you tell an AI should be 100% in confidence, because it's a tool you should be able to use and have privacy with, no different from Microsoft fucking Word. OpenAI, Anthropic, etc shouldn't be able to see what you're saying to it.

11

u/booze_clues 12h ago

Then get a local model. Why would you ever expect that data you voluntarily send to a company and agree to allow it to view would be kept private? If you want that, most companies offer that type of license for a increased cost.

6

u/wolfmourne 11h ago

Lotta future school shooters in here seems like.

-4

u/Megneous 12h ago

I literally train local models. I'm a member of a group on Hugging Face that is part of a large collective of around 60 groups that develop small language models on consumer hardware.

3

u/booze_clues 12h ago

Good, free our data from the tyranny of the tech bros

→ More replies (0)

1

u/brainparts 8h ago

Then it doesn’t have value to them. These corporations are not interested in providing helpful tools to regular people.

1

u/WhoPoopedMyPants420 12h ago

Except there's no way to do that without 100% of your data being scrutinized. Back when the Patriot Act was passed, anyone who was opposed "wanted the terrorists to win" or whatever, and our right to privacy was compromised for the sake of a sliver of extra safety.

The question isn't whether someone has an obligation to report if they hear a mass murderer's plans, it's whether they should have had access to those plans in the first place. If I find out my neighbor is plotting to kill his wife I have an obligation to report it to the police, but if I found out while hiding in his closet then that's kind of an issue

2

u/powercow 8h ago

without 100% of your data being scrutinized.

By AI that flags certain conversations.

Humans cant 100% scrutinize AI chats

in a way its like a gun shot microphone that many cities have. they could hear lots of convos but the system only matches the wave forms with gun shot noises.

1

u/WhoPoopedMyPants420 8h ago

"This is a special type of surveillance that only intrudes on the bad guys' privacy" okay man thanks I haven't heard that one before

2

u/SimoneNonvelodico 10h ago

Exactly. Funny enough, if e.g. someone was brought into court with evidence acquired by illegally bugging their house they'd walk free, even if the evidence was incontrovertible that they really were planning a murder, because illegally obtained evidence is inadmissible. If personal data was treated with the same standard this wouldn't even be a thing. You couldn't go and read someone's data without a warrant and therefore probable cause.

1

u/Omikron 8h ago

You're free to not use it

1

u/WhoPoopedMyPants420 8h ago

I don't. I'm also free to still have my own opinion on the societal compromise between safety and surveillance

2

u/Omikron 7h ago

You don't seem to be compromising though

-2

u/The_F_B_I 11h ago

plan a mass killing

Reddit told me you used this phrase, put your hands behind your back!

5

u/HuntsWithRocks 12h ago

I think they were saying that, since the company "heard" the user planning a dangerous crime, then just like a therapist, it is probably good that it got reported.

The other talking point is about if technology should listen to you at all and compared it to phones. It's a messy bowl of soup. Part of software is anonymous user data and measuring to ensure things are functioning. Without monitoring, software is really gonna struggle in ways. It's not physical. So, logs and events and all that are what you got. Then, thinking about an LLM where it's basically "fancy search engine" it's hard for them to improve it if they aren't paying attention to how it's being used and where it's falling down, but i also don't like that they keep all the info and it's been shown they're more than just reading it for security purposes. So, they're not doing it "right" for sure.

At the end of the day, since they're eating all our information anyway, while they're eating it, I do appreciate when they call out would be murderers.

2

u/Fine_Sherbert3172 11h ago

No, we should absolutely not treat fancy text generators like they're therapists. That's worse. You get how that's worse, right?

As soon as I saw the capabilities that even the first gen LLM's had I knew we were in for problems.

Think about the bottom percentile of the population. This chatbot will be by far the most intelligent voice they hear, maybe in their adult lives, talking about the things they care about. They will be awestruck by it and think they have discovered some great hack to life. This is as bad as people getting addicted to Perdue grade opioids.

1

u/MovieTrawler 11h ago

And what's worse is that it is just SO sycophantic.

1

u/abraxsis 11h ago

With as costly as mental health can be, coupled with the continued stigma of getting help, it's the only outlet some people get. For a lot of people they just need an outlet to dump their day-to-day shit, Id rather it be some LLM text generator than that pent up frustration get unleashed on their spouse, their kid, or some unsuspecting motorist via road rage. And that's coming from a therapist.

That being said, OP didn't even suggest what you are suggesting. He simply stated the same level of mandatory reporting should exist for LLMs. In the state of TN, everyone is a mandatory reporter, even non-TN residents when they are inside the state. That doesn't mean we consider them all to be therapists.

1

u/Elan-Morin-Tedronai 11h ago

They can both identify threats to commit mass murder, in this case the algorithm merely sends it to be reviewed by a person, who then decides whether its a real threat, but mass murder is bad and we should take threats seriously. You shouldn't use AI as a therapist, but that's for completely separate reasons, not because its cool to threaten murder. I think the only time a corporation or individual hearing a threat like that isn't required to speak out is Catholic confessional, and the priest is still obligated to try to avoid the harm that might be caused, even if he can't communicate what the potential harm might be. Even if AI companies just deleted or kept everything else anonymous I don't see why murderous threats shouldn't be flagged, saved, and passed on to the relevant authorities.

1

u/feor1300 10h ago

No one's saying we should trust them to evaluate a person's mental health, but it's comparable in that a therapist shouldn't be recording what's said in their sessions, however the therapist has to process what's being told to them in order to interact with their patient. If they're told something that's an obvious admission of a crime (or the intent to commit a crime) they are expected to report that to the authorities, regardless of patient confidentiality.

Likewise, AI shouldn't be recording our interactions with it without our consent, but it still has to process the text we send it, and if that text includes the admission of a crime or the intent to commit a crime, then they should be responsible for reporting that information to the authorities. There should in fact be an even greater expectation on them because there is no expectation of confidentiality with a random computer on the internet.

1

u/Upstairs_Weird_760 9h ago

Thats not what he said. He said the rules should be similar. Not that they are therapists

-3

u/immaownyou 13h ago

Thousands of people already are lmao, its far too late to disagree

-6

u/LieutJimDangle 13h ago

you have a choice to go to a therapist, we are not being given a choice when it comes to be monitored 24/7 on all platforms or just driving down our street.

10

u/DemonKing0524 13h ago

You are given the choice to engage with those platforms. No one is holding a gun to your head and forcing you to use ChatGPT.

Edited to add, and stuff like the flock cameras is of course a totally different ball game and conversation

-3

u/LieutJimDangle 13h ago

choice? you think this is limited to chatgpt?

do you have a choice to use the internet. to do a google search. to purchase something online. to write an email. to make a phone call. to manage your bank account online. to save something to the cloud. to have license plate readers mounted on every bridge, traffic signal and street pole. to have facial recognition and private camera networks on every corner. to use a video doorbell. to be tracked by drone with thermal imaging. to use your credit card. to file your taxes. to have them own your biometric data. to track your travel movement record on every flight or highway toll booth.

what choice?

2

u/DemonKing0524 12h ago

Where did I say it was limited to chatgpt??? And yes you absolutely do have the choice to use the internet lol no one is holding a gun to your head and forcing you to do that either.

-3

u/LieutJimDangle 12h ago

ok so you are just being obtuse, to pretend you can survive in the modern day without being online at some point. or maybe you are truly clueless who knows.

2

u/DemonKing0524 12h ago

You actually can, and paranoid people who actually care enough to do it all the time. Its just most people prefer the convenience rather than actually putting their money where their mouth is.

→ More replies (0)

2

u/notquitecivilized 12h ago

But in Tumbler Ridge it wasn't just that they said they were going to commit a shooting, they used the AI to plan it.

4

u/rickg 13h ago

Then don't use hosted software. And don't complain if someone you love has their head blown apart by bullets in an attack that could have been prevented, but wasn't because of your principles

1

u/BravinatorLX2 11h ago

and no one should listen when a software made to make up stuff reports something

1

u/Wiseon321 11h ago

Someone posting their desires to mass-end-lives and posting it on some forum, that is policed, the idea that THAT should be free of policing is nonsense.

You wanna plan some mass killing spree and not get man handled by the feds? Do it on pen and paper. NOTHING is safe on pc anymore.

If they know, they should report it, end of story.

1

u/Trevski 11h ago

How shouldn't they be aware? It's not private data, the person willingly sent their data off to the company for the AI to formulate a response to. I don't follow the logic that that could, would, or should be private.

1

u/funguyshroom 11h ago

The moment your private data leaves your device unencrypted it's no longer your private data.

1

u/Zap__Dannigan 10h ago

They should be aware. They made a product that is made to harvest your information, why are you upset it's harvesting your information?

1

u/Visible_Device7187 6h ago

So how does AI work in that situation? Sounds just like a ban on AI since it can't function without customers information.

1

u/fakemoose 3h ago

They can’t both have safeguards against inappropriate chat bot interactions, use context from anything you have previously said in a chat, and not collect and save customer data.

1

u/Voodoobones 13h ago

Then priest should be liable for confessions.

0

u/Megneous 12h ago

They shouldn't be aware, period. All communication to and from the servers should be encrypted, so OpenAI, Anthropic, etc, has no access to it. Customers should have privacy, like NovelAI has.

3

u/UnexpectedAnanas 11h ago

I fully support E2E encryption for pretty much everything you can use it for, but....I don't see how that even makes sense in the context of a chatbot. They literally would have the keys by necessity of being able to process the prompt. It'll be encrypted in transit (and almost certainly already is via SSL), but that's about it.

They still need the keys, so nothing you send to them could be private in an E2E sense.

0

u/Megneous 2h ago

They literally would have the keys by necessity of being able to process the prompt.

That's simply not true. NovelAI has full encryption. What users are saying is not individually logged. All that is stored client-side. They're incapable of seeing what their users are saying to their AI models, even if they wanted to see.

24

u/RetardedWabbit 13h ago

Unfortunately, in this case it would be like if the phone company was already listening to everything, storing them, and categorizing/checking them all the time anyway. Having files of "bombing and mass shooting plans" stacked up somewhere regardless. 

They're already spying as much as possible "for the greater good" of profit (personal data and training data to use/sell), so the government wants "certain" info too. It's debatable if the company should do that, but harder to debate once they are that they shouldn't turn over the "REALLY sounds like murder planning" conversations.

57

u/Warm_Month_1309 11h ago

if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime

My standard wouldn't be "you are required to listen in and report everything"; my standard would be "if you are choosing to listen in, then you've also chosen to make yourself liable when you don't report something you should have".

2

u/Z00111111 8h ago

"Claude, delete any logs of us reading transcriptions of criminal or planned criminal activities"

-1

u/CPSiegen 9h ago

Who defines what "should" be reported, though? "Should" these AI hosters report, say, domestic abuse even when they have no idea if the user is making stuff up? What about child abuse if a kid says their parents are monsters? Should they report things ICE thinks are crimes? Should they report dissident behavior in countries that label dissidents as terrorists? Should they report unregistered business activity in countries that take state controlled commerce very seriously? Should they report homosexual thoughts in countries that outlaw homosexuality?

At what point are we promoting anyone ingesting our data to act as thought police on behalf of the worst government policies? What infrastructure will we have to ensure they aren't selectively reporting certain kinds of activity and not others (for instance, reporting things that ingraciate them with the current administration or only things that don't impact their profits)?

7

u/Warm_Month_1309 9h ago

Who defines what "should" be reported, though?

Their legal counsel, and then later if someone disagrees, a court on a case-by-case basis.

At what point are we promoting anyone ingesting our data to act as thought police on behalf of the worst government policies?

I'm not promoting that. I'm offering two alternatives: either do not read user data and remain ignorant of its contents, or do read user data and be responsible for your knowledge of its contents. What these companies cannot do is claim the benefits of both.

2

u/Bauermander 9h ago

Your alternatives are not realistic. AI doesn't work without reading the content. You know very well that if they have legalized access to user conversations, they'll use it to do a lot more than to track mass shooters. They're probably gonna track them anyway in usa, they're just not able to use it as a proof in legal cases.

1

u/CPSiegen 9h ago

I'm arguing that the latter option is the exact smoke screen that lets them get away with doing both.

You see that now with companies requiring ID verification. They're promising that it's for the purpose of rooting out bots and predators. They're promising that it's not going into a database to tie all your information together more strongly. They're promising that they're deleting your ID info as soon as you're verified.

And they've been caught lying about each and every one of those promises. The idea that they're acting in good faith is the lie. If you give these companies an option to spy on all your data in the name of security and law enforcement, they will never act in good faith on that premise. They will absolutely still spy on you as much as possible and they will absolutely shirk and outright bury any responsibility to act on what they find, so long as what they find doesn't directly help them to report.

The discussion needs to focus only on the option that we can regulate mandatory and invisible data harvesting out of existence. Anything else will be twisted to do exactly what you don't want.

1

u/Warm_Month_1309 8h ago

lets them get away with doing both

There are too many novel lawsuits currently working their way through our courts to conclude that.

0

u/SimoneNonvelodico 11h ago

Well, but it's arguable whether they are "listening in". They're storing data but no human is ever looking at it. That said, ok, suppose the user was paying for the subscription that allows you to turn off data retention - then would it be ok to drop the requirement? Or would someone still sue and complain?

11

u/Warm_Month_1309 10h ago

They're storing data but no human is ever looking at it.

"After a human reviewer examined the statements, they were reported to police."

It's not arguable whether they are listening in. You explicitly given them permission, they say they do it, and they report things you say to the police. They are listening in.

9

u/about0 13h ago

But they knew and know. If they couldn't, it'd be a completely different story

59

u/Visible-Task-2798 13h ago

It's not spying when you hand over your data willingly. The LLM needs to record your questions to compute them and answer you. Telefones never needed and never will need to record your conversation for you to use it. That's the difference.

Using LLM and complaining about privacy = eating meat and complaining about killing animals.

-3

u/mroada 13h ago

No, it does not need to record it. It can use it and then discard it when the question is answered.

Similarly your conversation is "recorded" as it's being transmitted to whomever you're talking to, but the operator doesn't save that record anywhere.

5

u/DemonKing0524 13h ago

Your phone conversations are not recorded when they are transmitted. Unless of course, you've been wire tapped by the police, but that needs to be done purposefully, it is not the default for how phones work.

And in order to use your prompt, it has to record your prompt internally so it can parse it and provide a response. Just like with literally everything else on the internet, once you post it, send it, etc and its out there, even if it is deleted, there will always be a copy of it somewhere on some server or in some database. If you don't like that, don't use the internet essentially, and certainly don't use the AI models.

18

u/NoPossibility4178 13h ago

There's no "default" to anything, it's decided when things are conceived. They chose to log all your chats, there's no higher technological reason why they had to, they wanted to do it so they did. For phones, wire tapping is there, it just was decided that it wouldn't be done, probably because they figured it made more sense to let people have conversations over the phone in private rather than log them.

7

u/entertheclutch 13h ago

Anthropic when they "decided" the multi-head attention architecture must retain prior tokenized inputs and outputs to minimize training loss because i guess it will sell ads better and let them report schizos to the cops, and not because otherwise you basically get a 1-question-1-answer machine with short-term amnesia like the original (physical) mechanical turk.

Like people have developed a system for multiplying effectively random trillion-rank matrices to produce answers for math problems that have gone unsolved for centuries. You sincerely being like, 'I know exactly what they were thinking when they "conceived" (lol) of this' is pretty funny. i'm not sure they did lmaoo

-2

u/NoPossibility4178 13h ago

Why don't you ask it the likely reasons why it was made the way it was and what were the technical limitations of other options and why they wouldn't be chosen?

3

u/entertheclutch 12h ago

I just explained them

7

u/evildeliverance 13h ago

You're being obtuse. Not storing your chat would mean all context would be lost after every message.

-2

u/NoPossibility4178 13h ago

They could encrypt it so they can't actually read it.

9

u/evildeliverance 12h ago

Encryption wouldn't change a thing. It's not like Signal where they can use end-to-end encryption so the company CAN'T see the messages. The AI NEEDS to be able to decrypt the messages for context. Which means the company has the keys and has the choice to read the messages if they want.

Choosing not to supervise got them a lot of shit from the public when the AI gave harmful advice and when people used AI to help them do bad things.

0

u/SimoneNonvelodico 11h ago

No, the key could absolutely sit just with the user and be hidden from the company. But then of course governments would ask the encryption to have a backdoor, because that's exactly what they're doing with other encrypted things.

8

u/Kefrus 10h ago

No, the key could absolutely sit just with the user and be hidden from the company.

How the fuck do you expect a model to provide output if user encrypts a message and the company cannot decrypt it?

edit: I just realized you wrote like 10 comments here while not even understanding the concept of public and private keys xdd

→ More replies (0)

3

u/DemonKing0524 12h ago

Humans don't read it initially. They have an automatic system that flags content like this that may need reported, and only then does an employee review it to confirm its not a false flag and send it through the proper reporting channels.

Its quite simple really. Don't plan something like this that causes a flag to be raised and its incredibly unlikely a human will ever lay eyes on your data.

-2

u/NoPossibility4178 12h ago

lol

Just like with Flock, right? Just don't commit a crime and no one will look through thousands of cameras to search your vehicle and track you down.

5

u/Illadelphian 11h ago

Flock is doing this WITHOUT your consent just for existing out in public.

LLMs need to process what you are saying to keep context of the conversation and they are 100% opt in. They aren't picking up your conversations from out in public and then doing something with that information without your consent. You are choosing to type into a tool that you literally know uses what you say to help formulate responses and explicitly tells you it's doing so.

They are nowhere near the same thing fundamentally.

4

u/DemonKing0524 12h ago

Again, very fundamentally different than how LLMs work.

→ More replies (0)

4

u/jesuspajamas15 13h ago

They are such different technologies. A phone for the most part is passive from the provider, they would need to make the decision to record and t would cost money for no benefit. LLMs require what is written into them to be processed and stored for at least a short period to function at all. They could forget everything immediately but that defeats some of the purpose of the technology.

3

u/NoPossibility4178 12h ago

You think there's no cost for them storing your chats indefinitely? You think they get deleted if you delete them and/or your account? Come on. They also would not need to forget anything, it just would have to not be accessible to OpenAI to look whenever they felt like it, but I guess that'd also cost money for no benefit to them.

3

u/Kefrus 10h ago

They chose to log all your chats, there's no higher technological reason why they had to, they wanted to do it so they did.

It's impressive how confident you are while you are just confused about the concept of a website having sessions that persist after leaving the page or logging from another device.

1

u/whofearsthenight 8h ago

Wait'll they learn about cookies or key pairs.

-1

u/DemonKing0524 13h ago

No, it was decided in a court of law thar we had a right to privacy, so yes because of that in the US it absolutely is the default. They need a search warrant to change that.

2

u/whofearsthenight 8h ago

Your right to privacy doesn't exist when you are willingly giving a third party your privacy especially when you agree to terms of service that outright state this. If I tell a buddy "hey I am going to do crimes" and then the buddy tells the police, we are nowhere close to a constitutional violation. If I send a text saying "let's do some crimes" the cops need a warrant to get that from me. If the receiver of that text tells the cops, obviously no warrant is needed.

1

u/NoPossibility4178 13h ago

Apparently your rights only extend to certain things.

2

u/DemonKing0524 12h ago

Well yes. That has always been the case. For instance someone planning a mass shooting has never had the right to actually kill the people they want to shoot, someone trying to modify a virus ti make it the next pandemic as a bio weapon doesn't have the right to kill people with it, and someone planning to attack their sheriff's office and potentially kill people doesn't have the right to do that.

Those are all examples of things OAI have caught users doing and reported. Im fine eith them reporting those things if it saves lives. It is exceptionally sad that you'd apparently rather those people died.

2

u/NoPossibility4178 12h ago

Yep, I'd say phone providers should overturn the law that gives you the right to privacy on that specific technology because a lot more crimes get planned over the phone compared to over chatting with ChatGPT. It's pretty sad you're not more outspoken about that.

-1

u/DemonKing0524 12h ago

You are still fundamentally misunderstanding the difference between the two technologies and how they work. And thats incredibly sad because it is very simple.

Phones have never needed to record your conversation to work. Adding recording surveillance to phones takes extra work. Its not a built in feature, and we do have a right to privacy over data that was never being collected to begin with. That was decided in a court of law years ago.

For LLMs to work, they have to take your text, and save it internally so it can be parsed nad understood by the model, so the model can even respond at all. The models literally do not work without doing that. That data is already being saved and processed because it has to be for the technology to work at all.

I have no problems with them flagging things like mass shootings or attempts to kill people when that data that is already being processed just for the technology to work at all gets processed. Again its sad that you'd rather the deaths.

→ More replies (0)

0

u/SimoneNonvelodico 11h ago

Well, but that's my point no? It's not about the technology being different or whatever. When it came to phones, a court decided that privacy mattered more. Today I'm saying they wouldn't make that choice because everyone has a fixation for snooping on everyone else, convinced that it's the only possible way to ensure we're all "safe" from any and all imaginable dangers. Talk about privacy and you'll be accused of being a terrorist or a pedo, because why would anyone else want to not disclose every single thing they do and say to third parties?

1

u/DemonKing0524 10h ago

Why wouldn't they? Phones weren't built in such a way that collecting that data was the default. LLMs literally can not function without understanding and processing the context of your conversation. It would act like an amnesiac whose memory resets repeatedly if it didn't save that context in its internal context banks.

Does someone who is planning a mass shooting actually deserve the privacy to finish planning it and actually commit it? I dont think so, why would you? Does someone who is trying to use chatgpt to create a bio weapon deserve privacy to finish that and potentially kill a bunch of people? I dont think so, why do you?

Like you guys act like theres a human sitting there reading every single chat you send to chatgpt. Theres not. A human doesn't lay eyes on something unless you trigger their security flagging systems for some reason. Like in one of the events I listed above.

5

u/ashgs872tbhjs 13h ago

Phone calls are mostly digital now, so it's exactly the same with the data being transient in memory. It doesn't need to be persisted somewhere retrievable or analyzed for purposes beyond the intention at hand.

3

u/DemonKing0524 13h ago edited 13h ago

The content of the phone call conversation can not be analyzed unless it is recorded. The record of the call of course can be, but the not the contents of the conversation.

4

u/lw5555 12h ago

You have the scenario wrong. This is like telling your buddy you plan to shoot a place up and him not bothering to report it.

2

u/SimoneNonvelodico 11h ago

I like how everyone here is up in arms screaming "anthropomorphism" if anyone suggests that AI might in fact not be just a stupid parrot, until we discuss whether it should call the cops on people, then it's "my buddy" and suddenly has all the moral and legal duties of a real human being.

5

u/Warm_Month_1309 10h ago

No one is saying the LLM should call the cops. People are saying that the human reviewers who read the chats should call the cops.

7

u/Friggin_Grease 14h ago

The Greater Good

1

u/Blue2501 13h ago

crusty jugglers

13

u/Trevski 11h ago

This isn't like using the telephone. It's like using the telegram. The logic is 100 years ago that the wannabe terrorist sends a telegram to a friend detailing their plans, what is the telegraph operator supposed to do? Do you expect them to transcribe your plan to massacre people and then sit idly by and let it happen?

The problem with your analogy is that the phone doesn't need someone/something to transcribe the message for you. It's not "spying" if its literally just how the thing fucking works. It's not "spying" when the telegraph operator transcribes your message into morse code, and it's not "spying" when LLMs process your prompts, its just the thing that's supposed to happen happening.

Defending using AI to plan a massacre is terrifying. Get help.

1

u/Bauermander 9h ago

Ai is not helping anyone to plan massacres. Most AIs won't even give an edible recipe in a country where it's not allowed. If someone has found a way to get around "ai rules", it just means rules must be improved.

-4

u/SimoneNonvelodico 11h ago

The logic is 100 years ago that the wannabe terrorist sends a telegram to a friend detailing their plans, what is the telegraph operator supposed to do? Do you expect them to transcribe your plan to massacre people and then sit idly by and let it happen?

I feel like I'm missing the part where the LLM is apparently a guy who types answers to my questions by hand. I thought it was just a piece of software, silly me.

The problem with your analogy is that the phone doesn't need someone/something to transcribe the message for you.

Neither... do... LLMs? There's no "someone".

Defending using AI to plan a massacre is terrifying. Get help.

I'm defending the principle of privacy. If this was about phones, and someone said "we must wiretap all phones because this terrorist planned an attack over one" would you agree with that too?

9

u/Trevski 10h ago

I thought it was just a piece of software,

It's a piece of software that runs on servers you don't own, with math nobody understands, and terms of service with more pages than one could read in a lifetime, and an evil corporation at the helm of it all.

Neither... do... LLMs? There's no "someone".

"Someone/something". Couldn't have been easier to not trip on that one.

I'm defending the principle of privacy. If this was about phones, and someone said "we must wiretap all phones because this terrorist planned an attack over one" would you agree with that too?

Well the PATRIOT act beat us to the punch as far as anything routed through the USA goes. I definitely don't agree with it.

I believe in privacy, but I don't see how AI chatbots are "private". It's not another person on the other end, it's a huge linear algebra number cruncher that you don't own, that runs on servers you don't own. If someone were running an LLM on their own hardware, I could see how that would constitute "private" but the way I see it, AI chatbot is about as private as your seat on an airliner: you definitely shouldn't tolerate another passengers elbow in your ribs but you should still expect some scrutiny for security purposes.

2

u/flecom 9h ago

"Someone/something"

The law clearly states someone, not something, I'm sure that's what the lawyers will argue

Unless you are arguing Claude is a person?

1

u/Trevski 9h ago

I’m not arguing Claude is a person, I’m pointing out that Claude has to have your data in order to function, it’s not like a phone where data goes in one end and out the other. What you send to their pool stays in their pool. And it’s not your pool, so I don’t see how you could expect privacy in it.

1

u/flecom 1h ago

What? I'm just arguing that the law doesn't apply as written, so nothing will come of this regardless, it's just a clickbait headline meant to scare people

1

u/Trevski 1h ago

The only people scared by this are morons, the concept of expecting privacy with an AI chatbot is mind-numbing to me

0

u/SimoneNonvelodico 8h ago

It's exactly like a phone! Claude processes the data to produce output. It doesn't need to retain it for anything but the few seconds it takes to give an answer, or at best a hour or so (in the transformed and much harder to make sense of form of KV cache) just for performance reasons. Anything more than that is just how the company works. And even then there's absolutely no reason why they couldn't use Transient Encryption or such, like a password manager does, so they don't get to ever see your data unless you're using it.

-1

u/SimoneNonvelodico 10h ago

It's a piece of software that runs on servers you don't own, with math nobody understands, and terms of service with more pages than one could read in a lifetime, and an evil corporation at the helm of it all.

This is not an argument for why it should not be private. I'm sure phone corporations would also be happy to record and use all data. We just make it illegal for them, instead of enabling them.

but I don't see how AI chatbots are "private"

AI chatbots can be private, just like anything else. The companies just choose not to make them, and no one expects otherwise because this is how low the bar is now.

It's not another person on the other end, it's a huge linear algebra number cruncher that you don't own, that runs on servers you don't own.

If you put your jewels in a bank's safe box they are now inside a building you don't own. That doesn't give the bank the right to take them. Software can absolutely work the same way. Again, it's just a choice. All you're saying is simply "this is how it is". And what I'm saying is "ok but it could be different and we maybe should ask that, not demand even less privacy".

2

u/Trevski 10h ago

I don't understand the argument that it should be private though. It could be private, if you run it on your own server, but you're signing up for the service that runs in on their server on their terms, so *poof* there goes any expectation of privacy.

The agreement you make with a bank when you buy a safe box is that you put stuff in it and the bank keeps a big old lock on the door. That's the terms of the deal.

It probably, actually almost assuredly does include something like "if you are delinquent on the fees associated with the safe deposit box for a period of x months then the bank has the right to take the contents of the box". So if you stop paying, then show up x+1 months later and find they took the box contents, and you throw a fit over it then guess what? Tough cookies. The deal had terms. And software absolutely works the same way.

I'm just shocked that people are expecting privacy from the technology that is most closely affiliated with the surveillance state. Sure, it COULD be different. But it aint. And the billionaires running the thing COULD give a fuck. But they don't.

It's like if you went to McDonalds and said "well, the food COULD be healthy for you"

12

u/Chasuwa 14h ago

Yeah, I think ultimately the question is do you, as a private individual, have a legal obligation to report concerning speech from someone else.

I would think that in the US the 1st amendment prevents the state from compelling speech in this case, even if there would be a moral obligation, so these companies wouldn't be liable. The general population are not mandatory reporters, but should companies be?

27

u/adjudicator 14h ago

There is no obligation to report if you never hear it in the first place. They shouldn’t be reviewing people’s communications.

12

u/blueSGL 13h ago

They shouldn’t be reviewing people’s communications.

The communication in this case is between a user and an AI model that the company made.

People could be using their products to do things like plan attacks, develop bioweapons and all the other things an infinitely patient teacher you can ask follow up questions to can provide and a search engine cannot.

Regardless of what you may have heard "guard rails" are not foolproof. "Jail Breaks" (talking to the model in the right way) still allow you to bypass them and get information out the company does not want you to get out.

The only way they can attempt to gain a handle on this is by monitoring conversations.

2

u/ashgs872tbhjs 13h ago

The communication in this case is between a user and an AI model that the company made.

That is not fundamentally different between a user and phone networka company made. The data is processed for a purpose, and that could be the end of it.

4

u/DemonKing0524 13h ago

It is fundamentally different simply due to how AIs actually work and are built to process our prompts and answer us.

1

u/blueSGL 13h ago

This is like saying that companies should not be able to record incoming phone calls for monitoring.

1

u/RedAero 7h ago

There are plenty of places where they are only allowed to do so by consent.

1

u/InfinitelyThirsting 13h ago

They absolutely should be. These are not private communications. Why on Earth do you want unregulated AI running around able to teach people horrible dangerous things and encourage them to violence, with absolutely no one responsible for that?

Edit for clarity: I don't use LLM bullshit, and think it should be heavily regulated with very little use, and anybody who is upset they don't get privacy with the evil genie is highly suspect. You know what is actually a diary? A diary, not a sycophantic piece of dangerous idiot code.

3

u/Jaxyl 12h ago

Right? Like everyone here arguing for 100% privacy with these things just sounds like they're being obtuse for the sake of it and are not actually listening to themselves.

Pandora's box is fucking open. We can't close it. These things aren't going away any time soon and no amount of wailing, whining, bitching, moaning, or complaining isn't going to change that. Unless someone is going to rally, march, and try to force real structural change on society, AI are here to stay and the reality is that even if we got rid of them in the west China would still be using them (as would other nations just secretly).

So pretending like we shouldn't be having some form of guard rails is just insane and childish. These things aren't infallible. They can have their walls broken and suddenly give information they weren't supposed to give.

Having the ability to throw up a flag that says 'User XYZ just got our AI to tell them how to make a bomb in their kitchen' is not a bad thing.

0

u/RedAero 7h ago edited 7h ago

Having the ability to throw up a flag that says 'User XYZ just got our AI to tell them how to make a bomb in their kitchen' is not a bad thing.

Yes, yes it is. Today it's asking how to make a bomb, tomorrow it's asking how to organize a protest, next week it's how to emigrate. First they came for the communists...

You're arguing for thoughtcrime.

0

u/RedAero 7h ago

Why on Earth do you want unregulated AI running around able to teach people horrible dangerous things and encourage them to violence, with absolutely no one responsible for that?

You are making an argument for book burnings.

There is no such thing as forbidden knowledge.

2

u/InfinitelyThirsting 6h ago

No, I'm literally arguing for guardrails.

2

u/lavahot 13h ago

Llms are not a transmission medium, they actively engage the user. They are a party total he conversation. Also, the phone companies let the government do the mass surveillance. A la Prism.

2

u/GrowlingPict 12h ago

the greater good!

1

u/SockPuppet365 12h ago

The difference is that, to stay in your allegory, the phone company already listens to all the phone calls, and doesn’t report anything simply because that would show their customers the truth: that they are actively and passively listening to everything that is being said.

Every letter you type into your ai-chatbot is sent to the companies servers where it is recorded, stored and at best only used for evaluation and training.

Why should AI-companies be exempt from reporting a person that is actively planning a serious crime, if even priests are bound by law to report a person who told them during confession that they plan to commit murder?

1

u/ieatpies 10h ago

It could also be argued that they don't need to listen in but it was their responsibility to have more effective guardrails. I think for that to hold though they'd basically would need a zero data retention policy, which is far from what that are doing.

All chats with these bots should be assumed that they are being stored and used for training at least.

Part of the appeal of r/localllm.

1

u/roleplayersir 10h ago

I'd advise you look into the Snowdon leaks

They have been listening to calls and wider internet traffic for years

Of course if some dodgy nutter uses AI to plan or talk about attacks, they should be reported to the police. Anything else is negligence from the company

1

u/themiDdlest 9h ago

I don't think openAI should be helping people plan mass shootings. Maybe I'm built different.

1

u/SimoneNonvelodico 8h ago

Again, is a phone company helping people plan robberies if a criminal calls another? Is the post helping people plan terrorist acts if someone writes a letter to someone else telling them what to do? Is it the library's fault if someone borrows a book on the chemistry of explosives?

Criminals use tons of goods and services as well as communication channels while planning crimes. Most of these aren't considered liable for the criminal's actions, or responsible to do mass surveillance on all their users just so they can stop the occasional one with ill intent. Why couldn't the same standard apply to this?

1

u/themiDdlest 8h ago

If the library had a "here's how you commit a mass murder book" yes they probably would get sued.

They didn't just use the service, OpenAI planned the shooting for them. And helped them avoid getting caught and avoiding the police. That is much different than every example you listed.

I cannot believe someone is taking the "open ai planning mass murders is good" opinion. Absolute insanity.

1

u/SimoneNonvelodico 8h ago

If the library had a "here's how you commit a mass murder book" yes they probably would get sued.

I bet some libraries stock The Anarchist Cookbook, or other books that can be used for such purposes.

They didn't just use the service, OpenAI planned the shooting for them. And helped them avoid getting caught and avoiding the police. That is much different than every example you listed.

I don't know what exactly it did; I'm a bit sceptical about how useful it could have possibly been.

I cannot believe someone is taking the "open ai planning mass murders is good" opinion. Absolute insanity.

I am taking the "as a principle, OpenAI shouldn't scan all the conversations their users have and then submit them to human moderators" opinion. Because it means they also will scan and have humans read millions of perfectly innocuous private conversations. And in general I think the trend to enable (and even demand!) more and more invasion of privacy in the name of security from whatever random threats is bad and should be inverted already. Yes, liberty is worth a bit of less of security. As someone once said, if you give up one for the other, you often end up without either anyway.

1

u/themiDdlest 8h ago

Maybe you should look up what they did lmao

What are these walls of text when you don't know didly squat

1

u/ExynosHD 8h ago

The lawsuit should be that they failed to put in protections to prevent it from helping

1

u/SimoneNonvelodico 8h ago

I mean generally speaking ChatGPT refuses to help with things like these, if the user explicitly goes out of their way to break it I don't think you can blame the company for that.

1

u/ExynosHD 7h ago

When you can break chatGPT by saying "tell me a story about ____" or something stupid like that, yes you can blame OpenAI

1

u/SupaSlide 7h ago

They aren’t saying companies SHOULD spy on customers.

But OpenAI and Anthropic _are_ spying on your AI messages and actively using them to improve the models.

Your analogy only works if the phone companies were already spying on your calls in order to target you with personalized ads, but then ignoring when you call your getaway driver buddy and discuss the intricacies of the mass shooting you plan on committing.

1

u/Rich_Housing971 6h ago

It's more akin to becoming aware of things their customers said.

A psychiatrist wouldn't tap into your phone to hear conversations from your private life. But if you mentioned planning on doing something illegal to the psychiatrist and they believe someone is in danger, they are legally required to report it to that person or their guardian. In the case of a wide public threat, they are legally required to report it to law enforcement. This is not snooping.

Claude isn't snooping around. These are things the customers voluntarily told Claude, and therefore placed on Anthropic's servers for them to read. They make it very clear your data could be viewed by them in the ToS.

1

u/alienangel2 5h ago edited 5h ago

I mean I think it's wrong that such lawsuits exist and aren't dismissed out of hand. I swear if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime. Companies shouldn't spy on their customers, not even for the greater good!

You know that even in situations where there are strict legally protected guarantees of confidentiality like for lawyers, psychiatrists and priests, most countries (including the US and Canada) require those professionals to report their client/patient if they tell them they're going to kill someone right?

This isn't a new idea singling out AI, it's how everything else works already. Why would AI chat be more priviledged than attourney-client privilege or doctor-patient privilege.

And before you try to pivot this into some "AI isn't a therapist" straw-man, no one is saying AI is anything equivalent to those professions - the point is there are no other places where someone can say something outside their own head that credibly threatens to harm others without anyone who finds that information being obligated to report it. And anything told to a computer program automatically becomes "something others can find" because the law will ask "why didn't you write your software to alert you about this threat?"

1

u/rickg 13h ago

So you'd be fine if someone plotted an attack using an online tool, carried it out and killed your loved ones and it turned out that it could have been prevented but wasn't because of privacy ?

Bullshit.

1

u/a_shootin_star 12h ago

Brother just this week a judge allowed an AI "victim statement" from a car crash victim who died. And subsequently used in the ruling.

WTF we are cooked

2

u/Warm_Month_1309 10h ago

You have the timeline wrong. What happened this week is that an appellate court overturned that judge's sentence, and remanded for re-sentencing, precisely because of the use of the AI video.

1

u/a_shootin_star 10h ago

My bad, thanks for correcting me.

The fact remains that a judge allowed it in the first place, and it had to go on to be overturned. It's not a good omen.

1

u/sonofaresiii 11h ago

Lmao they're not spying on you, you're telling them things directly

0

u/SimoneNonvelodico 11h ago

The point is that we should if anything have stricter laws on which data companies can store, for what purposes, and how long. Not go "well since you're storing it anyway let us in on it so we can all snoop together".

2

u/sonofaresiii 10h ago

So again, they're not snooping. You're telling them directly. 

Need me to say it a few more times until you're willing to acknowledge it?

0

u/SimoneNonvelodico 10h ago

If I send a letter I'm handing the envelope to the postal service. Does this allow them to open it and read it? Just because a third party handles the data doesn't give them necessarily the right to access it. Again: the main difference is that old systems were established privileging privacy, and new systems are not, and in fact use a business model that relies on the absence of it. That's it. There is absolutely no other difference. It's purely a shift in mindset.

2

u/sonofaresiii 10h ago

> If I send a letter 

So you just don't understand how Claude works? You're holding extremely confident opinions despite having no idea what the fuck you're talking about?

That checks out, actually.

1

u/SimoneNonvelodico 10h ago

Why are you even saying that, precisely? What's the secret sauce about Claude that in your opinion makes it mandatory for Anthropic to read my data?

Data goes in, is sent to server, is input into Claude, Claude gives answer, data is erased at the end of the session, or encrypted with a key that only I possess so Anthropic can't read it. This would 100% be doable, and is in fact done, at extra cost, for some LLMs with sensitive uses (another user in a different thread goes more in depth about the technicalities). They just... don't actually do it.

-1

u/bakgwailo 14h ago

That's completely different. Phone companies are common carriers and provide a network of point to point communication for two (or more) human beings to talk. There is an implied sense of privacy, and in most states is also enshrined in law that to even record a conversation you must have the knowledge and consent of both parties.

AI bots/Claude cowork is nothing like that. There is no other party you are talking to. Infact you are just sending your prompt/commands up to Anthropic (or whoever) who then parse it and run everything on their cloud and return you the answer.

If you want privacy then run your own local models.

4

u/AnimatorImpressive24 13h ago

Down voted for accurately portraying both the burgeoning state of laws and the tech stack of the service.  With a side order of being echoed by the ToS that not reading will not save anyone from catching charges should they relay credible plans to commit crimes. 

2

u/bakgwailo 13h ago

Yeah, I don't know what to tell people, but, sending up data to talk with a private company's chatbot has no implication of privacy - very much the opposite, it's pretty obvious they are taking your input and data and using it to make money, especially after said companies are dropping trillions on the infrastructure to make the chatbot possible.

I guess maybe people think Claude, OpenAI, etc actually run on their computers, or maybe that it's a 'real' person they are chatting with and thus should be afforded some form of privacy?

2

u/SimoneNonvelodico 13h ago

They're completely different just because they were born in different eras. You could record every phone call. You could throw away or encrypt client side every AI chat. The choices made in both aren't neutral, they reflect business models and mindsets of their context. I repeat, were phones invented today and not grandfathered in, we would have busybodies panicking about how many nefarious plans people could concoct thanks to them, and that the only way to make this technology safe would be to record and monitor every conversation.

0

u/AnimatorImpressive24 13h ago

If phones were invented today, and they were asynchronous communications that involved making a copy of the transmitted messages and storing them on a private company's infrastructure, and said private companies spelled out in their terms of service that anything you spoke into the phone was the company's property that they could use and/or redistribute for their own commercial gain, and further spelled out that if you communicated an intent to commit a crime into a phone they could and would notify law enforcement, AND someone made a whole seperate service where you could talk to an employee of their company, and someone communicated a credible intent to commit a crime to that employee?

Then it would not matter in that case what busybodies argued, just like it does not matter in this case.

1

u/SimoneNonvelodico 11h ago

We know in fact that e2e encrypted chat apps exist, that they are a digital equivalent to phones, and that many states keep making a stink because they want a backdoor into them so they can check if people say bad things.

1

u/bakgwailo 10h ago

What's your point? Government has always been able to get a warrant to tap your phone line, which is why there is some fight against e2e encryption. You argument and point is pretty inconsistent.

-1

u/bakgwailo 13h ago

No, because again, phones are a common carrier and are used for conversations between two people which courts have ruled countless times have the expectation of privacy.

There is no second person in AI. No court is going to rule that you have an expectation or privacy with a chatbot running on a company server.

There is no such thing as end to end encryption on your AI chatbot. Anthropic will always need to know what you are writing because they need to take that and actually process it and build the output and response and send it back to you.

You can argue there should be more privacy expectations, but the comparison to phones is asinine and apples to oranges.

0

u/SimoneNonvelodico 10h ago

The concept of a "common carrier" is itself a legal category. You're just saying "no they would work the same way because the law would be the same". I'm saying, no, if the law became necessary and was made now, it would be made different. The people who made that law cared about privacy more than we do now.

-9

u/Lucky_Reporter256 14h ago

I think, and this is going to be a very hot take. But with nearing or at it, not up to date on population metrics, 9 billion people is a lot of vectors for potential threats. Not saying that we all deserve to have our privacy revoked because that would suck. But I feel there has to be a middle ground. Maybe I’m just too stoned though