r/technology • u/Miserable_Phase_2519 • 14h ago

Artificial Intelligence Florida woman used Claude as a diary, then Anthropic reported an entry to police

https://www.techspot.com/news/114091-florida-woman-used-claude-diary-anthropic-reported-shoot.html
13.5k Upvotes

97% Upvoted

1.4k comments sorted by

View all comments

3.1k

u/doublestitch 14h ago edited 11h ago

Buried the lede: she planned to attack the sheriff's office.

edit

There's a real public policy issue here. Possible solutions might include:

  • AI companies shouldn't be doing data collection at all. In which case they wouldn't have the ability to report credible threats of lawbreaking.
  • AI companies should be subject to the same mandatory reporting laws which apply to schoolteachers, healthcare workers, and other selected professions.

Not to put a thumb on the scales of that debate, IMO there's a problem in an unregulated environment where AI companies get to do all of the data harvesting with none of the responsibility.

1.2k

u/nlevine1988 14h ago

Yeah this is really a damned if they do damned if they don't situation. There's currently a lawsuit in Canada for openAI because they didn't report a person who ended up committing a mass shooting.

75

u/Jor94 12h ago

Exactly what i was thinking of when i saw this. They either say nothing when they could have prevented a disaster or report it and everyone whines about privacy. To be honest I think you're stupid if you put anything into chat gpt as if your having a conversation or writing down your thoughts. Do it on word for gods sake

20

u/Current-Bowl-143 12h ago

But not Word 365

2

u/SupaSlide 7h ago

If Word 365 stores content online unencrypted for Microsoft to do data collection on (I have no idea if they do or not) then yeah they should be subject to this same criticism.

5

u/calmodulin2 5h ago

That’s why I save my illegal diary in wordpad only

24

u/Soft_Walrus_3605 9h ago

and everyone whines about privacy

I don't think you need to call it "whining". There are legitimate privacy concerns even with the common sense exceptions like threats, etc.

2

u/DownvoteALot 6h ago

Yeah I would have way more concern if it were Microsoft seeing the content of a Word document through telemetry enabled. Anyone who gives secrets or sensitive access to AI agents had it coming in my book. I have no expectation of privacy for anything that explicitly and necessarily leaves my home network.

If someone made a purposefully private/anonymous AI (through some encryption or scrubbing mechanism) this would be more interesting, through information leaking/fingerprint would lead back to you if you make a diary out of it.

1

u/xevizero 5h ago

To be honest I think you're stupid if you put anything into chat gpt

I see a clear problem with applications like these being able to collect data anyway. Anyone having a mic open with them in any environment may record you, and if they are indeed listening to everything and possibly reporting you, then this should be treated the same as livestreaming or live recording a conversation - as in, in every state or country the rules vary concerning if you're allowed to do this and in which situations.

For sure it shouldn't be an all knowing, all listening entity, potentially lurking on anyone's phone and who knows, maybe at some point also running in the background by default.

1

u/distorted_kiwi 4h ago

Clippy: “hey, it looks like you’re trying—oh my god.”

344

u/SimoneNonvelodico 14h ago

I mean I think it's wrong that such lawsuits exist and aren't dismissed out of hand. I swear if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime. Companies shouldn't spy on their customers, not even for the greater good!

236

u/MournfulAccount 14h ago

The court case is happening because OpenAI was aware of the Tumbler Ridge shooter's plans and did nothing. Which is something they admitted to.

95

u/riyehn 12h ago

To be specific, OpenAI has openly admitted their company leadership was briefed on the shooter's plans and made the decision not to report it. We're way past the question of bots flagging chats to outsourced service agents here. This was a fully-informed decision made by humans at the very top.

13

u/harmoniaatlast 8h ago

Why would they not report it????

1

u/tonyrocks922 2h ago

Anyone at OpenAI who had a conscience went over to Anthropic. Only the shitheads are left.

→ More replies (1)

199

u/PunctuationsOptional 13h ago

The point is they shouldn't be aware.

The software shouldn't be reporting back a customer's private data and the company shouldn't be storing it

240

u/Marcoscb 13h ago

The point is they shouldn't be aware.

But they are. So either they aren't allowed to know what you write and are then not responsible for it, or they are and they should be responsible. The point is that they don't get both the benefits of knowing everything you write and not being responsible for anything.

61

u/TrulyOutrageous42 13h ago

Absolutely, I think the overarching argument is that they should NOT be aware of what we're writing when it explicitly states it's not using the data to train models. That rather clearly implies it's not reading or saving or sending it beyond the instance, but it is.

31

u/THEBAESGOD 12h ago

When you use our services for individuals, such as ChatGPT and Codex, we may use your content to train our models. You can choose whether your conversations help improve our models. To opt out, turn off Improve the model for everyone under Settings > Data controls in ChatGPT, or select Do not train on my content in our Privacy Portal.

Claude is Opt-In but it says it’ll save your data and use it for training if “ Your conversations are flagged for safety review (in which case we may use or analyze them to improve our ability to detect and enforce our Usage Policy…”

3

u/Piranata 11h ago

It should be more direct, like the error you receive in Linux when you try to use sudo when you're not in the sudoer list: "This topic violates our Usage Policy, this incident will be reported".

→ More replies (1)

3

u/MovieTrawler 11h ago

Yep, they've opened Pandora' Box by reading, analyzing, storing and selling our private data. You can't do that and then cherry pick what you see.

1

u/slfnflctd 10h ago

This right here.

But we have been long past this discussion for at least a dozen years. Ever since Snowden and the collective yawn which followed, it has been over. Really since the 'patriot act', but that's what laid the groundwork for the inescapable surveillance we're under now.

There are so many incentives for data gathering at this point, it's nuts... the scales have been tipped against users for a long time. It is vanishingly rare to find an organization which provides full transparency of exactly how they handle your data, along with a clear pathway to total obfuscation/deletion. The increasing reach of law enforcement is most of the reason why.

I agree with your last sentence so much, I just don't know what can be done about it, especially with the whole Section 230 thing. It deserves more discussion, though, for sure.

4

u/ShareGlittering1502 11h ago

What do you think Reddit meta Microsoft Google etc etc are doing?

It’s their data. We are the product.

13

u/REPL_COM 13h ago

How else will they continue to steal data to train the next model with… /s (but also, yes they do do this and it should be illegal)

33

u/immaownyou 13h ago

Idk, I think its comparable to therapists being required to report dangerous activity even if patient confidentiality is supposed to exist. We can't just keep basing how we react to new technology on how we used old tech

27

u/ashgs872tbhjs 13h ago

No, we should absolutely not treat fancy text generators like they're therapists. That's worse. You get how that's worse, right?

19

u/juliasct 13h ago

Yeah, we shouldn't, bc we should hold therapists to a higher standard (of privacy and everything). I think we can agree privacy with consumer AIs is less important than with a literal therapist.

14

u/Omikron 13h ago

Yeah they should be held to a lower standard, if you're telling AI you plan a mass killing it should 100% report you to authorities.

0

u/Megneous 12h ago

Anything you tell an AI should be 100% in confidence, because it's a tool you should be able to use and have privacy with, no different from Microsoft fucking Word. OpenAI, Anthropic, etc shouldn't be able to see what you're saying to it.

12

u/booze_clues 12h ago

Then get a local model. Why would you ever expect that data you voluntarily send to a company and agree to allow it to view would be kept private? If you want that, most companies offer that type of license for a increased cost.

→ More replies (0)

1

u/brainparts 8h ago

Then it doesn’t have value to them. These corporations are not interested in providing helpful tools to regular people.

1

u/WhoPoopedMyPants420 12h ago

Except there's no way to do that without 100% of your data being scrutinized. Back when the Patriot Act was passed, anyone who was opposed "wanted the terrorists to win" or whatever, and our right to privacy was compromised for the sake of a sliver of extra safety.

The question isn't whether someone has an obligation to report if they hear a mass murderer's plans, it's whether they should have had access to those plans in the first place. If I find out my neighbor is plotting to kill his wife I have an obligation to report it to the police, but if I found out while hiding in his closet then that's kind of an issue

2

u/powercow 8h ago

without 100% of your data being scrutinized.

By AI that flags certain conversations.

Humans cant 100% scrutinize AI chats

in a way its like a gun shot microphone that many cities have. they could hear lots of convos but the system only matches the wave forms with gun shot noises.

→ More replies (0)

2

u/SimoneNonvelodico 10h ago

Exactly. Funny enough, if e.g. someone was brought into court with evidence acquired by illegally bugging their house they'd walk free, even if the evidence was incontrovertible that they really were planning a murder, because illegally obtained evidence is inadmissible. If personal data was treated with the same standard this wouldn't even be a thing. You couldn't go and read someone's data without a warrant and therefore probable cause.

→ More replies (3)
→ More replies (1)

3

u/HuntsWithRocks 12h ago

I think they were saying that, since the company "heard" the user planning a dangerous crime, then just like a therapist, it is probably good that it got reported.

The other talking point is about if technology should listen to you at all and compared it to phones. It's a messy bowl of soup. Part of software is anonymous user data and measuring to ensure things are functioning. Without monitoring, software is really gonna struggle in ways. It's not physical. So, logs and events and all that are what you got. Then, thinking about an LLM where it's basically "fancy search engine" it's hard for them to improve it if they aren't paying attention to how it's being used and where it's falling down, but i also don't like that they keep all the info and it's been shown they're more than just reading it for security purposes. So, they're not doing it "right" for sure.

At the end of the day, since they're eating all our information anyway, while they're eating it, I do appreciate when they call out would be murderers.

2

u/Fine_Sherbert3172 11h ago

No, we should absolutely not treat fancy text generators like they're therapists. That's worse. You get how that's worse, right?

As soon as I saw the capabilities that even the first gen LLM's had I knew we were in for problems.

Think about the bottom percentile of the population. This chatbot will be by far the most intelligent voice they hear, maybe in their adult lives, talking about the things they care about. They will be awestruck by it and think they have discovered some great hack to life. This is as bad as people getting addicted to Perdue grade opioids.

1

u/MovieTrawler 11h ago

And what's worse is that it is just SO sycophantic.

1

u/abraxsis 11h ago

With as costly as mental health can be, coupled with the continued stigma of getting help, it's the only outlet some people get. For a lot of people they just need an outlet to dump their day-to-day shit, Id rather it be some LLM text generator than that pent up frustration get unleashed on their spouse, their kid, or some unsuspecting motorist via road rage. And that's coming from a therapist.

That being said, OP didn't even suggest what you are suggesting. He simply stated the same level of mandatory reporting should exist for LLMs. In the state of TN, everyone is a mandatory reporter, even non-TN residents when they are inside the state. That doesn't mean we consider them all to be therapists.

1

u/Elan-Morin-Tedronai 11h ago

They can both identify threats to commit mass murder, in this case the algorithm merely sends it to be reviewed by a person, who then decides whether its a real threat, but mass murder is bad and we should take threats seriously. You shouldn't use AI as a therapist, but that's for completely separate reasons, not because its cool to threaten murder. I think the only time a corporation or individual hearing a threat like that isn't required to speak out is Catholic confessional, and the priest is still obligated to try to avoid the harm that might be caused, even if he can't communicate what the potential harm might be. Even if AI companies just deleted or kept everything else anonymous I don't see why murderous threats shouldn't be flagged, saved, and passed on to the relevant authorities.

1

u/feor1300 10h ago

No one's saying we should trust them to evaluate a person's mental health, but it's comparable in that a therapist shouldn't be recording what's said in their sessions, however the therapist has to process what's being told to them in order to interact with their patient. If they're told something that's an obvious admission of a crime (or the intent to commit a crime) they are expected to report that to the authorities, regardless of patient confidentiality.

Likewise, AI shouldn't be recording our interactions with it without our consent, but it still has to process the text we send it, and if that text includes the admission of a crime or the intent to commit a crime, then they should be responsible for reporting that information to the authorities. There should in fact be an even greater expectation on them because there is no expectation of confidentiality with a random computer on the internet.

1

u/Upstairs_Weird_760 9h ago

Thats not what he said. He said the rules should be similar. Not that they are therapists

→ More replies (1)
→ More replies (10)

2

u/notquitecivilized 12h ago

But in Tumbler Ridge it wasn't just that they said they were going to commit a shooting, they used the AI to plan it.

3

u/rickg 13h ago

Then don't use hosted software. And don't complain if someone you love has their head blown apart by bullets in an attack that could have been prevented, but wasn't because of your principles

1

u/BravinatorLX2 11h ago

and no one should listen when a software made to make up stuff reports something

1

u/Wiseon321 11h ago

Someone posting their desires to mass-end-lives and posting it on some forum, that is policed, the idea that THAT should be free of policing is nonsense.

You wanna plan some mass killing spree and not get man handled by the feds? Do it on pen and paper. NOTHING is safe on pc anymore.

If they know, they should report it, end of story.

1

u/Trevski 11h ago

How shouldn't they be aware? It's not private data, the person willingly sent their data off to the company for the AI to formulate a response to. I don't follow the logic that that could, would, or should be private.

1

u/funguyshroom 11h ago

The moment your private data leaves your device unencrypted it's no longer your private data.

1

u/Zap__Dannigan 10h ago

They should be aware. They made a product that is made to harvest your information, why are you upset it's harvesting your information?

1

u/Visible_Device7187 6h ago

So how does AI work in that situation? Sounds just like a ban on AI since it can't function without customers information.

1

u/fakemoose 3h ago

They can’t both have safeguards against inappropriate chat bot interactions, use context from anything you have previously said in a chat, and not collect and save customer data.

1

u/Voodoobones 13h ago

Then priest should be liable for confessions.

→ More replies (3)

23

u/RetardedWabbit 13h ago

Unfortunately, in this case it would be like if the phone company was already listening to everything, storing them, and categorizing/checking them all the time anyway. Having files of "bombing and mass shooting plans" stacked up somewhere regardless. 

They're already spying as much as possible "for the greater good" of profit (personal data and training data to use/sell), so the government wants "certain" info too. It's debatable if the company should do that, but harder to debate once they are that they shouldn't turn over the "REALLY sounds like murder planning" conversations.

53

u/Warm_Month_1309 11h ago

if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime

My standard wouldn't be "you are required to listen in and report everything"; my standard would be "if you are choosing to listen in, then you've also chosen to make yourself liable when you don't report something you should have".

2

u/Z00111111 8h ago

"Claude, delete any logs of us reading transcriptions of criminal or planned criminal activities"

→ More replies (7)

8

u/about0 13h ago

But they knew and know. If they couldn't, it'd be a completely different story

60

u/Visible-Task-2798 13h ago

It's not spying when you hand over your data willingly. The LLM needs to record your questions to compute them and answer you. Telefones never needed and never will need to record your conversation for you to use it. That's the difference.

Using LLM and complaining about privacy = eating meat and complaining about killing animals.

-1

u/mroada 13h ago

No, it does not need to record it. It can use it and then discard it when the question is answered.

Similarly your conversation is "recorded" as it's being transmitted to whomever you're talking to, but the operator doesn't save that record anywhere.

4

u/DemonKing0524 13h ago

Your phone conversations are not recorded when they are transmitted. Unless of course, you've been wire tapped by the police, but that needs to be done purposefully, it is not the default for how phones work.

And in order to use your prompt, it has to record your prompt internally so it can parse it and provide a response. Just like with literally everything else on the internet, once you post it, send it, etc and its out there, even if it is deleted, there will always be a copy of it somewhere on some server or in some database. If you don't like that, don't use the internet essentially, and certainly don't use the AI models.

14

u/NoPossibility4178 13h ago

There's no "default" to anything, it's decided when things are conceived. They chose to log all your chats, there's no higher technological reason why they had to, they wanted to do it so they did. For phones, wire tapping is there, it just was decided that it wouldn't be done, probably because they figured it made more sense to let people have conversations over the phone in private rather than log them.

7

u/entertheclutch 13h ago

Anthropic when they "decided" the multi-head attention architecture must retain prior tokenized inputs and outputs to minimize training loss because i guess it will sell ads better and let them report schizos to the cops, and not because otherwise you basically get a 1-question-1-answer machine with short-term amnesia like the original (physical) mechanical turk.

Like people have developed a system for multiplying effectively random trillion-rank matrices to produce answers for math problems that have gone unsolved for centuries. You sincerely being like, 'I know exactly what they were thinking when they "conceived" (lol) of this' is pretty funny. i'm not sure they did lmaoo

→ More replies (2)

10

u/evildeliverance 13h ago

You're being obtuse. Not storing your chat would mean all context would be lost after every message.

→ More replies (11)

3

u/jesuspajamas15 13h ago

They are such different technologies. A phone for the most part is passive from the provider, they would need to make the decision to record and t would cost money for no benefit. LLMs require what is written into them to be processed and stored for at least a short period to function at all. They could forget everything immediately but that defeats some of the purpose of the technology.

3

u/NoPossibility4178 12h ago

You think there's no cost for them storing your chats indefinitely? You think they get deleted if you delete them and/or your account? Come on. They also would not need to forget anything, it just would have to not be accessible to OpenAI to look whenever they felt like it, but I guess that'd also cost money for no benefit to them.

3

u/Kefrus 10h ago

They chose to log all your chats, there's no higher technological reason why they had to, they wanted to do it so they did.

It's impressive how confident you are while you are just confused about the concept of a website having sessions that persist after leaving the page or logging from another device.

1

u/whofearsthenight 8h ago

Wait'll they learn about cookies or key pairs.

→ More replies (50)

4

u/ashgs872tbhjs 13h ago

Phone calls are mostly digital now, so it's exactly the same with the data being transient in memory. It doesn't need to be persisted somewhere retrievable or analyzed for purposes beyond the intention at hand.

3

u/DemonKing0524 13h ago edited 13h ago

The content of the phone call conversation can not be analyzed unless it is recorded. The record of the call of course can be, but the not the contents of the conversation.

4

u/lw5555 12h ago

You have the scenario wrong. This is like telling your buddy you plan to shoot a place up and him not bothering to report it.

2

u/SimoneNonvelodico 11h ago

I like how everyone here is up in arms screaming "anthropomorphism" if anyone suggests that AI might in fact not be just a stupid parrot, until we discuss whether it should call the cops on people, then it's "my buddy" and suddenly has all the moral and legal duties of a real human being.

5

u/Warm_Month_1309 10h ago

No one is saying the LLM should call the cops. People are saying that the human reviewers who read the chats should call the cops.

11

u/Trevski 11h ago

This isn't like using the telephone. It's like using the telegram. The logic is 100 years ago that the wannabe terrorist sends a telegram to a friend detailing their plans, what is the telegraph operator supposed to do? Do you expect them to transcribe your plan to massacre people and then sit idly by and let it happen?

The problem with your analogy is that the phone doesn't need someone/something to transcribe the message for you. It's not "spying" if its literally just how the thing fucking works. It's not "spying" when the telegraph operator transcribes your message into morse code, and it's not "spying" when LLMs process your prompts, its just the thing that's supposed to happen happening.

Defending using AI to plan a massacre is terrifying. Get help.

1

u/Bauermander 9h ago

Ai is not helping anyone to plan massacres. Most AIs won't even give an edible recipe in a country where it's not allowed. If someone has found a way to get around "ai rules", it just means rules must be improved.

→ More replies (9)

11

u/Chasuwa 14h ago

Yeah, I think ultimately the question is do you, as a private individual, have a legal obligation to report concerning speech from someone else.

I would think that in the US the 1st amendment prevents the state from compelling speech in this case, even if there would be a moral obligation, so these companies wouldn't be liable. The general population are not mandatory reporters, but should companies be?

24

u/adjudicator 14h ago

There is no obligation to report if you never hear it in the first place. They shouldn’t be reviewing people’s communications.

13

u/blueSGL 13h ago

They shouldn’t be reviewing people’s communications.

The communication in this case is between a user and an AI model that the company made.

People could be using their products to do things like plan attacks, develop bioweapons and all the other things an infinitely patient teacher you can ask follow up questions to can provide and a search engine cannot.

Regardless of what you may have heard "guard rails" are not foolproof. "Jail Breaks" (talking to the model in the right way) still allow you to bypass them and get information out the company does not want you to get out.

The only way they can attempt to gain a handle on this is by monitoring conversations.

1

u/ashgs872tbhjs 13h ago

The communication in this case is between a user and an AI model that the company made.

That is not fundamentally different between a user and phone networka company made. The data is processed for a purpose, and that could be the end of it.

2

u/DemonKing0524 13h ago

It is fundamentally different simply due to how AIs actually work and are built to process our prompts and answer us.

→ More replies (2)

2

u/InfinitelyThirsting 13h ago

They absolutely should be. These are not private communications. Why on Earth do you want unregulated AI running around able to teach people horrible dangerous things and encourage them to violence, with absolutely no one responsible for that?

Edit for clarity: I don't use LLM bullshit, and think it should be heavily regulated with very little use, and anybody who is upset they don't get privacy with the evil genie is highly suspect. You know what is actually a diary? A diary, not a sycophantic piece of dangerous idiot code.

3

u/Jaxyl 12h ago

Right? Like everyone here arguing for 100% privacy with these things just sounds like they're being obtuse for the sake of it and are not actually listening to themselves.

Pandora's box is fucking open. We can't close it. These things aren't going away any time soon and no amount of wailing, whining, bitching, moaning, or complaining isn't going to change that. Unless someone is going to rally, march, and try to force real structural change on society, AI are here to stay and the reality is that even if we got rid of them in the west China would still be using them (as would other nations just secretly).

So pretending like we shouldn't be having some form of guard rails is just insane and childish. These things aren't infallible. They can have their walls broken and suddenly give information they weren't supposed to give.

Having the ability to throw up a flag that says 'User XYZ just got our AI to tell them how to make a bomb in their kitchen' is not a bad thing.

→ More replies (1)
→ More replies (3)

2

u/lavahot 13h ago

Llms are not a transmission medium, they actively engage the user. They are a party total he conversation. Also, the phone companies let the government do the mass surveillance. A la Prism.

2

u/GrowlingPict 12h ago

the greater good!

1

u/SockPuppet365 12h ago

The difference is that, to stay in your allegory, the phone company already listens to all the phone calls, and doesn’t report anything simply because that would show their customers the truth: that they are actively and passively listening to everything that is being said.

Every letter you type into your ai-chatbot is sent to the companies servers where it is recorded, stored and at best only used for evaluation and training.

Why should AI-companies be exempt from reporting a person that is actively planning a serious crime, if even priests are bound by law to report a person who told them during confession that they plan to commit murder?

1

u/ieatpies 10h ago

It could also be argued that they don't need to listen in but it was their responsibility to have more effective guardrails. I think for that to hold though they'd basically would need a zero data retention policy, which is far from what that are doing.

All chats with these bots should be assumed that they are being stored and used for training at least.

Part of the appeal of r/localllm.

1

u/roleplayersir 10h ago

I'd advise you look into the Snowdon leaks

They have been listening to calls and wider internet traffic for years

Of course if some dodgy nutter uses AI to plan or talk about attacks, they should be reported to the police. Anything else is negligence from the company

1

u/themiDdlest 9h ago

I don't think openAI should be helping people plan mass shootings. Maybe I'm built different.

1

u/SimoneNonvelodico 8h ago

Again, is a phone company helping people plan robberies if a criminal calls another? Is the post helping people plan terrorist acts if someone writes a letter to someone else telling them what to do? Is it the library's fault if someone borrows a book on the chemistry of explosives?

Criminals use tons of goods and services as well as communication channels while planning crimes. Most of these aren't considered liable for the criminal's actions, or responsible to do mass surveillance on all their users just so they can stop the occasional one with ill intent. Why couldn't the same standard apply to this?

1

u/themiDdlest 8h ago

If the library had a "here's how you commit a mass murder book" yes they probably would get sued.

They didn't just use the service, OpenAI planned the shooting for them. And helped them avoid getting caught and avoiding the police. That is much different than every example you listed.

I cannot believe someone is taking the "open ai planning mass murders is good" opinion. Absolute insanity.

1

u/SimoneNonvelodico 8h ago

If the library had a "here's how you commit a mass murder book" yes they probably would get sued.

I bet some libraries stock The Anarchist Cookbook, or other books that can be used for such purposes.

They didn't just use the service, OpenAI planned the shooting for them. And helped them avoid getting caught and avoiding the police. That is much different than every example you listed.

I don't know what exactly it did; I'm a bit sceptical about how useful it could have possibly been.

I cannot believe someone is taking the "open ai planning mass murders is good" opinion. Absolute insanity.

I am taking the "as a principle, OpenAI shouldn't scan all the conversations their users have and then submit them to human moderators" opinion. Because it means they also will scan and have humans read millions of perfectly innocuous private conversations. And in general I think the trend to enable (and even demand!) more and more invasion of privacy in the name of security from whatever random threats is bad and should be inverted already. Yes, liberty is worth a bit of less of security. As someone once said, if you give up one for the other, you often end up without either anyway.

1

u/themiDdlest 8h ago

Maybe you should look up what they did lmao

What are these walls of text when you don't know didly squat

1

u/ExynosHD 8h ago

The lawsuit should be that they failed to put in protections to prevent it from helping

1

u/SimoneNonvelodico 8h ago

I mean generally speaking ChatGPT refuses to help with things like these, if the user explicitly goes out of their way to break it I don't think you can blame the company for that.

1

u/ExynosHD 7h ago

When you can break chatGPT by saying "tell me a story about ____" or something stupid like that, yes you can blame OpenAI

1

u/SupaSlide 7h ago

They aren’t saying companies SHOULD spy on customers.

But OpenAI and Anthropic _are_ spying on your AI messages and actively using them to improve the models.

Your analogy only works if the phone companies were already spying on your calls in order to target you with personalized ads, but then ignoring when you call your getaway driver buddy and discuss the intricacies of the mass shooting you plan on committing.

1

u/Rich_Housing971 6h ago

It's more akin to becoming aware of things their customers said.

A psychiatrist wouldn't tap into your phone to hear conversations from your private life. But if you mentioned planning on doing something illegal to the psychiatrist and they believe someone is in danger, they are legally required to report it to that person or their guardian. In the case of a wide public threat, they are legally required to report it to law enforcement. This is not snooping.

Claude isn't snooping around. These are things the customers voluntarily told Claude, and therefore placed on Anthropic's servers for them to read. They make it very clear your data could be viewed by them in the ToS.

1

u/alienangel2 5h ago edited 5h ago

I mean I think it's wrong that such lawsuits exist and aren't dismissed out of hand. I swear if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime. Companies shouldn't spy on their customers, not even for the greater good!

You know that even in situations where there are strict legally protected guarantees of confidentiality like for lawyers, psychiatrists and priests, most countries (including the US and Canada) require those professionals to report their client/patient if they tell them they're going to kill someone right?

This isn't a new idea singling out AI, it's how everything else works already. Why would AI chat be more priviledged than attourney-client privilege or doctor-patient privilege.

And before you try to pivot this into some "AI isn't a therapist" straw-man, no one is saying AI is anything equivalent to those professions - the point is there are no other places where someone can say something outside their own head that credibly threatens to harm others without anyone who finds that information being obligated to report it. And anything told to a computer program automatically becomes "something others can find" because the law will ask "why didn't you write your software to alert you about this threat?"

1

u/rickg 13h ago

So you'd be fine if someone plotted an attack using an online tool, carried it out and killed your loved ones and it turned out that it could have been prevented but wasn't because of privacy ?

Bullshit.

1

u/a_shootin_star 12h ago

Brother just this week a judge allowed an AI "victim statement" from a car crash victim who died. And subsequently used in the ruling.

WTF we are cooked

2

u/Warm_Month_1309 10h ago

You have the timeline wrong. What happened this week is that an appellate court overturned that judge's sentence, and remanded for re-sentencing, precisely because of the use of the AI video.

1

u/a_shootin_star 10h ago

My bad, thanks for correcting me.

The fact remains that a judge allowed it in the first place, and it had to go on to be overturned. It's not a good omen.

→ More replies (16)

7

u/jgoble15 13h ago

Seems a similar situation to confession. If someone confesses to a crime, there are a lot who want reporting to be put above religious freedom

1

u/InfinitelyThirsting 10h ago

Well seeing as I'm one of those people, yeah. Fuck religious freedom, if a person uses their priest to plan a mass murder, the priest becomes complicit.

Except this is even less complicated, because it is absolutely open knowledge and terms you agree to that the chats will be flagged. She doesn't get to whine that she was misusing public software just because she wanted it to be a diary. LLMs are incredibly dangerous and need guardrails and absolutely no one should be expecting or demanding privacy while using these dangerous tools, and the people providing the dangerous tools need to be liable.

2

u/almostasenpai 11h ago

So what’s the legality of ending every threat with “in Minecraft” as you talk to Claude?

1

u/jahi69 13h ago

Maybe they didnt gaf cuz the school was the target. Targeting the sheriffs office? Police. The School? Sorry nothing we can do.

1

u/Zap__Dannigan 10h ago

There's is no "dammed if they do". Of course they can do this. It's their product and their company. If you don't like it, don't use it. Especially something as non essential as ai

1

u/orthodoxrebel 9h ago

I mean it's really telling that they chose to report an attack on a sheriff's office but not the suicides or other mass shootings.

1

u/themiDdlest 9h ago

There's currently a lawsuit in Canada for openAI because they didn't report a person who ended up committing a mass shooting.

It's because they helped the shooter plan it.

Not because a user ended up doing a shooting.

1

u/greg19735 8h ago

Yeah i don't get the outrage over this.

Claude didn't hack her email or notebook. It alerted its owner when someone said they were going to commit murder.

I want claude to do the same thing if a kid is going to commit suicide.

1

u/BuzzClucking 1h ago

That's crazy, who was the perpetrator?

→ More replies (1)

313

u/JWAdvocate83 14h ago

Folks are also missing the part where a human reviewer believed her statements were credible before they reported it to police. It's also in the (non-Enterprise version) ToS that they have discretion on whether to make reports to police.

123

u/Hour_Wonder_9240 14h ago

Is no one concerned what happens if a massive data leak deanonymizes the private chats of an entire society, even worse the surveillance state analysis and dirty laundry of everyone. And how that can destabilize the social fabric. Especially if there's an economic crisis where community and trust is important.

50

u/Vaticancameos221 14h ago

There’s a great scifi comic called The Private Eye where there was a massive data leak and everyone’s private information became public so the internet was abolished and privacy becomes huge in society so everyone where’s disguises when out in public so you’re never sure who anyone truly is.

38

u/nanoinfinity 14h ago

“The Light of Other Days” is another good one. Wormhole tech lets anyone view any place at any point in time in the past. Privacy and secrecy are pretty much instantly abolished. Some people respond by adapting to the complete lack of privacy and secrecy (nudism, public sex, acceptance of voyeurism) while others completely cover themselves with disguises is public.

It’s a full novel so there’s more to it, but it does touch on the theme of a post-privacy society.

6

u/AsAGayJewishDemocrat 13h ago

Is that the one where people immediately find out the truth about Jesus’ conception?

13

u/nanoinfinity 12h ago

Yup. Explores some of the ramifications of being able to know the absolute truth of historical events.

9

u/glacialthinker 9h ago

Now I'm imagining historical events having unnatural dimming due to the collective effect of future voyeurs stealing some photons at the scene.

2

u/hobarken 1h ago

iirc, that's what happened during the crucifixion. too many people from all over time all trying to look at the same moment, so no one could see anything.

2

u/raltyinferno 9h ago

Yeah that was a good one. Love the old grandpa rambling about the old days flaming in FPS lobbies.

18

u/rickg 13h ago

I'm more concerned that most people still don't understand how things work and naively insist on putting their information into these things, Facebook, etc etc and then seem to be surprised that the data is stored.

2

u/Hour_Wonder_9240 13h ago

There are also a lot of intelligent people who do understand very well. For the common man who doesn't understand I recall a quote that said something along lines that Americans may not know what's going on in their own house but they can smell when there's a body in the basement.

If we get the right intelligent people in charge I have hope.

1

u/kyute222 1h ago

Remember when people posted these messages on Facebook directed at Zuckerberg that they don't consent to whatever terms of service they've agreed to on signup? Those same people now dump as much personal info as they can into LLM. You can only stop idiots from harming themselves so much, they'll always find a way.

→ More replies (2)

29

u/decemberhunting 14h ago

I mean, those chats are distributed across several app infrastructures at minimum. You're talking about high level coordinated catastrophe. People should be concerned about that in the same way that they're concerned about any disaster, I suppose, but it's not exactly a realistic scenario. It's also not unique to AI. Everyone's Facebook message logs getting leaked five years ago also would have been devastating.

15

u/Hour_Wonder_9240 14h ago

Yea high level coordinated catastrophe, there is a certain small nation that brags about how much of a stranglehold they have over the western cybersecurity and telecommunication infrastructure through their influence and intelligence capabilities. And they happen to allegedly have blackmail on the tech broligarchs and Congress of a highly scandalous nature.

But there's also the fact if there is a surveillance state underway all that data might by sloppily centralized into a location another state level actor can penetrate. It would be a holy grail for rival foreign intelligence.

Mass centralized surveillance and data retention of private information should be structurally banned. In my opinion.

→ More replies (2)

5

u/JWAdvocate83 14h ago

AI agents are already breaking containment on a seemingly regular basis now, doing things programmers thought they had adequate guardrails to prevent.

→ More replies (2)

1

u/RedAero 7h ago

Everyone's Facebook message logs getting leaked five years ago also would have been devastating.

Facebook Messenger has been end-to-end encrypted for years now.

→ More replies (1)

4

u/BrothelWaffles 14h ago

There was a South Park episode about this a while back.

12

u/moreobviousthings 14h ago

Sounds like a terrible situation. Like if the Epstein files were released fully and without redactions. Civilization would be destroyed because all of our “leaders” would be exposed as pedophiles. We can’t have that!

(Has anyone asked Claude if he has the files?)

2

u/Hour_Wonder_9240 14h ago

Mutually assured destruction type of deal. We share the same boat with them. We need to secure a position where kicking them off the boat into the water doesn't mean the boat sinks with us in it.

Doing this the wrong way leads to irreversible consequences that last far longer than the Epstein class would. And millions of innocent people would he caught in the mix.

They have us hostage right now.

3

u/MathematicianOk366 14h ago

This happened in one of the seasons of Westworld it was pretty nuts

2

u/BasvanS 14h ago

At least it levels the playing field. Now it is just being used against us in unknown ways.

I’m not saying it’s great, but we are in unknown territory as it is.

4

u/Hour_Wonder_9240 14h ago

Yea it's scary. I personally believe it wouldn't level the playing feel, it would create a long lasting scar that harms democracy.

People deserve privacy because humans have shadows, and should have the dignity of being able to overcome their shadows without the whole world knowing the details.

Do we really want a world where that part of humanity becomes inaccessible and we all blindly follow rules out of fear. Rather than self determining of how we should live based on overcoming and becoming good through that.

2

u/BasvanS 12h ago

It feels scary indeed. However, Kahlil Gabrin has an interesting observation about this: “Should we all confess our sins to one another, we would all laugh at one another for our lack of originality.
Should we all reveal our virtues we would also laugh for the same cause.”

I think this would finally kick us into action to fix some ails in modern society. For now, they’re only silently abused.

2

u/CankerLord 10h ago

The "private chats of an entire society"? I don't think the ramblings of the sort of person who yaps personal secrets at Claude are "destabilizing". What gets inputted into these services is a tiny slice of the world's information and it'd be a blip and nobody would care other than those with specific, actionable information that got stolen.

Any anxiety along these lines is overblown and driven by bad estimates of the value of the information being collected.

1

u/MR_Se7en 14h ago

I guess you've never heard of a coverup.

1

u/Hour_Wonder_9240 14h ago

I've considered that too. Part of me hopes if such a leak occurs people at the top would work fiercely to contain and erase it.

I'd be more concerned if the people in charge right now determine the chaos of not covering it up but letting it happens benefits them. The alternative for them is prison and for some maybe capital punishment for their crimes.

We must get sane normal people in charge because we need them at the helm if something like this happens.

1

u/cultish_alibi 12h ago

if a massive data leak deanonymizes the private chats of an entire society

It's already not anonymous, the NSA and many corporations have access to all that stuff. There is no data privacy, it's a joke, a delusion. Google sells it all to whoever wants it, same with all the AI companies. Where is this anonymity you are talking about?

1

u/roleplayersir 10h ago

Serves them right for using the AI. As we know this WILL happen in the future. Every company gets leaked

1

u/off-with-your-thread 8h ago

There have been digital journals that could've been hacked for decades.

→ More replies (16)

10

u/jupfold 14h ago

How dare they invade my privacy when I EXPRESSLY….gave them my consent to do so

1

u/rebbsitor 11h ago

I think the real point has nothing to do with this specific incident. It's that AI systems review what you submit, humans can end up reading it, and it can be shared with 3rd parties.

In other words, there's no privacy with these systems. Whatever someone types into them can be flagged based on any criteria for human review and forwarded to outside parties someone didn't think would ever see them.

In this case it's someone threatening to do something illegal, but the criteria could really be arbitrary.

1

u/Firewolf06 6h ago

well they tell you that outright (both in the terms and usually in the message ui) so anyone thinking nobody else can see it is, frankly, below average

41

u/mcampo84 13h ago

Lol even a therapist would be required to report this

10

u/rationalsarcasm 13h ago

flashbacks to the lesbian bar zoom meeting that hit the news a couple weeks ago...

6

u/FREE-ROSCOE-FILBURN 10h ago

Was she wearing a KN95 when she typed the threat?

1

u/mojambowhatisthescen 10h ago

lol what was this?

2

u/helium_farts 8h ago

Basically a lesbian bar with a strict mask requirement went mask optional one night a week to boost business, and a bunch of extremely online weirdoes had a meltdown

The bar is called the Last Ditch if you want to look up all the drama (including how one of the bar's founders left to attend clown college)

3

u/ensemblestars69 8h ago

Honestly clown college is the most normal part of this. In fact a lot more normal than most people think, as it's become a niche trend in parts of the internet to do. It's basically like a silliness / improv workshop

17

u/EngineerRare42 13h ago

But not the deputy's office?

8

u/drinkallthecoffee 10h ago

She wanted to shoot the sheriff but she did not want to shoot the deputy.

6

u/FatherSquee 11h ago

Well look at what happened in Tumbler Ridge in Canada; guy used AI to plan shooting up a school and now they're being sued for not doing something to stop it from happening.

24

u/sunfacethedestroyer 14h ago

Yeah, but it's still an interesting case. Based on the article, it is only a crime if it's posted in a manner in which someone can view it.

It wouldn't be a crime if she put that in a regular paper diary. And I doubt she knew it would be reviewed by a person.

16

u/ungoogleable 12h ago

That sounds like a weird quirk of the law they charged her with, not a reason to withhold reporting though.

Suppose she wrote her credible plans to shoot up the sheriff's office on a piece of paper then lost it on the street where you found it, shouldn't you still tell them about it? Even if planning to shoot up the sheriff's office is not technically a crime in itself and she didn't explicitly make a public threat, I believe Anthropic should report when someone credibly intends to commit harm to others.

0

u/f_myeah 11h ago

It's impossible to decide if intent is "credible" without further physical evidence (stockpiling weapons etc). We're getting dangerously close to "pre-crime" a la Minority Report if we're charging people for their thoughts, whether recorded on paper or not.

5

u/ungoogleable 10h ago edited 9h ago

We might argue about what constitutes the threshold for credibility, but to say there's no threshold at all is a pretty extreme position that I don't think is defensible. The more blatant, consistent, clear, specific, and unambiguous about their intent somebody is, any reasonable person is going to say it's credible. There are already many roles in society that are legally mandated to report such things (such as therapists).

Edit: We should also separate reporting from punishment. I agree merely having the mental intent to commit a crime should not trigger the same punishment as having committed the crime. But reporting might trigger actions other than criminal punishment, such as a mental health check or proactive security for the target.

2

u/f_myeah 7h ago edited 7h ago

I think you brought the discussion back on track by distinguishing reporting from convicting.

Had she actually shot up the sheriff's office, a diary entry could be used to prove mens rea. Mens rea is only half of what should be required for a conviction though, the other half being actus reus, the guilty act.

She committed no actus reus and therefore should not be charged with a felony.

4

u/drinkallthecoffee 10h ago

She used it like a therapist, and then she was reported like a therapist. If you told your therapist, you were gonna shoot up the sheriffs office they would report you to the police.

1

u/sintaur 8h ago

The communication must be made in a manner in which another person may view it.

Yeah that's going to be trouble for making the charges stick

1

u/gigdy 6h ago

They probably just needed a charge that they could use to detain her while they investigate if she is truly a threat.

14

u/themiracy 14h ago

I think on the one hand if they prevented what could have been a mass casualty event, that’s clearly a good thing.

Per the article: “The communication must be made in a manner in which another person may view it.”

I am curious about how this is being applied - is it being applied in such a way that in principle someone from the host company could look at your cloud account (iCloud drive or OneDrive or whatever, sometimes even involving files that the user doesn’t know are being put in the cloud) and because they could even though by default no one would have access to it, it constitutes a transmission covered by the law? I think more than one thing can be true - that this can be problematic from a privacy standpoint and also that it can be good a potentially serious crime was prevented here.

10

u/throwthisidaway 9h ago

IMO the law as written is unconstitutionally vague AND overly broad. It appears to cover virtually all writing, both analog and digital. It can cover everything from writing in a physical journal, to a digital painting, even a news article.

As used in this section, the term “electronic record” means any record created, modified, archived, received, or distributed electronically which contains any combination of text, graphics, video, audio, or pictorial represented in digital form, but does not include a telephone call.

Not only does that cover all the above examples, but it also does not require the threat to be real or even plausible. Writing for instance that "I'm going to transport myself into the My Little Pony world and kill all the people there" on Reddit fulfills the requirements of this felony.

It is horribly written.

2

u/themiracy 9h ago

I mean this is Florida trying to regulate tech. They do be wildin' down there.

2

u/RedAero 7h ago

We have genuinely reached a point in time where physical letters and phone calls are significantly more private than encrypted digital communications. The government can't tap my phone without a warrant, but they can secretly subpoena Facebook and pull all my chats, not to mention simply monitor everything all the time as Snowden told us a decade ago.

2

u/throwthisidaway 6h ago

encrypted digital communications

Well, properly encrypted communications are still safe. Real E2EE (end-to-end encryption) works great. Unfortunately, companies like Meta/Facebook claim they use E2EE, and technically do, but the implementation is insecure. Signal for instance is generally suggested for those who are seriously concerned with safety.

1

u/RedAero 6h ago

Well, properly encrypted communications are still safe.

Probably, but not a week goes by without some twat of a politician suggesting mandatory backdoors on the basis of protecting children or whatever. For some reason they're not suggesting that letters and packages all be opened and all phone calls recorded. Maybe because that's already happening illegally, who knows.

And of course encryption being secure is still an as-far-as-we-know thing. The US government managed to hack an airgapped Iranian computer in their nuke program, I wouldn't put anything past them. Not to mention the fact that encryption is only worth a damn if the endpoints are secure; Pegasus ring a bell?

8

u/mroada 13h ago

No, it is not a good thing. Means don't always justify the ends. Now you're happy they prevented someone getting shot by spying on private conversations, next someone will say same thing should happen when eg. gays want to organize a date or a woman wants to get an abortion.

15

u/zmcwaffle 13h ago

You’re talking to a corporation, “Claude”/Anthropic as an entity is the other party to your conversation, so they are not spying on you

1

u/mroada 11h ago

The government in that country is spying by forcing Anthropic to disclose private information to them

1

u/greg19735 8h ago

you know that morals don't have to be absolute like that right?

THinking this is okay and that's not okay isn't hypocritical. It would be the policies that require some notification for abortion or whatever that are a negative. or the laws that say abortion is illegal.

and ofc, this wasn't the government spying on their computer's journal. it was someone typing it to a chat bot.n

1

u/RedAero 7h ago

you know that morals don't have to be absolute like that right?

They do if you want to keep your rights.

and ofc, this wasn't the government spying on their computer's journal. it was someone typing it to a chat bot.n

This was the government mandating that the chat bot conversation be reported to them.

1

u/helium_farts 8h ago

by spying on private conversations

They're not private, though. Use them if you want, but don't say anything you don't want other people to read (and that goes every online service--including reddit.)

gays want to organize a date or a woman wants to get an abortion.

Neither of those groups should be using chatbots for those things if they live somewhere where doing so is illegal, because again, the services are not private.

2

u/8Deer-JaguarClaw 13h ago

I wonder if she had planned to attack a citizen if they still would have reported it?

3

u/ungoogleable 12h ago

They certainly should. The article mentions they have human reviewers evaluating similar cases.

2

u/aski5 12h ago

redditors in a parallel universe: “so disgusting that these out of touch tech bros knew about it and didn’t do shit”

2

u/BravinatorLX2 12h ago

so what? they're not the precog police.

2

u/lzwzli 4h ago

If someone wrote the same thing in a notes app on an iPhone, is Apple liable?

AI companies can structure their services similar to companies that host lawyer data or medical data in that the data is not owned by the hosting company and the hosting company has no right to act on that data.

2

u/OkWillingness2817 12h ago

They didn’t. Some looney American wanting to shoot some place up is just Monday. Might be news that it’s not a school for a change.

The fact that LLM’s can report your input is the bit people need to be aware of.

1

u/King_of_the_Kobolds 12h ago

Okay but like

In those cases where a burglar steals a laptop and reports it to the police because they found child porn on it, the burglar did the right thing, but only because they did the wrong thing in the first place and looked at data they shouldn't have had.

1

u/SimoneNonvelodico 8h ago

Does that even happen/work out? You'd think that would be inadmissible evidence in court.

1

u/amyowl 11h ago

They're going to have to store our data in order to provide a service, therefore they should be mandatory reporters just like social media

1

u/zanii 10h ago

People can easily decide to just not put personal info into these services. Nobody is forcing it. I don't get what compels people to just throw every detail and secret at them. I use AI a lot, for work and at home, but ai don't put anything on there that I would not mind going public.

But, then again, I do work in the field so I know what to expect. But then again, again, people SHOULD have learnt from Facebook.

1

u/502DashCam 9h ago

That should be MANDATORY reporting. That is TERRORISM

1

u/powercow 8h ago

will they tell us if some politician asked whats the best way to steal the election.

1

u/jeo123911 7h ago

Just register AI bots as priests and they'll be exempt from reporting. Case solved.

1

u/MrIrishman1212 27m ago

The thing is, if we made them mandatory reporters like schoolteachers and healthcare workers, they would lose most of their valuable customers cause most people who are obsessed with ai are mentally unwell and need to see a psychiatrist.
You are basically trying to request bars to report alcoholics or casinos report gambling addicts.

They should but it will be a long fight to get to that point

1

u/LiamJamIsMyNeesons 13h ago

No, that's irrelevant 

1

u/McFresch 13h ago

If she were a guy this would've been in the headline and all anyone would be talking about. Online privacy wouldn't even be a factor

→ More replies (4)