Florida woman used Claude as a diary, then Anthropic reported an entry to police

[u/Miserable_Phase_2519]

https://www.techspot.com/news/114091-florida-woman-used-claude-diary-anthropic-reported-shoot.html

[u/Warm_Month_1309]

if we applied this logic 100 years ago the rule would have been that someone has to listen in on all your phone calls or the phone company is liable if you ever use one to organise a crime

My standard wouldn't be "you are required to listen in and report everything"; my standard would be "if you are choosing to listen in, then you've also chosen to make yourself liable when you don't report something you should have".

[u/Z00111111 → u/Warm_Month_1309]

"Claude, delete any logs of us reading transcriptions of criminal or planned criminal activities"

[u/CPSiegen → u/Warm_Month_1309]

Who defines what "should" be reported, though? "Should" these AI hosters report, say, domestic abuse even when they have no idea if the user is making stuff up? What about child abuse if a kid says their parents are monsters? Should they report things ICE thinks are crimes? Should they report dissident behavior in countries that label dissidents as terrorists? Should they report unregistered business activity in countries that take state controlled commerce very seriously? Should they report homosexual thoughts in countries that outlaw homosexuality?

At what point are we promoting anyone ingesting our data to act as thought police on behalf of the worst government policies? What infrastructure will we have to ensure they aren't selectively reporting certain kinds of activity and not others (for instance, reporting things that ingraciate them with the current administration or only things that don't impact their profits)?

[u/Warm_Month_1309 → u/CPSiegen]

Who defines what "should" be reported, though?

Their legal counsel, and then later if someone disagrees, a court on a case-by-case basis.

At what point are we promoting anyone ingesting our data to act as thought police on behalf of the worst government policies?

I'm not promoting that. I'm offering two alternatives: either do not read user data and remain ignorant of its contents, or do read user data and be responsible for your knowledge of its contents. What these companies cannot do is claim the benefits of both.

[u/Bauermander → u/Warm_Month_1309]

Your alternatives are not realistic. AI doesn't work without reading the content. You know very well that if they have legalized access to user conversations, they'll use it to do a lot more than to track mass shooters. They're probably gonna track them anyway in usa, they're just not able to use it as a proof in legal cases.

[u/CPSiegen → u/Warm_Month_1309]

I'm arguing that the latter option is the exact smoke screen that lets them get away with doing both.

You see that now with companies requiring ID verification. They're promising that it's for the purpose of rooting out bots and predators. They're promising that it's not going into a database to tie all your information together more strongly. They're promising that they're deleting your ID info as soon as you're verified.

And they've been caught lying about each and every one of those promises. The idea that they're acting in good faith is the lie. If you give these companies an option to spy on all your data in the name of security and law enforcement, they will never act in good faith on that premise. They will absolutely still spy on you as much as possible and they will absolutely shirk and outright bury any responsibility to act on what they find, so long as what they find doesn't directly help them to report.

The discussion needs to focus only on the option that we can regulate mandatory and invisible data harvesting out of existence. Anything else will be twisted to do exactly what you don't want.

[u/Warm_Month_1309 → u/CPSiegen]

lets them get away with doing both

There are too many novel lawsuits currently working their way through our courts to conclude that.

[u/SimoneNonvelodico → u/Warm_Month_1309]

Well, but it's arguable whether they are "listening in". They're storing data but no human is ever looking at it. That said, ok, suppose the user was paying for the subscription that allows you to turn off data retention - then would it be ok to drop the requirement? Or would someone still sue and complain?

[u/Warm_Month_1309 → u/SimoneNonvelodico]

They're storing data but no human is ever looking at it.

"After a human reviewer examined the statements, they were reported to police."

It's not arguable whether they are listening in. You explicitly given them permission, they say they do it, and they report things you say to the police. They are listening in.