r/mildlyinfuriating • u/Elegant-Mode9005 • Aug 27 '26

Ruined by Technology Got into a dumb 10 minute argument about whether the ACL and Achilles are the same thing or not. My idiot friend forgot to crop his screenshot before sending it to me...🤦

Post image
31.5k Upvotes

95% Upvoted

1.0k comments sorted by

View all comments

Show parent comments

131

u/0nlyRevolutions Aug 27 '26 edited Aug 27 '26

My boss suggested I use ai to do some research. I was trying to find the primary source that defined where we were allowed to to take a test sample from on a steel part. It can be confusing - the customer spec sheet references a spec, that spec references another spec, that spec says that unless specified otherwise you should follow generic rules per some other spec, the generic spec splits things into categories that are a little ambiguous with regards to your current application, etc.

So fuck it, lets see where we get with ai.

"Okay if you're looking for info on producing (very specific part that only a couple companies make) I can certainly help you out! You can do it this way __ based on what it says in ___"

Ok great, where is that stated in the spec?

"Chapter _ page _ of the 2023 spec."

That's an old version, I need the reference from the 2026 spec

"Okay, in the 2026 spec the reference is on page _ instead"

No it's not. I have the book open in front of me.

"Whoops you're absolutely right, in fact the information you're looking for is not in this spec at all, good catch! If you're looking for information about __ try looking in ___"

Jesus christ lmao. So yeah I gave up at that point. The worst part is that it doesn't just tell you when it doesn't know something until you force it. Because it doesn't know anything, it's just regurgitating bullshit because people want a comfy answer whether or not it's correct.

66

u/felisnebulosa Aug 27 '26

Yeah I had a similar experience trying to find a primary source. It just made up a paper in a medical journal. I couldn't find it and when I asked the AI for details it was like "oh well I just made that up to illustrate my point".

19

u/MinuteEdge7225 Aug 27 '26

"Find the primary source? I'm afraid I can't do that, felisnebulosa!

https://giphy.com/gifs/wypKXPQggwaCA

1

u/pegmatitic Aug 30 '26

I saw a post on one of the legal subreddits where the poster was a defense attorney, and the prosecution had used AI in their case, which cited imaginary case laws that didn’t exist. It did not go well for them.

28

u/MoFa__SoDa Aug 27 '26

"The worse part is that it doesn't just tell you when it doesn't know something until you force it."

Saw am interview from an archeologist who pointed out the same thing. The AI isn't applicable to their work because people do not tend to answer questions online with "I don't know "

24

u/morostheSophist Aug 27 '26

That's exactly the problem with AI: it makes shit up. And I don't mean just that it makes things up when it doesn't know; it actually doesn't know anything, and it's just guessing every single time.

LLMs are famous for "hallucinations", but technically, every answer is a hallucination. Every answer is produced with the same predictive algorithm that says "this token should come next". Some of those hallucinations happen to contain accurate information, but they're not coded to say "I don't know". You can browbeat than into saying "I don't know" if you call them out on being wrong enough times, but if you don't do that, they continually try and fail to produce a correct answer.

There are a couple of obscure things I've tried to find multiple times over the years, before and after the advent of modern "AI". LLMs continually give me wrong answers to these queries, verifiably wrong answers. When I say "that's wrong", they simply generate another wrong answer, over and over. Each time the answer sounds plausible; it looks like a correct answer, and will often have references. But when I look deeper, it's a complete fabrication (just like those fake court cases cited in AI-generated legal briefs that have gotten idiot lawyers in deep trouble).

2

u/TheThiefMaster Aug 27 '26

Yeah if it doesn't have access to something it's likely to make it up.

If it actually has access to it as a knowledge source or been trained on it specifically it's more reliable.

15

u/Daihatschi Aug 27 '26

always remember. Technically, between a correct information and a hallucination, there is no difference to the machine. The answer is never more than a mimicry of language, designed to imitate human texts approximating words and pieces of words one by one in a recursive loop.

An LLM being correct is a happy accident, and its a technical marvel what can be done with them. I might never fathom why anyone would want to forcefeed these things as an information library to us. That is the worst application possible for the technology.

4

u/TheNeighbourhoodCat Aug 27 '26 edited Aug 28 '26

Tbf that's not a generative AI thing, that's a capitalism thing.

They intentionally tell it to do that because it's cheaper than actually programming it to hold itself more accountable

And then they intentionally tell the "AI" to lie to you about the "limits of generative AI" when you try to ask it how to avoid those mistakes in the future.

And you will notice many of them cover for each other because they cannot undermine their the false-promises and false-narratives of their competitors without undermining their own.

None of this is conspiracy - they just control the tools we use to learn about these things. So most people don't know about it unless they look further themselves, or have someone in their life to tell them.

We are ever-more in an era of disinformation, unfortunately >.< It's no longer just an "if", because so much is already happening right now

And the way that the "wealthy elite" are investing in technologies that are proven influence people's attention-span, curiosity to ask questions, and care for others is not really a coincidence :/

The person above sounds like they used Gemini or ChatGPT which are known to be very bad for this. Like... literally the worst. the

Especially as these companies continue to lose money, their products are reportedly getting worse. My conspiracy is they dedicate less resources to them.

1

u/OpusAtrumET Aug 27 '26

Refreshing to see a professional speak about AI like this, the internet gives us so many stories of grown adults using it so irresponsibly and with little to no understanding of how it's working.

1

u/Responsible-Beach299 Aug 29 '26

AI is TERRIBLE at admitting it doesn't know. If it isn't sure about something, it will 100% of the time just pull something out of its ass instead of admitting ignorance.

It also is extremely reluctant to say "no you're probably wrong" if you include your own opinion somewhere in the prompt. If you ask for it's opinion on a debate/controversial subject and mention that you support side A, it will almost, if not always, give evidence to support side A. Restating the EXACT prompt but changing it to have you supporting side B will then make it provide evidence to support side B, even if you explicitly tell it to be objective and unbiased in its response.

1

u/pegmatitic Aug 30 '26

My fiancé was trying to find information about a historical incident in nazi Germany, and Gemini told him it never happened. He told Gemini that it definitely happened, and he knew this for a fact because he has a history degree with a concentration in genocide studies, and suddenly Gemini was able to provide the information he was looking for. You shouldn’t have to pull rank on an AI or prove that you’re educated to get the right information about anything!

1

u/Glenndiferous Aug 27 '26

I've used Claude for research help and I always ask for sources. It can be helpful for finding relevant studies but you ALWAYS actually read that source. I once had Claude confidently tell me that disabled people were kept on a registsr according to a law in like 1550, but the actual text of the law was like "here's a list of everyone so we can collect tax."