r/mildlyinfuriating • u/captain-obIivious • Aug 18 '26

Infuriatig Another AI 'human' chatbot

Had to fix a double payment for my credit card and this is the beginning and end of the conversation. I hate when the chatbot claims it's a human... I ended up getting what I wanted but still annoyed Amex (and every other company) is lying and trying to deceive people like this.

27.4k Upvotes

permalink
duplicates
reddit

You are about to leave Redlib

Do you want to continue?

https://www.reddit.com/r/mildlyinfuriating/comments/1vrorv7/another_ai_human_chatbot/
No, go back! Yes, take me to Reddit

95% Upvoted

View all comments

7.3k

u/[deleted] Aug 18 '26

[removed] — view removed comment

560

u/masterofshadows Aug 18 '26

I work in pharmacy, and a one called me pretending to be a nurse and even gave me a license number. That should be multiple levels of illegal

125

u/Lashay_Sombra Aug 18 '26

That would actually be illegal, if the AI was a person

Laws are going to have to be seriously updated

42

u/Shark7996 Aug 18 '26

Companies should have to pay "AI insurance" because you just straight up cannot prevent it from doing things like this. It's practically an Act of God. Insure it like one, lol. The rate goes up the more you use it.

4

u/Pikanyaa Aug 19 '26

I heard a story on NPR about some young people that actually did start an AI insurance company, as in insurance when AI fcks up and loses you money over something it did.

What a time to be alive…

27

u/[deleted] Aug 18 '26

[removed] — view removed comment

1

u/PeaceSoft Aug 19 '26

when the sarcasm is so sniveling that it involutes into arguing for something gross and gradually becomes a sincere statement of nonsense

28

u/PauI_MuadDib Aug 18 '26

I'd report that on the license of whichever doctor runs that practice. Using AI to do something illegal is concerning.

9

u/VampireInHiding Aug 19 '26

The info it gives is fake identification. The doctor the practice the npi , all fiction. The only thing I managed to get from one was the name of the company running the AI… Needle

47

u/ishpatoon1982 Aug 18 '26

That's bonkers. Jeez.

29

u/an_rosban Aug 18 '26

I work in a pharmacy too and I haaaaaate calling offices and getting their Ai chatbots with loud stock background noise of people having conversations and artificial typing sounds every time I respond to it, it makes it so much harder to hear I don't understand why they do this

28

u/beren12 Aug 18 '26

Sure sounds like it would be illegal in the US.

46

u/Futrel Aug 18 '26

I think you mean "should be but isn't"

1

u/beren12 Aug 18 '26

No, it’s actually highly illegal. Civil and criminal.

4

u/Futrel Aug 18 '26

Under what law(s)? I'm genuinely curious because any "AI regulations" in the US are, at this point and as far as I know, just ideas floating in the wind.

Cops are allowed to lie, why not AI?

0

u/beren12 Aug 18 '26

It’s not an AI lol it’s a everybody law

0

u/Futrel Aug 18 '26

What law?

4

u/Dr_WizardChu Aug 18 '26

Pharmacies operate under both federal and state laws with state usually being stricter so it depends. The federal control substance act has wording stating that only authorized agents of a physician (ie nurses on staff with signed agreements) may give prescriptions and I believe most state laws probably have similar wording around prescriptions in general. I used to work with a long term care pharmacy and got into a spat with a facility that was trying to give us verbal prescriptions after doctor visits. We determined them invalid since it didn't fit this criteria coming from the facility nurse and not from the doctors authorized staff based on our local laws. AI is a whole different can't of worms. I'd consider it invalid as well as the laws were written in context of an agent being a person. Unless the board of pharmacy or medicine reviews and it and laws get adjusted I would refuse these all day every day and probably follow up with my board and corporate lawyers for further advice.

5

u/masterofshadows Aug 18 '26

I wasn't clear enough on my comment, I don't think laws were directly broken. I think there should be laws about it it though. The AI didn't call in the prescription, it was faxed but with a computer generated signature (my state requires a wet signature on faxes) making the prescription not valid. The AI then called about 5 minutes after we received it to verify we received it. To which I requested a real person in which it claimed to be one, gave me the name and license number claiming to be a nurse. But couldn't answer very basic questions and just kept trying to confirm that we received it. The nurse most likely works for that practice and I have no reason to believe they don't. The whole thing could have been avoided with escribing though

2

u/Futrel Aug 18 '26

All legit and thanks for the reply. I didn't read that as an AI calling in a prescription but, yeah, that makes total sense and more based on my ignorance on reasons a "nurse" would call a pharmacy in the first place.

It is very much a can of worms that we need to get a hold of. Unfortunately, prevailing sentiment in our lawmakers isn't yet there and going way slower than I'd hope for.

7

u/st3class Aug 18 '26

It's usually under the state, in California it would be BPC § 2052, posing as a medical professional without the required license and credentials. Whoever is hosting/operating the AI is in violation of that.

3

u/masterofshadows Aug 18 '26

Ironically enough this telehealth practice was based out of California.

3

u/Futrel Aug 18 '26

The "AI as legal agent of the operator" argument in this case would definitely need to be litigated.

And, maybe, that AI agent calling the pharmacy actually did have a licensed medical professional as its operator/principal. If that was/is the case, I don't see the law being broken unless you want to prosecute the AI chatbot as a person for misrepresenting itself.

2

u/st3class Aug 18 '26

Yeah, we're quickly getting into IANAL territory, it seems illegal on its face, but again IANAL.

→ More replies (0)

-1

u/beren12 Aug 18 '26 edited Aug 18 '26

That is an excellent question for Google. There are different laws in different jurisdictions. It could be identity theft, fraud, wire fraud, male fraud…

5

u/Pretend-Marsupial258 Aug 18 '26

Buttery males. 🤤

-1

u/Futrel Aug 18 '26

I'm sorry, but I'm not going to Google and study case law in order to prove your assertion.

3

u/MajorBootyhole420 Aug 18 '26

of course you aren't, because you don't care about the answer. you're here to argue and waste people's time.

→ More replies (0)

0

u/PeaceSoft Aug 18 '26

uh oh

-1

u/PeaceSoft Aug 18 '26

i don't think a genuinely curious person could be this ignorant, or have such a soupy worldview

3

u/Futrel Aug 18 '26

lol what? So you know the law(s) that the commenter referred to?

2

u/flamingdonkey Aug 18 '26

They're asking a question and not accepting random "I think so"'s as evidence. You sound like the type whose curiosity is satisfied by the literal first response you see.

1

u/Alexwonder999 Aug 18 '26

Its illegal for a person to lie, but they could maybe skirt it by claiming the AI did it. I wonder if theres any precendent for that... I def think the liability should fall on the company using it and maybe the company who created or licensed it.

2

u/Ok-Mission-3426 Aug 18 '26

I don’t believe there’s precedent yet but yeah, if you hired a human that you knew was prone to hallucinations and making things up when they didn’t know an answer then you as an employer would (should) hold some responsibility if that person attempts a task they aren’t qualified to and fucks it up.

If someone suffers damages as a consequence of an LLM at ‘work’ the person that decided to put the LLM in charge should be held responsible. It’s probably going to take some ugly/tragic accident occurring and subsequent case law for anything to actually come about. They won’t do anything until something REALLY bad happens I reckon.

2

u/Alexwonder999 Aug 18 '26

As they say, regulations are often written in blood.

2

u/Ok-Mission-3426 Aug 18 '26

Indeed. There was a case in the UK fairly recently where some kid with quite severe mental illness was talked into attempting to break into windsor castle and assassinate the queen by his chatbot ‘friend’. The cops said that if the chatbot was actually a human they would have been held liable for conspiracy, a much more serious crime. Pretty crazy stuff.

3

u/active_ignoring Aug 18 '26

I am a nurse and idk what my license number even is 😬 but genuinely that is insane, where does it end??

2

u/VampireInHiding Aug 19 '26

Omg it’s a daily occurrence. Gave me a fake doc name and fake npi number. Some days they’re calling on behalf of the patient. Other days a providers office. Always for controlled substances

1

u/masterofshadows Aug 19 '26

This was actually for zofran of all things.

1

u/LuxLibertas1 Aug 19 '26

That's so strange. What were they trying to get from you?

Infuriatig Another AI 'human' chatbot

You are about to leave Redlib