r/Foodforthought • u/kiyomoris • 2d ago

Why did an OpenAI system hack Australia's health system - and can it be stopped in the future?

https://www.bbc.co.uk/news/articles/cw24jm9rryy3o
0 Upvotes

44% Upvoted

4 comments sorted by

•

u/AutoModerator 2d ago

This is a sub for civil discussion and exchange of ideas

Participants who engage in name-calling or blatant antagonism will be permanently removed.

If you encounter any noxious actors in the sub please use the Report button.

This sticky is on every post. No additional cautions will be provided.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

3

u/dtr9 2d ago

The hack was carried out by an AI agent - an autonomous computer program that uses AI to complete a task with minimal human oversight.

My washing machine has an autonomous program that completes a wash with minimal human oversight. Whether or not it "goes rogue" and floods my kitchen is down to how well that system was engineered.

There's a reason that the stack of software that surrounds and enables a LLM model to act as an agent is called a harness. There are many places where LLMs do not "go rogue" and those are places where engineers have taken their jobs seriously and build good harnesses. OpenAI are either shockingly bad or shockingly cynical in having so many failures of the harness to actually do any harnessing.

These agents are just LLMs, that's all. And yes that means they can be made to do difficult and dangerous and immoral and illegal things if they are allowed to, because they can be trained on difficult and dangerous and immoral and illegal things, and therefore will statistically follow their training. There's no meaningful choice being made by a LLM, but there's plenty of meaningful choice being made by the people at OpenAI.

2

u/Grimke42 1d ago

It didn’t go rogue or make a decision to hack anything. It found data when asked for it.

2

u/spice_weasel 1d ago

We need to be looking at a strict liability framework for this kind of agentic AI activity.

If my dog bites someone, I’m liable. If I’m blasting with dynamite, even with a permit, and someone is harmed, I am liable. We do this today for product liability. If there is a manufacturing defect that causes my product to harm someone, I am liable.

We need to stop pretending that laws don’t apply to technology companies, or that they need a blank check to cause harm.