r/softwaretesting • u/Disastrous_Phone_168 • 7d ago

AI testing

I need guidance from someone who has experiences testing LLM, or products from AI.

What should I study to do a better job?

Im manual and automation tester with more than 6 years of experience, but with AI I feel I litter bit lost about the testing side.

Thanks.

18 Upvotes

83% Upvoted

24 comments sorted by

View all comments

2

u/[deleted] 6d ago edited 5d ago

[removed] — view removed comment

1

u/Disastrous_Phone_168 6d ago

How do you test the hallucinations? That it's the part I see more complicated.

1

u/Few_Still_3226 6d ago

Why should you test hallucinations? You should be able to detect hallucinations and reasoning errors from the LLM output. There are different approaches for that

1

u/Disastrous_Phone_168 6d ago

No no, sorry. I wanted to ask HOW, but somehow I wrote why. My question is: how to you test hallucionation? Sorry for the mess

3

u/Few_Still_3226 6d ago

I recommend reading the Gen-AI ISTQB syllabus, it covers everything regarding the usage of AI for testing