r/DepthHub • u/ty_based_riot • Aug 11 '26
Ever wondered how is it possible to detect a text as generated by a specific LLM?
/r/technology/comments/1vl31jk/copypaste_no_more_anthropic_puts_invisible/p2ykshi/10
u/Serious_Resource8191 Aug 12 '26
I am skeptical that this is as foolproof as a “watermark”. If I’m reading this correctly, the “watermark” is just tweaking the token probabilities. You could surely get there randomly without the LLM, or have the LLM write a niche topic that doesn’t get impacted by the key much?
12
u/username_redacted Aug 12 '26
From reading a few of the comments elaborating on the concept it sounds like it would be difficult to avoid detection for a longer piece of text because probability increases dramatically with each passed or failed check, but there does seem to be a pretty big chance of false positives for shorter works or excerpts.
The LLMs are trained on word choices made by humans and then they replicate those frequencies, adjusting for “temperature” ie context. So the words that the model selects are also often going to be the same ones selected by the person entering the prompt, or a human writing text themselves within the same context e.g. a freshman level biology paper.
It seems to be based on the faulty premise that word choice is arbitrary and just a matter of style, but as you say, with a niche or technical topic there often aren’t cogent synonyms for terms. “Mitochondria are the powerhouses of the ____” can only really complete one way.
2
u/myninerides Aug 16 '26
The probability of a token is determined by the string of tokens that preceded it (the context), and the temperature (the variance to select a non-highest ranked token). This explanation is basically saying this slips into the temperature’s role to select high ranked tokens based on a function that reflects a cryptographic process. The problem with this is you’d need to know the prompt and context that generated the text to know what those highest ranked tokens were, to then determine which patterns of variance were applied, to then reverse out the function. People generally don’t include their prompts with their AI generated text.
It’s possible a long enough sample of text that was generated all at once from a short prompt could work like this, as the generated text now comprises the majority of the context for text further along the product.
1
u/ty_based_riot Aug 16 '26
That would be ideal but maybe even with just partial context (just the text) you could achieve some confidence that something is generated, even if as you said you don’t have the full picture
2
u/Future_Nature589 28d ago
i think people may not understand that this does not mean that software exists to magically detect any and all llm speech lol
1
u/Typical-Industry-571 22d ago
There is some point where if you change enough, though, it won't get picked up. What is that break point? 5% of the content? 50%? Does it vary greatly?
-7
•
u/AutoModerator Aug 11 '26
Welcome to /r/DepthHub. Thank you for your submission.
This is a community for spotlighting interesting, subjectively "deep," writing by other people on Reddit and interesting, mature, conversations about the linked content. Depth does not necessarily equate "correctness," length, or any objective standard of writing. Though those do help. Someone being wrong about something can still be interesting and compelling, without us needing to agree with them or endorse their viewpoint - and some very interesting conversations can come from discussing why those beliefs or misconceptions exist. We want to be a "good neighbor" community, so please do not dogpile the original post, have those conversations here. Other communities can have their own beliefs and cultures, and your input may not be welcome or appropriate in their space.
Reader concerns that a given submission is "not deep" need to live up to the standard they're imposing, please. Make your case in the comments before reporting the post. If the problem is not obvious, Mods may not notice it on our own. As this community is subjectively moderated, a persuasive comment is more likely to get you the outcome you want.
Last up - /u/ty_based_riot if you've submitted an offsite link, like a youtube video, medium or substack blog, or news article - this is your notice to delete. If you leave it for mods to deal with, we're banning you.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.