How Claude marks AI-generated content: (1) watermarks embedded in text, and (2) signed provenance metadata attached to files.
https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content
log in to comment.
https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content
log in to comment.
The text version must necessarily constrain the output, which will impact readability and semantic meaning, whatever they say.
Not necessarily, there are valid “characters” that are not rendered in most text parsers, or are rendered as whitespace. A great example of this is the byte-order-mark (BOM). You can embed a BOM code in the whitespaces of a text string and it looks exactly the same to a human as one without.
I assume that’s what they’re talking about here
Here’s an example of a steganography encoding technique using this method. 330k.github.io/…/unicode_steganography.html
Maybe, if it’s created after the text is generated. Keep in mind there really is no white space to an LLM. There’s a lot of holes here. If it isn’t embedded within the text itself, it is trivially easy to strip such markings. If it is, it at least requires a human to paraphrase.
On the other hand, if that’s all it is, it would be easily detectable to disallow AI from editing human-written code while allowing it to revise its own output.
A model could hypothetically be trained to insert zero-width characters (I doubt any have though). But any other layer could also very trivially insert these codes. The inference engine could be designed to delay output streaming by however many tokens is required to embed their coding and ninja-insert them during the decode stream. A proxy between the inference engine could insert them. A harness could insert them. Hell, even the rendering javascript frontend in your browser could insert them.
Either the inference engine or proxy would be the prime target if they want to enable this on api responses as well as copy/paste from a chat interface. They could also do a combination of the above depending on final output mode.
You are definitely correct though that it’d be trivial to detect and strip by someone aware of it.
Nah, token biasing can be really subtle. Enough pairs of one word being slightly more probable than the other, and it’s imperceptible, and below the “noise floor” of the sampling itself.
Can someone ELI5 how this would actually work, especially with copying and pasting?
(Sorry, afk so can’t post specific animal example image).
So, what Claude is likely doing is biasing these word probabilities. Take our example. Let’s say the top 4 words are:
“Cat”
“Dog”
“Horse”
“Fox”
Claude can bias “Dog” and “Fox” to be slightly more likely than “Horse” and “Cat.”
It doesn’t seem like much. But do this for ALL of Claude’s vocabulary, for many thousands of possible words, and you make an embedded “fingerprint” for an LLM, where it slightly prefers a random half of the dictionary so subtly it doesn’t color output to humans, but is statistically detectable if you know which half it prefers.
I believe one aspect to this is that the “dictionary” is a secret, otherwise bad actors could theoretically compensate for the bias.
This is just to catch the lowest common denominator, though, so I think Claude should publicize it anyway. 99% of people abusing Claude would never know any of this.
If you’re still curious, I’d suggest visualizing sampling yourself with Mikupad: github.com/lmg-anon/mikupad
…You can’t actually use Claude with it though, as they hide the output probabilities of their models because they’re jerks who think LLMs should be black boxes their users don’t understand.
You know how these AIs output tokens? I’m going to explain the concept with words, because that makes it easier to understand. But it means that the explanation is [ETA:]NOT quite right.
An AI has a vocabulary. The words in that vocabulary are assigned to 2 groups.
Sometimes when the AI is outputting something, it could use different words equally well. It’s not quite the same as having synonyms, cause this isn’t really about words. But let’s say you have synonyms in those different groups. At those points in the text, you can pick from one group or the other to embed a hidden pattern in the text.
Limitations are obvious. To embed the watermark, you need enough opportunities to pick “synonyms”. It won’t work for very short texts, or if the word choices are very constrained.
I’m curious if the negative effects are really as minor as they say.
my guess is something like this
330k.github.io/…/unicode_steganography.html
[removed]