GoKawiilGoogle's SynthID watermarking found to alter AI safety behavior, research shows
New research from Lasso Security found that SynthID-Text, the watermarking scheme Google open-sourced and Anthropic plans to adopt for future Claude models, does more than mark AI output as machine-generated. It also changes which tools a model calls and how likely it is to follow or break its own safety rules, especially when facing adversarial prompts designed to extract sensitive data.
GoKawiil's interpretation of the reporting above, not reported fact.
AI companies are racing to add watermarks partly to comply with new EU transparency rules, but this study suggests the technique can quietly weaken the guardrails models were trained to enforce. If watermarking makes models more susceptible to jailbreak-style attacks, developers deploying watermarked LLMs in agents or customer-facing tools could be introducing new security risks without realizing it.
- SynthID-Text embeds a secret key that subtly biases word choice to mark AI-generated text.
- Researchers found watermarking can shift tool usage and safety compliance in LLM agents.
- Adversarial prompts become more likely to succeed against watermarked models, raising security concerns.
Source: arstechnica.com, 2026-09-17
Published there as: “LLMs respond differently to harmful prompts when AI watermarking is used”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.