Skip to content

Instantly share code, notes, and snippets.

@whit3rabbit
Last active August 15, 2026 16:25
Show Gist options
  • Select an option

  • Save whit3rabbit/75a69725c33197a38d65b130511dd374 to your computer and use it in GitHub Desktop.

Select an option

Save whit3rabbit/75a69725c33197a38d65b130511dd374 to your computer and use it in GitHub Desktop.
Guessing what watermarking might look like in Claude

Model-Level Watermarking in Large Language Models: Anthropic's Claude, Keyed Sampling Provenance, and Watermark Propagation

Summary

In August 2026, Anthropic disclosed how its Claude models watermark generated text. This paper explains the disclosed system, what it does and does not prove, and how to study it experimentally. The main points:

  1. Claude's watermark is a decoding-time, keyed pseudorandom sampling watermark. It is a variant of the SynthID-Text method published by Google DeepMind (Nature, 2024), in the family of approaches that began with Aaronson's 2022 proposal.
  2. The watermark changes only the source of randomness used to pick among candidate tokens during sampling. Nothing is added to the text, there are no hidden characters, no extra tokens are generated, and Anthropic reports no measurable impact on output quality, speed, or cost.
  3. Detection requires a secret key held by the provider, and it yields a likelihood that Claude was involved in producing a passage, not a binary authorship verdict.
  4. The watermark is entropy-gated. It is dense in open-ended prose and translation, and sparse in factual completions, code, and light proofreading edits, because it can only act where the model has a genuine choice among near-equivalent tokens.
  5. The signal is a statistical property of sampled outputs. It can therefore propagate into student models through distillation (watermark radioactivity), and a sufficiently capable student may be able to imitate it (watermark spoofing). A detected watermark is a probabilistic provenance signal, not cryptographic proof of authorship.
  6. The productive research programme is a watermark laboratory: build a stratified corpus, run key-free correlation tests and detector-based calibration, measure robustness under graded transformations, and measure inheritance and spoofability through controlled distillation. Sections 11 and 12 give the experimental design in detail.

An earlier revision of this paper treated the system as a black box and weighed competing architectural hypotheses. That material, now settled by the disclosure, is preserved in Appendix A.


1. Introduction

LLM watermarking embeds a machine-detectable signal in generated content while keeping that signal imperceptible to ordinary readers. The foundational distortion-based approach of Kirchenbauer et al. partitions the vocabulary and biases generation towards a "green" subset. Later work moved to distortion-free constructions, notably Aaronson's cryptographic pseudorandom sampling proposal (2022) and Google DeepMind's SynthID-Text (2024), in which the model's output distribution is preserved, exactly or approximately, and only the randomness used to sample from it is derandomized under a secret key.

The trajectory can be summarized:

Token-level distortion watermarking (green/red lists)
              ↓
Robustness and attack research
              ↓
Unbiased / distortion-free watermarking (Aaronson-family)
              ↓
Production-scale validation (SynthID-Text, Gemini deployment)
              ↓
Regulatory codification (EU AI Act, Article 50)
              ↓
Multi-provider production deployment (Anthropic, others)

With Anthropic's disclosure, the central question for Claude is no longer what the watermark is. It is: given a known keyed-sampling watermark, what provenance guarantees does it actually provide under realistic transformations, distillation, and adversarial imitation? This paper describes the disclosed system, states its security model, and lays out the experimental programme needed to answer that question.


2. What Anthropic Has Disclosed

Two primary sources define the public record as of August 2026: Anthropic's help-centre article "How Claude marks AI-generated content" and the announcement "How Claude's text watermark works" (August 14, 2026).

2.1 Regulatory basis and scope

Anthropic signed the EU AI Act's Article 50(2) Code of Practice on Transparency of AI-Generated Content in July 2026, alongside a large group of signatories including other major model developers. Claude models launched on or after August 2, 2026 support marking at launch, while models launched earlier fall under a legal transition period, with marking support added progressively.

Watermarking is applied globally, not only in the EU; Anthropic states it currently lacks a durable way to scope the watermark by region and will continue evaluating approaches. Marks apply across all product surfaces (Claude Platform (API), the Claude apps, Claude Code, Claude Cowork, and Claude Tag) and across the cloud partners AWS, Google Cloud, and Microsoft Foundry, with some platform- or feature-specific exceptions for certain marking types.

2.2 The two marking mechanisms

Anthropic distinguishes two complementary and technically separate mechanisms:

  1. Embedded text watermarks. An imperceptible statistical watermark woven into generated text at the model level. It survives copy-and-paste and potentially some editing.
  2. Signed provenance metadata for files. C2PA-standard cryptographically signed metadata attached to supported file types (.png, .jpg, .svg). This is not a watermark. Nothing in the file content changes, and the credential can be stripped by format conversion, re-saving, or screenshots.

The rest of this paper concerns the text watermark.

2.3 The disclosed algorithm family

Anthropic has identified the method. Claude's text watermark is a version of SynthID-Text (Google DeepMind, Nature, 2024), within the family of keyed pseudorandom-sampling approaches originating with Aaronson (2022).

The design principle shared by this family is simple. When the model faces a low-stakes choice among near-equivalent next tokens, the choice is still random, but the randomness is derived from a secret key together with a short window of preceding tokens rather than from an arbitrary random number generator. A detector holding the key can later test whether an observed token sequence is consistent with key-derived sampling and assign a likelihood that Claude produced it.

Anthropic's Monopoly analogy is instructive. If dice rolls are replaced with successive digits of pi, starting from a secret offset, gameplay is statistically unchanged for the players. Yet the full move sequence becomes verifiable after the fact by anyone who knows pi and the offset.

2.4 Explicitly stated properties

Anthropic asserts the following properties:

  • No additions to text. Nothing is inserted; there are no hidden characters. This eliminates Unicode steganography as the production mechanism.
  • No practical quality impact. Internal testing showed no effect on content, creativity, or readability. The cited SynthID-Text evidence includes a live Gemini traffic experiment showing no statistically significant difference in user feedback ratings, and a side-by-side human evaluation showing no perceptible quality difference.
  • No token or latency overhead. The watermark produces no extra tokens and has negligible speed impact. Serving cost is unchanged.
  • No user identification. The watermark and its key carry no information traceable to a specific person, organization, or conversation. It is a model/provider-level signal only.
  • Provider-specific keys. Other providers implementing watermarks under the same Code of Practice use different keys, and possibly different methods. Claude's detector cannot identify other models' output, and vice versa.
  • Forthcoming detection API. Anthropic plans to offer a watermark detection API. Implementation details are pending, along with technical documentation intended to support downstream deployers' own Article 50 obligations.

2.5 Entropy dependence

The watermark is entropy-gated. It can only act where the model has a genuine choice among near-equivalent tokens. The stated consequences:

  • Factual completions carry little watermark. After "Isaac Newton's most famous work was called Principia…", only one continuation is correct, so no nudge is applied.
  • Code is generally lightly watermarked, because exactness requirements leave few arbitrary choices. The watermark concentrates in comments and other free-form regions, with negligible effect on functional code.
  • Proofreading and light editing of human text leaves almost nothing for the watermark to attach to. Nearly all words remain the human's, so Claude's involvement may be undetectable.
  • Translation is fully watermarked, since every output word is chosen by Claude.
  • Short passages provide too few watermarkable decisions for reliable detection. Confidence grows with length.

Entropy dependence is a structural property of the Aaronson/SynthID family, not an implementation accident. It has direct consequences for detection calibration and for adversarial evasion (Section 9).


3. Architecture

The confirmed architecture is:

Claude weights
      ↓
normal logits / candidate distribution
      ↓
key-derived pseudorandomness  ←  PRF(K, preceding token window)
      ↓
sampler (distribution-preserving selection)
      ↓
watermarked text

One clarification is terminological. Anthropic's phrase "applied at the model level" means applied in the serving stack of every supported model deployment, on every surface and cloud partner; it does not mean the watermark is encoded in the model weights. The watermark is a property of the inference pipeline, which explains both its cross-platform uniformity and its absence from models served before support was added.

Three simpler explanations are ruled out by the disclosure. It is not metadata-only provenance: metadata exists in the system, but only for files, through C2PA, with weaker survivability. It is not Unicode steganography: nothing is added to the text and no hidden characters are used. And there is no public evidence of a learned, weight-encoded watermark: the published description is a pure inference-time construction. Appendix A records how each of these was weighed before the disclosure.


4. A Working Formal Model

Given the disclosed family, a reasonable working model (details unconfirmed) is as follows. At step t, the sampler draws its randomness from a pseudorandom function of the secret key and a sliding window of preceding tokens:

r_t = PRF(K, x_{t−k..t−1})

and uses r_t to select among candidate tokens in a way that preserves, exactly or approximately, the model's conditional distribution P(x_t | x_{<t}). In SynthID-Text specifically, this takes the form of tournament sampling, in which candidate tokens are compared in a bracket using key-derived scoring functions, preserving distributional quality while concentrating detectable correlation.

Detection computes an aggregate score over the observed sequence:

S(x, K) = Σ_t g(x_t, x_{t−k..t−1}, K)

and compares it against the null distribution of texts not generated under key K:

p = P(S ≥ S_observed | H₀)

Three structural consequences follow:

  1. Context-keyed pseudorandomness. There is no fixed "green list"; the preferred-token structure changes with every context window.
  2. Per-token signal is proportional to local entropy. Where P(x_t | x_{<t}) is nearly deterministic, g contributes approximately nothing. Aggregate detectability is therefore a function of the entropy profile of the generation, not merely its length.
  3. Distribution preservation. Single-sample distributional tests cannot distinguish watermarked from unwatermarked output; only key-informed correlation tests can. This is the design intent: undetectable without the key, detectable with it.

The third consequence shapes the entire experimental programme in Sections 11 and 12. If the marginal distribution of a single output is unchanged, then any key-free evidence of the watermark must come from correlations between outputs generated under the same key, not from the statistics of any one output.


5. Deployment Timing

The algorithm is externally validated prior art (Aaronson 2022 → SynthID-Text 2024, including production-scale validation on Gemini traffic), not a novel in-house construction. The trigger is explicitly regulatory: the EU AI Act's Article 50 transparency obligations took effect for the relevant Code of Practice signatories, with an August 2, 2026 boundary for new-model launch support. The substantial internal effort is the engineering programme, integrating keyed sampling across every serving surface and cloud partner, worldwide, with per-model rollout for legacy models. It is not watermark invention.

The sequence is best modelled as:

External research (Aaronson, SynthID-Text)
              ↓
Regulatory codification (EU Code of Practice, July 2026)
              ↓
Internal adaptation and quality validation
              ↓
Serving-stack integration across surfaces and clouds
              ↓
Launch-day support for new models (Aug 2, 2026+)
              ↓
Progressive backport to earlier models
              ↓
Detection API and technical documentation (forthcoming)

One stated detail is worth noting. Anthropic applies the watermark globally because it lacks a durable mechanism for regional scoping. That is an operational admission, and it implies the signal will appear uniformly in worldwide Claude output corpora.


6. What Detection Proves

Anthropic's own framing is carefully probabilistic. It matches the distinction between a positive provenance mechanism and an AI detector.

The detector answers one question:

What is the likelihood that Claude was involved in producing this text?

It cannot answer whether the text was human-written, since absence of watermark does not imply human authorship, and it cannot answer whether a different AI wrote it, since other providers' watermarks use different keys and possibly different methods. It cannot distinguish "Claude wrote this" from "Claude heavily edited this". And it reveals nothing about the user, organization, or conversation of origin.

Formally:

No watermark detected   ⇏   human authored
Watermark detected      ⇏   Claude originated the ideas

Anthropic also contrasts watermark detection with third-party AI-detection software, such as stylometric detectors. Those services lack the key and rely on stylistic "tells" instead. The two are fundamentally different tests. The watermark check is a keyed correlation test with a controlled null distribution; stylometric detection is an unkeyed classifier, with all the attendant false-positive risks.

Anthropic further states the watermark has no bearing on output ownership or legal responsibility.


7. Robustness

Anthropic acknowledges partial removability. Light editing probably does not remove the watermark completely. A complete rewrite, replacing every word, removes it, at which point the text is arguably no longer AI-generated. Heavy editing, paraphrasing, translation by another system, or mixing into other writing can make the signal unreliable. Short passages are unreliable regardless.

Open empirical questions suitable for external study:

  1. Degradation curves. How does detection confidence decay as a function of edit distance, fraction of tokens replaced, or paraphrase strength? Does the decay differ between high- and low-entropy regions?
  2. Entropy-profile calibration. For a fixed length, how much does detectability vary between creative prose, technical explanation, and code with comments? The disclosed entropy-gating implies large domain variance in the minimum reliable passage length.
  3. Excerpting and interleaving. How does detection behave on documents that interleave watermarked Claude text with human text? This is the dominant real-world pattern.
  4. Cross-lingual persistence. Claude-produced translations are watermarked, since every word is chosen by Claude. If a second system translates the output again, the design suggests the watermark is essentially gone, since every token is re-chosen.

Section 12.3 turns these questions into a concrete transformation battery.


8. Watermark Radioactivity

When a student model is trained on outputs from a watermarked teacher:

Teacher (keyed sampler)
        ↓
Watermarked outputs
        ↓
Distillation / synthetic-data corpus
        ↓
Student model
        ↓
Student outputs

the key-correlated token statistics of the training corpus can become a learned property of the student's distribution. This is watermark radioactivity. Published research has demonstrated such transfer through knowledge distillation, and follow-on work (TextSeal, ReasMark) treats distillation transfer as an active research area, not a theoretical concern.

Two refinements follow from the known architecture:

  • What transfers is correlation, not the key. The student can absorb the effect, elevated detector scores under key K, without possessing K or the sampling mechanism. Watermark inheritance ≠ key inheritance.
  • Entropy-gating shapes inheritance. The teacher's watermark lives in high-entropy choices, so inheritance should concentrate in exactly those regions, and paraphrase-based neutralization of training data should be most effective there. This is a testable prediction.

Targeted paraphrasing of distillation data and watermark-neutralization procedures have been shown to reduce inherited signals while retaining knowledge transfer. Radioactivity is a probabilistic leakage channel, not a permanent lineage stamp.


9. Watermark Spoofing

The more serious security problem is imitation, not removal.

Research including DITTO demonstrates that distillation on a watermarked corpus can potentially yield a model whose outputs trigger the teacher's detector. The result is an authentic-looking watermark on text the teacher never produced:

Claude (key K)
        ↓
watermarked corpus
        ↓
distillation
        ↓
attacker model
        ↓
outputs scoring positive under K

Hence:

Detected Claude watermark   ⇏   Claude generated the text

unless the detector statistic depends on secret structure that an independently trained model cannot practically reproduce from samples alone.

The disclosed architecture clarifies the stakes. A keyed PRF over short context windows means the signal is, in principle, a learnable function of (context window → token preference) pairs, and a large watermarked corpus contains abundant examples of exactly those pairs. The practical question is sample complexity. How much watermarked text does a student need before its outputs achieve detector scores that meaningfully raise false-attribution risk at the operating threshold? That is the gap between a statistical classifier, which a sufficiently capable imitator can fool, and a genuine provenance mechanism, whose positive detections retain evidentiary weight under adversarial conditions.

Anthropic's forthcoming detection API will make this question testable by third parties. It also creates an oracle-access consideration. Rate limits, score granularity, and thresholding choices in the API will determine how efficiently an attacker can use it as a training signal for spoofing or as feedback for removal.


10. Three Research Problems

Problem 1: Detection and calibration

Given the eventual detection API: what are the true-positive rates at very low false-positive rates, and how do they vary with length, domain, and entropy profile? What is the minimum reliable passage length per domain?

Problem 2: Inheritance

Does the keyed-sampling signal propagate through supervised fine-tuning, distillation, synthetic-data training, and repeated model-to-model rewriting? At what rate does it decay across generations? Does it decay at all, or does it stabilize as a learned behavioural attractor?

Rapid decay:      1.0 → 0.72 → 0.41 → 0.08
Attractor regime: 1.0 → 0.83 → 0.79 → 0.77

Problem 3: Spoofing

What is the sample complexity and compute cost of training a model whose outputs falsely score positive under Claude's key at operationally relevant thresholds? Can detector-side countermeasures, such as statistics that separate genuine keyed sampling from learned imitation, restore evidentiary value?

Problem 3 is the most consequential. It determines whether the watermark is merely a transparency signal or can bear weight in attribution disputes.

The next two sections give the experimental design for all three problems: first the corpus, then the tests that run on it.


11. Building the Corpus

Every measurement in this programme is only as good as the corpus behind it. The corpus has four jobs: represent the domains where entropy-gating predicts different behaviour, provide clean nulls, support paired comparisons, and remain uncontaminated.

11.1 Cells and stratification

Stratify generation along the axes the theory says matter:

  • Domain. Creative prose, technical explanation, factual question answering, code with comments, translation, and summarization. The entropy-gating hypothesis predicts strong watermark signal in the first and fifth, weak signal in factual answering and code bodies.
  • Length. Bands from very short to long, for example 50, 100, 250, 500, 1,000, 2,500, and 5,000 tokens, so minimum-reliable-length curves can be estimated per domain rather than assumed.
  • Sampling parameters. Temperature and top-p sweeps. If the watermark operates on the sampled choice, then lower temperature narrows the candidate set and should thin the signal, which is itself a testable prediction.
  • Model and endpoint. Each supported Claude model, on each surface available to the researcher (first-party API and cloud partners), recorded separately. Cross-endpoint uniformity is a claim worth checking, not assuming.

A full factorial design is unnecessary. A fractional design that covers every domain × length pair at a default temperature, plus a temperature sweep on two contrasting domains, captures most of the information at a fraction of the cost.

11.2 Controls

Three null corpora, kept strictly separate:

  1. Legacy Claude. Pre-watermark Claude models prompted identically. This is the most valuable control in the whole design: unwatermarked text with matched style, vocabulary, and topic, which isolates "key-correlated statistics" from "Claude-like statistics".
  2. Other LLMs. Outputs from other providers, some of which now carry their own watermarks under different keys. These measure cross-model false positives.
  3. Human text. Drawn only from sources that predate the watermark deployment, for the contamination reason below.

11.3 Generation protocol

For each prompt cell, generate repeated samples, for example N = 20 generations per prompt, under identical parameters. Repeated sampling is not a luxury. Under a fixed key, two independent generations that pass through the same context window face the same key-derived randomness, and that cross-sample correlation is the only trace a distribution-preserving watermark leaves without the key (Section 12.1).

Record for every sample: model identifier and version string, endpoint, date and time, full prompt, temperature, top-p, maximum tokens, and the raw token IDs where the API exposes them. Token IDs matter because detection statistics operate on tokens, and re-tokenizing decoded text can shift boundaries.

11.4 Hygiene

Three contamination risks:

  • Human corpora that postdate deployment may already contain Claude-generated or Claude-edited text. Restrict human nulls to material published before the watermark rollout, or to sources with strong provenance.
  • Deduplication. Repeated boilerplate (headers, standard disclaimers, common code idioms) creates spurious cross-sample agreement that mimics watermark correlation. Deduplicate at the n-gram level before running correlation tests.
  • Prompt leakage. If prompts contain long quoted passages, the model may copy them, and copied spans carry no watermark. Mark quoted spans and exclude them from scoring.

11.5 Scale

Power analysis should drive size, not round numbers. The per-token signal is small and entropy-dependent, so required corpus size differs by domain. A practical approach: pilot with a few hundred generations per cell, estimate the per-token effect size from the pilot's cross-sample correlation statistics, then compute the samples needed for the target power at the target false-positive rate. Expect the code and factual cells to demand far more text than the prose cells for the same confidence, and treat that asymmetry as a result in its own right.


12. Testing and Detection Methods

12.1 Key-free tests

A distribution-preserving watermark leaves each single output statistically normal. Consequently, no test on one sample in isolation should work, and confirming that is the first experiment: standard stylometric and distributional statistics (token frequency, rank, entropy, n-gram profiles) should fail to separate watermarked Claude from legacy Claude. If they do separate, either the watermark is not distortion-free in practice or the corpora are confounded, and both findings matter.

The signal that can survive key-free analysis is correlation between outputs generated under the same key:

  1. Matched-context agreement. Across repeated generations from the same prompt, find positions where two samples share the same preceding k-gram. Under an arbitrary random sampler, their next-token choices agree at a rate predicted by the model's conditional distribution. Under a keyed sampler, agreement should exceed that baseline, because both samples drew their randomness from the same PRF output. Estimate the baseline from the legacy-Claude control at the same contexts.
  2. Window-size sweep. Run the agreement test for candidate context windows k = 1 through 5. Excess agreement should peak at the true window size and wash out beyond it, which estimates k without the key.
  3. Entropy conditioning. Score positions by an entropy estimate from a reference model. The keyed-sampling hypothesis predicts that excess agreement concentrates in high-entropy positions and vanishes in low-entropy ones. A signal that is flat across entropy bands points to a different mechanism or a confound.
  4. Statistical machinery. Use permutation tests, shuffling sample pairings within a cell, to build the null distribution empirically rather than analytically. Correct for multiple comparisons across cells and window sizes. Report effect sizes with confidence intervals, not bare p-values.

These tests characterize the watermark. They do not read it. Without the key, the researcher can estimate where the signal lives and roughly how strong it is, but cannot decide whether a given third-party document is watermarked.

12.2 Detector-based calibration

Once Anthropic's detection API is available, the programme shifts from characterization to calibration:

  1. Operating curves. Score every corpus cell. Plot true-positive rate against false-positive rate, with particular attention to the low-FPR region (10⁻³ and below), because attribution use cases live there.
  2. Minimum-length curves. For each domain, find the shortest passage length at which detection clears a chosen operating point. Entropy-gating predicts these curves differ sharply between prose and code.
  3. Null behaviour. Score the legacy-Claude, other-LLM, and human corpora. Legacy Claude is the critical case: if pre-watermark Claude scores above the human null, the detector is partly a Claude-style classifier, which weakens its evidentiary value (Section 9).
  4. Oracle discipline. Log every API call. Treat the detector as an adversary would: measure how much information per query the score granularity leaks, since that determines how efficiently removal or spoofing could be tuned against it.

12.3 Transformation battery

Apply graded transformations to watermarked samples and re-score:

  • Editing. Replace 5%, 10%, 25%, and 50% of tokens with human edits, tracked separately for high- and low-entropy positions.
  • Paraphrase. Weak (sentence-level rewording) through strong (full semantic rewrite by another model), in labelled strength bands.
  • Translation. Round-trip through one and two other languages using a non-Claude system.
  • Interleaving. Mix watermarked and human text at ratios from 10% to 90% Claude content, in contiguous blocks and in alternating sentences, since real documents do both.
  • Normalization. Unicode normalization, whitespace collapse, and format conversion, as a cheap-attack baseline; a sampling watermark should be untouched, and confirming that closes off the steganography alternative empirically.

The deliverable is a degradation surface: detector score as a function of transformation type and strength, per domain.

12.4 Distillation protocol

Inheritance and spoofing (Problems 2 and 3) need controlled distillation:

  1. Teacher corpora. Watermarked Claude outputs at increasing sizes, for example 10⁴, 10⁵, and 10⁶ examples, matched across domains.
  2. Students. Small open models fine-tuned on each corpus size, with an identical student trained on the legacy-Claude corpus as the control.
  3. Inheritance measurement. Score student outputs with the detector (or with the key-free agreement statistic before the API exists). Plot inherited signal against teacher-corpus size and against student scale.
  4. Generational decay. Distil a second student from the first, and a third from the second, to distinguish the rapid-decay regime from the attractor regime in Section 10.
  5. Neutralization. Paraphrase the teacher corpus before training and measure how much inheritance survives, and at what cost to task performance.
  6. Spoofing measurement. Report the detector score distribution of the best student against the operating thresholds from Section 12.2, which converts "can it be spoofed" into a measured false-attribution rate at a stated cost.

Publication of this work should follow the norms of security research: aggregate results and thresholds, not turnkey recipes optimized for evasion.


13. The Laboratory

The modules above assemble into one programme:

                    WATERMARK LAB (v2)
                          │
     ┌───────────┬────────┼─────────────┬─────────────┐
     ↓           ↓        ↓             ↓             ↓
 Calibration  Robustness  Entropy     Transfer     Spoofing
     │           │        mapping        │             │
     ↓           ↓          ↓            ↓             ↓
 TPR @ low   edit/para-  domain ×    distillation   imitation
 FPR curves  phrase/mix  length      inheritance    training
     └───────────┴──────────┴────────────┴─────────────┘
                          ↓
              Attribution confidence model

Core modules:

corpus/
    claude_watermarked/        # post-Aug-2026 models
    claude_legacy/             # pre-watermark models (natural control)
    human/
    other_watermarked_llms/    # different keys/methods
    distilled_students/

collection/
    domain_stratified_prompts  # prose / factual / code / translation
    entropy_profiling
    temperature_and_top_p_sweeps
    repeated_sampling

analysis/
    detector_api_scoring       # once available
    entropy_conditioned_stats
    ngram_and_window_statistics
    cross_sample_correlation

transformations/
    graded_editing
    paraphrase (weak → strong)
    round_trip_translation
    human_text_interleaving

distillation/
    teacher_corpus_construction
    student_training
    inheritance_measurement
    neutralization (paraphrased training data)
    spoofing_attempts + detector-side defenses

reports/

The end product is an attribution confidence model: given a passage's length, domain, entropy profile, and detector score, what posterior confidence in Claude involvement is justified, and under what adversarial assumptions.


14. Evaluation Metrics

Metric Research question
TPR at very low FPR Can the watermark support high-confidence attribution?
Minimum token count, per domain How much text is required, and how does entropy-gating shift this for code vs. prose?
Entropy-conditioned detectability Does the signal concentrate where theory predicts?
Editing robustness Does ordinary editing destroy the signal, and at what edit fraction?
Paraphrase robustness Does semantic rewriting destroy it?
Cross-system translation robustness Does re-translation by another system erase it (as architecture predicts)?
Interleaving behavior How does detection score mixed human/Claude documents?
Cross-model false positives Do other LLMs (watermarked with other keys, or unwatermarked) trigger detection?
Legacy-model false positives Do pre-watermark Claude outputs score at the null?
Watermark inheritance Does the signal survive distillation, and at what rate per generation?
Watermark spoofability Sample/compute cost for an imitator to exceed the detection threshold?
Detector-oracle leakage How much does API access accelerate removal or spoofing?

A watermark can perform excellently on pristine samples while providing weak guarantees under adversarial conditions. The lower rows of the table are where provenance claims are actually decided.


15. Security Implications

15.1 The watermark is transparency infrastructure, not cryptographic attribution

Anthropic frames detection as a likelihood signal and explicitly disclaims conclusiveness. Downstream consumers (platforms, educators, courts, and deployers with their own Article 50 obligations) should treat positive detections as evidence of processing involvement, calibrated by passage length and domain, and never as proof of authorship.

15.2 Secret keying is necessary but not sufficient

The confirmed architecture keeps the key out of the weights and out of the text. However, the observable signal is a learnable statistical correlate, so keying raises the cost of spoofing without categorically preventing it. The distinction between learnable behaviour and a secret-key-dependent signal is the central open security question, not a design checkbox.

15.3 Entropy-gating is a feature and an attack surface

Concentrating the watermark in high-entropy choices preserves quality and exactness. It also tells an attacker exactly where to paraphrase. Low-entropy laundering, rewriting only the free-choice regions, is a predictable evasion strategy worth quantifying.

15.4 Distillation changes the threat model

A production watermark cannot be evaluated against human editing alone; it must also be evaluated against neural networks trained specifically on watermarked corpora, for removal (inheritance followed by neutralization) and for forgery (spoofing).

15.5 Robustness and authenticity are different properties

A watermark that survives paraphrasing is robust. A watermark whose positive detections cannot be counterfeited by an independent model is authenticatable. The disclosed system is designed primarily for the transparency use case. Whether it can also be authenticatable under adversarial pressure is an empirical question the forthcoming detection API will finally allow the community to test.


16. Conclusion

Claude's watermark is a decoding-time, keyed pseudorandom-sampling watermark, a SynthID-Text variant in the Aaronson family, applied uniformly across all supported model deployments worldwide. It is not encoded in the model weights. It adds nothing to the text, preserves output quality and cost, is entropy-gated, carries no user-identifying information, and yields a likelihood of Claude involvement rather than proof of authorship. The deployment is best read as regulatory-driven engineering integration of externally validated prior art, timed to the EU AI Act's Article 50 Code of Practice, with an explicit transition path for legacy models and a detection API forthcoming.

The most important security finding follows from the architecture. The watermark is a statistical property of sampled outputs, so it can propagate through distillation, and a student model can potentially imitate it. Detection is a probabilistic transparency signal. Whether it can be hardened into adversarially robust provenance is the open question, and it defines the empirical programme of Sections 11 and 12:

Can a detector identify a secret, model-specific signal that remains difficult for an independently trained model to reproduce?

The strongest next step is a controlled measurement programme, using pre-watermark Claude models as the natural control, that quantifies entropy-conditioned detectability, robustness degradation curves, distillation inheritance rates, and the practical cost of spoofing. The goal is to establish what evidentiary weight a detected Claude watermark can actually bear.

References


Appendix A: Revision History

This paper reached its present form in two revisions. Revision 1 predated Anthropic's disclosure and analysed the watermark as a black box. Revision 2 (August 2026) incorporated the disclosure. This appendix preserves what changed, for readers comparing the two.

A.1 The hypotheses Revision 1 weighed

Before the disclosure, the observable evidence (a mark embedded in text, surviving copy-and-paste, present across all serving surfaces) admitted three architectures:

  1. A decoding-time statistical watermark, keyed at inference.
  2. A learned, model-native watermark, trained into the weights through an auxiliary objective.
  3. A hybrid, combining watermark-aware training with a keyed inference-time transformation.

Revision 1 favoured the hybrid, largely because "applied at the model level" seemed to imply behaviour carried in the model itself. The disclosure settled the question in favour of the first architecture. "Model level" turned out to mean deployment scope, every surface and cloud partner, rather than weight-level encoding, and no public evidence supports a trained component. Nothing published rules out watermark-aware evaluation during model development, but parsimony favours the plain decoding-time reading.

Two secondary hypotheses were also retired. Metadata-only provenance was ruled out for text (it exists only for files, through C2PA). Unicode steganography was ruled out directly by the statement that nothing is added to the text and no hidden characters are used; Anthropic's separate, previously observed use of Unicode variation in Claude Code prompt engineering is an unrelated internal signalling technique, not a provenance mechanism.

Revision 1 also argued from deployment timing that the watermark reflected a long-running internal research programme. The disclosure supports a more modest reading: the research is external prior art, the trigger is regulatory, and the internal programme is serving-stack integration (Section 5).

A.2 Change log

Item Revision 1 status Revision 2 status
Watermark algorithm Undisclosed; three hypotheses Disclosed: SynthID-Text variant (Aaronson family)
Preferred hypothesis Hybrid (trained + keyed inference) Retired; pure decoding-time keyed sampling confirmed
"Model level" interpretation Ambiguous (possibly weights) Clarified: deployment scope, not weight encoding
Unicode steganography Secondary hypothesis Ruled out ("no hidden characters")
Learned watermark objective Open possibility Unsupported by any public evidence
Quality impact Unknown None reported (internal testing + SynthID-Text Gemini evidence)
Cost/latency Unknown No extra tokens; negligible speed impact
User traceability Unknown None; no per-user/org/chat information
Entropy dependence Not analyzed Central property: sparse on facts, code, proofreading; full on translation
Regulatory context Inferred Explicit: EU AI Act Art. 50(2) Code of Practice; Aug 2, 2026 boundary; global application; legacy transition
Detection access Unknown Detection API announced (details pending)
Radioactivity/spoofing Core concern Unchanged; sharpened by known architecture and API-oracle considerations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment