Do AI responses naturally contain hidden characters?
We scanned 10,000 real model responses instead of inserting synthetic markers. Fifteen responses contained a supported hidden character; most findings were normal controls used in emoji—not evidence of an AI watermark.
A hidden-character scanner is useful for literal Unicode traces, but it is not a SynthID detector and should not be marketed as universal AI detection. A zero result means that no supported literal characters were found.
What we tested
The test used real generated responses from two public sources: Google DeepMind's SynthID Text evaluation corpus and the UltraFeedback preference dataset. The Google corpus supplied 3,000 watermarked generations paired with 3,000 clean controls. Additional sampled responses brought the total scan to 10,000.
We ran every response through the same deterministic Unicode rules used by the live text checker. The scanner records exact code points and counts; it does not call an AI classifier or infer authorship from writing style.
What we found
- Only 15 of 10,000 responses contained any supported hidden character.
- Most findings were language- or emoji-sensitive controls that the cleaner marks for review rather than deleting automatically.
- All 3,000 Google watermarked samples and all 3,000 paired clean samples returned zero supported hidden characters.
- The zero/zero result is expected: SynthID Text is encoded in token probabilities, not literal invisible Unicode.
What this test does—and does not—show
It shows that supported invisible characters are uncommon in these corpora and do not reveal Google's statistical watermark. It does not show that invisible characters never appear in other model outputs, tools, languages, or copy-paste pipelines. It also does not measure SynthID detection accuracy because we did not use Google's watermark detector.
Reproducibility notes
The corpus scan is a repeatable repository script, not a hand-counted demo. It reads the named dataset files, hashes the inputs, scans each response with the production text rules, groups results by corpus and label, and emits a JSON report. The homepage reports only the high-level result; this page records the interpretation and limits.
Download the aggregate JSON result. It contains corpus counts, the published Google input hash, group results, exact code-point totals, and interpretation boundaries. It deliberately excludes model-response text.
Why synthetic markers are excluded
Planting known characters is useful for unit tests, but it cannot answer whether those characters occur naturally in model output. The 10,000-response test therefore measures unmodified corpus text. Separate automated tests verify that each supported code point is detected and cleaned according to policy.