mirror-eval · 63 probes · two waves
Claude cited my site on 0 of 63 probes. Twice.
The note
A sheet rises, four engine names print, and the August column rolls up: 13, 62, 61. The Claude row stops at 0.
Claude cited my site or my repos on 0 of 63 probes in August, and on 0 of 63 again in September.
The run asks four AI search engines the same 63 questions about my work and logs every source each answer cites. The question I cared about was narrow: does the answer cite my site or my repos, or something else?
Between the waves I cleaned up the web surfaces the engines were reading. For one engine that check passed loudly. Perplexity went from 13 of 63 to 63 of 63. ChatGPT went from 62 to 63. Gemini from 61 to 63.
Claude stayed at zero. Whatever the cleanup changed, none of it reached Claude's answers. What Claude cited instead is a separate count, in citations rather than probes.
Limits. One subject: me. Two waves, 29 days apart. Engine versions were not verifiably frozen between waves. The six cleanup changes went out together, so I cannot say which one moved Perplexity. "Claude" here is the API with web search on, standing in for the product, under the alias claude-sonnet-5 rather than a pinned version. The raw per-citation file is not in the public repo.
When three engines move and one sits at 0 of 63 twice, is the zero a fact about the engine or a gap in your own pages?
Where the numbers live
- README.md
- Site page /writing/eval-harness-at-my-own-reflection/
- results/2026-09-04/manifest.json
- results/2026-08-06/scores.jsonl
- src/engines.py
The harness, the two waves and the cleanup, with the counts
One subject, two waves a month apart. Probe counts, not judge verdicts. Engine versions not verifiably frozen.