screener-eval · one resume · two models
One resume, one posting: 28 and reject, 79 and advance.
The note
Two LLM screeners read my unedited resume against 21 US AI PM postings and disagreed by 22.3 points out of 100 on average. The widest gap was 51.3.
That one was Product Manager for Sensitive Deployments at OpenAI. claude-haiku-4-5 scored the resume 28 and said reject on all six calls. gpt-5-mini averaged 79 and said advance on four of the six.
Same prompt, interleaved order, five complete reps and a partial sixth: 232 calls on the unedited resume, 116 per model. The gap is the mean of 21 per-posting absolute differences. gpt-5-mini scored higher on 19 of the 21.
I had built the run to test two edits to the resume, and that test came back with a clear answer: both edits moved the score by about a point or less. Which model read the resume moved it by 22. I had not counted the model as a variable at all.
Limits. One resume, so this says nothing about resumes in general. Two cheap models under a research prompt, not any vendor's system. Five complete reps per cell, so effects under about three points are not excluded. And the gap is between two models, so it says nothing about which one is closer to a human screener.
If swapping the reader moves the score 22 points and editing the resume moves it one, which of the two is your screening eval actually testing?
Where the numbers live
The run, both screeners, all 21 postings
One resume, a research prompt, cheap-tier models, not a vendor's system. Effects under 3 points not excluded.