AI screening goes wrong in five ways. None of them look like errors. Learn to spot them now, before you meet them in a real search.
1. It makes things up
Ask an AI to summarise a CV and it may add a responsibility the person never mentioned — usually something typical for that job title. It sounds right. It is often the best-written line in the summary.
The model is not looking something up and getting it wrong. It is writing text that fits the pattern, and an invented responsibility fits the pattern just as well as a real one.
How to protect yourself: ask for quotes. If a tool says "strong background in data pipelines", ask where. A tool that shows you the exact line from the CV can be checked in two seconds. One that only summarises cannot be checked at all.
2. It finds a back door to bias
Software does not need to know someone's age, race or gender to treat them differently. It only needs something that tends to go along with those things.
Postcode. University. How long a gap in their CV lasted. Which sports they list. How the English reads if it is their second language. Even which CV template they used.
The risk is highest with tools that learn from your past hires. "Find people like our best employees" sounds sensible. What it means is "repeat what we did before" — including the parts you would never write down as a rule.
How to protect yourself: write your own rules instead of letting a tool learn them from your history. Module 11 shows you how to check the results.
3. It filters out good people, and you never find out
This is the most important one on the list.
When screening rejects a good candidate, nothing happens. No error. No complaint. The job gets filled by someone decent from the people who survived, and everyone assumes it worked.
Compare the two kinds of mistake:
- Letting a weak candidate through costs you thirty minutes in an interview. You notice. You tighten the rules.
- Rejecting a strong candidate costs you a good hire. You never notice.
So your instincts push you to filter harder, in exactly the direction you cannot see. And every time you tighten the rules, it feels like an improvement.
How to protect yourself: look at the rejections on purpose. After every run, open the three highest-scoring people who were rejected, and the three lowest-scoring people who got through. Decide whether you agree. If you always agree, your rules are probably too blunt to be doing anything.
4. Things drift without telling you
Three things move underneath you. The job market changes. The people applying to you change. And the tool changes when the vendor updates it, usually with no announcement you would notice.
How to protect yourself: keep twenty CVs whose correct handling you have agreed with a colleague. Run them again every few months. If the results move and your rules did not, something else did.
5. You stop checking
When something is right most of the time, people stop looking. This is well documented in aviation and in medicine, and it works the same way here.
The danger is worst when accuracy is high but not perfect — which is exactly where these tools sit. Something that was wrong often would keep you alert.
How to protect yourself: design checks that need a real decision, not a click. "Which two of these ten would you drop?" makes someone read. "Approve this shortlist" does not.
All five, on one page
| Problem | Sign it is happening | What to do |
|---|---|---|
| Making things up | Claims you cannot find in the CV | Ask for quotes |
| Back door to bias | Shortlists look like your current team | Write your own rules; check results |
| Over-filtering | You never disagree with the system | Read the rejections every time |
| Drift | Results change but your rules did not | Keep twenty test CVs; re-run them |
| You stop checking | Reviews take less time each week | Ask a question, not for approval |
The rule this course follows
Let software do the reading. Never let it do the rejecting. It can sort the pile and explain itself. A person decides who leaves the pile.