Screen all 500 CVs, then find out where you and a human reviewer disagreed. The disagreements are the point, not the shortlist.
What to do
- Screen all 500 CVs using rules you wrote from the job brief in Module 2.
- Pick exactly 20.
- Open
/human-shortlist.csvin the practice folder. It holds 20 candidates chosen by a person who read everything by hand, with their notes. - Compare the two lists.
What to write down
| Group | Question to answer |
|---|---|
| On both lists | How many? This is your starting point |
| Only on yours | Did the human miss them through tiredness, or do they know something your rules do not? |
| Only on theirs | Where did they land in your run, and which rule cost them? |
| On neither | Check five. Do you agree they belong out? |
What your overlap means
| Overlap | What it means | What to do |
|---|---|---|
| 10–14 of 20 | Healthy. Two processes agreeing on the main thing, differing at the edges | Study the edges |
| 18–20 | Your rules copied one person's taste, blind spots included | Check whether both lists look like your existing team |
| Under 6 | One of the two processes is broken | Check the reading quality on their picks first |
That last instruction matters. In practice, a lot of "the software missed them" cases turn out to be reading failures — the tool never saw the evidence. Check the extracted data before you conclude anything about your rules.
What to hand in
One page: your rules as written, your 20, how many overlapped, and three sentences on the one rule you would change and why. Keep it — the final project in Module 12 builds on it.
Check yourself before Module 5
- Which field does your tool handle worst on these CVs?
- Can you explain, for one specific person, why they ranked where they did — pointing at the line in their CV?
- Did you find at least one person the tool ranked low who deserved a read?
- Would your rules have found the human's picks if the reading had been perfect?
If the answer to the third one is no, run the full-text search from Lesson 6 before moving on. In a pile of 500 there is almost always at least one.
Doing this without a screening tool
You do not need a tool for this one. The manual version teaches you something the automatic version cannot: how expensive attention really is at this volume.
- Get the text out of the CVs. On Mac or Linux, run
pdftotext -layouton each file; on Windows, use any bulk PDF-to-text tool. That is the whole reading layer, and doing it yourself makes the two-column failures obvious straight away. - Put one row per candidate in a spreadsheet: reference, filename, extracted text.
- Add one column per rule. Use
SEARCHorCOUNTIFagainst the text. That is exactly what keyword matching is, and seeing it written as a formula takes most of the mystery out of the category. - Add up the scores. Sort. Take the top 20.
- Then do the parts no tool does for you: read the rejections, check the formats, compare against the human list.
Allow two hours rather than forty minutes. The rules sheet already has the columns set up.
What this exercise is really testing
Not whether you can operate a screening tool — that takes ten minutes to learn. Whether you can look at its output and tell the difference between a weak candidate and a candidate the tool failed to see.