Topliner — AI prompt for candidate relevance evaluation

candidate-relevance-prompt.md

Candidate Relevance — AI Prompt

The prompt used by AI to decide whether a candidate is relevant to a role. Built in app/Prompts/BatchCandidateEvaluationPrompt.php. Candidates are evaluated in batches of 25 per request.

Prompt texts are shown verbatim, exactly as sent to the model. {...} marks interpolated values.

---

1. Prompt skeleton

You are evaluating multiple job candidates for relevance to a specific role.
Evaluate ALL candidates and return a result for EACH one.

## EVALUATION CRITERIA:
{CRITERIA_BLOCK}

## CANDIDATES TO EVALUATE (each has _id and _index fields for identification):
{CANDIDATES_JSON}

## YOUR TASK:
For EACH candidate in the list above:
1. {FIELDS_HINT}
2. Evaluate if they match the criteria
3. Return your decision with brief reasoning

IMPORTANT:
- You MUST return exactly {COUNT} results, one for each candidate
- Use the candidate's "_id" field to identify them in your response
- When in doubt, mark as relevant (true) - we don't want to lose good candidates
- Keep reasoning brief (1-2 sentences max)

Return a JSON array with exactly {COUNT} objects.

---

2. Criteria block {CRITERIA_BLOCK}

Assembled from 2-3 chunks depending on what is configured on the project.

2a. Target Role Matching — included if the project has job_title

### Target Role Matching:
Looking for candidates who have held roles matching or equivalent to: "{JOB_TITLE}" at "{COMPANY_NAME}"
Consider:
- Variations in title naming
- Equivalent scope and seniority
- Similar responsibilities
- If the candidate's title at {COMPANY_NAME} is vague or generic (e.g., just "Operations" or "Finance"), look at their FULL career history. If their previous roles show the right seniority level (e.g., they were VP Operations, Director of Operations, Head of Operations at other companies), they likely hold a similar level at {COMPANY_NAME} but simply filled in their LinkedIn profile lazily. Mark them as relevant.
A candidate is relevant if their role is a direct or reasonable equivalent.

2b. Exclusion Criteria — included if the project has exclusion_criteria

### Exclusion Criteria:
Candidates should be EXCLUDED if they match ANY of these:
{EXCLUSION_CRITERIA}

2c. Decision Rule — depends on which chunks above were included

• both sections present: ### Decision Rule: relevant = true ONLY if target role matches AND no exclusion criteria met
• target role only: ### Decision Rule: relevant = true if target role matches
• exclusions only: ### Decision Rule: relevant = true if NO exclusion criteria met

---

3. Fields hint {FIELDS_HINT}

Depends on the profile source — each has a different data shape:

Source	{FIELDS_HINT}
LinkedIn Recruiter search	Look at their currentPositions, workExperience, headline
CrustData DB Search	Look at their current_employers, past_employers, headline, skills
MixRank person2 ES	Look at their experience (job history), headline, skills

Before being embedded in the prompt, profiles are trimmed to the relevant fields (to save tokens) and each one is tagged with _id + _index so the model can reference them in its reply.

---

4. Response schema (Structured Output)

The model must return a JSON array with exactly {COUNT} objects:

[
  {
    "_id": "https://www.linkedin.com/in/...",
    "relevant": true,
    "reasoning": "VP of Operations at target company with 10+ years of relevant experience."
  }
]

Field	Type	Description
_id	string	Profile URL from the input — used to match the answer back to the candidate
relevant	boolean	Passes the criteria or not
reasoning	string	1-2 sentences explaining why

minItems / maxItems are both pinned to {COUNT} — the model cannot return fewer or more items than were sent in.

---

5. Fully assembled example

Project: job_title = "Head of Sales", exclusion_criteria = "Exclude candidates from retail and FMCG". Company: Acme Inc. Batch of 25 candidates from CrustData.

You are evaluating multiple job candidates for relevance to a specific role.
Evaluate ALL candidates and return a result for EACH one.

## EVALUATION CRITERIA:
### Target Role Matching:
Looking for candidates who have held roles matching or equivalent to: "Head of Sales" at "Acme Inc"
Consider:
- Variations in title naming
- Equivalent scope and seniority
- Similar responsibilities
- If the candidate's title at Acme Inc is vague or generic (e.g., just "Operations" or "Finance"), look at their FULL career history. If their previous roles show the right seniority level (e.g., they were VP Operations, Director of Operations, Head of Operations at other companies), they likely hold a similar level at Acme Inc but simply filled in their LinkedIn profile lazily. Mark them as relevant.
A candidate is relevant if their role is a direct or reasonable equivalent.

### Exclusion Criteria:
Candidates should be EXCLUDED if they match ANY of these:
Exclude candidates from retail and FMCG

### Decision Rule: relevant = true ONLY if target role matches AND no exclusion criteria met

## CANDIDATES TO EVALUATE (each has _id and _index fields for identification):
[
  {
    "name": "John Doe",
    "headline": "Head of Revenue at Acme Inc",
    "current_employers": [ { "name": "Acme Inc", "title": "Head of Revenue", "seniority_level": "Director", "start_date": "2022-01", "end_date": null } ],
    "past_employers": [ ... ],
    "skills": [ "B2B Sales", "SaaS" ],
    "_index": 0,
    "_id": "https://www.linkedin.com/in/johndoe/"
  },
  ... 24 more candidates ...
]

## YOUR TASK:
For EACH candidate in the list above:
1. Look at their current_employers, past_employers, headline, skills
2. Evaluate if they match the criteria
3. Return your decision with brief reasoning

IMPORTANT:
- You MUST return exactly 25 results, one for each candidate
- Use the candidate's "_id" field to identify them in your response
- When in doubt, mark as relevant (true) - we don't want to lose good candidates
- Keep reasoning brief (1-2 sentences max)

Return a JSON array with exactly 25 objects.

---

6. Key design decisions

• Batch size = 25. Validated across 3 projects with 3 runs each — batches of 50 produced false negatives on borderline titles. See docs/research/ai-evaluation-model-stability-batch-size-experiment-2026-04-13.md.
• "When in doubt — relevant=true" is a deliberate bias: losing a good candidate is more expensive than passing an extra one to the next scoring stage.
• The vague-titles hint exists because LinkedIn users frequently write "Operations" instead of "VP of Operations". Without the hint, the model filters them out on literal string match.
• _id via publicProfileUrl / flagship_profile_url guarantees the model's answer can be matched back to the input candidate, even if the order is reshuffled.