AI Match Follow-Through Dashboard — Requirements & Data Specification
This specification defines phase 1 of a dashboard that measures whether recruiters follow up on the candidates our AI tools surface, using one record per job order × candidate pair across a manual and an automated lane.
1. Scope
The dashboard answers one question: of the candidates our AI tools and recruiters qualify, which ones are we actually following up, how fast, and if not, why not and where?
Users: recruiters (their call list), recruiting managers (coverage and speed by recruiter and job order), and leadership (whether the AI tools translate into recruiter action).
In scope for phase 1:
- Job orders, and their matching to candidates in two lanes: Manual (recruiter) and Automated (Magic Matching plus digital-worker screening).
- Screening, and the first recruiter follow-up after screening.
- Coverage and speed of follow-up, a rule-based diagnostician, and a call list of untouched candidates.
Out of scope for phase 1: internal submittal to sales, client submittal, client interview, offer and placement. These are the gates for phase 2.
2. Process flow
Every job order branches into a Manual lane, an Automated lane, or both, and each lane runs straight to a recruiter follow-up.
- Matched by recruiterInternal Submission
- Screened by recruiter
- Passed screening
- Followed up
- Within 24 hours
- Matched by Magic Matching
- Match score 90+
- Screen score 90+digital worker
- Followed up
- Within 24 hours
In the Automated lane the first three steps are done by AI and the last two by a recruiter. That switch is the hand-off the dashboard watches.
3. Measuring points
Every count is of unique job order × candidate pairs unless the unit says otherwise. A candidate contacted four times on one job order counts once; the same candidate on two job orders counts twice. All counts apply to the selected period by the date named in the rule.
| # | Measuring point | Definition | Unit | Counting rule |
|---|---|---|---|---|
| MP1 | Active job orders | Unfilled job orders still active (not placed, not gone) at the end of the period. The headline number in the Job Orders box | Job order | opened_at ≤ period end, and no ended_at on or before period end |
| MP1a | Active at start | Opening balance: unfilled job orders already active when the period begins | Job order | opened_at before period start, and no ended_at before period start |
| MP2 | Created job orders | Job orders created in the period | Job order | opened_at is in the period |
| MP2a | Placed | Job orders placed in the period | Job order | end_status = Placed and ended_at in the period |
| MP2b | Gone, not placed | Job orders that left the active list without a placement, by reason and by the status they had reached | Job order | end_status is Closed, Wash, Cancelled or Filled elsewhere, and ended_at in the period |
| MP2c | Job orders with matches | Active job orders that got at least one candidate match in the period | Job order | Distinct jo_id where matched_at is in the period |
| MP3 | Matched pairs | Pairs created by a recruiter (Manual) or by Magic Matching (Automated) | Pair | Distinct pair_id where matched_at is in the period, split by lane |
| MP4 | Screened pairs | Matched pairs that received a screening result | Pair | MP3 where screened_at is not blank |
| MP5 | Match 90+ (Automated only) | AI matches scored 90 or above by Magic Matching | Pair | MP3 Automated where match_score ≥ 90 |
| MP6 | High-value pairs due | Pairs that qualify for follow-up and whose 24-hour window has started closing | Pair | Rule in section 4; excludes pairs not yet due |
| MP7 | Followed-up pairs | High-value pairs a recruiter contacted at least once | Pair | MP6 where contact_attempts ≥ 1; repeats never add |
| MP8 | Followed up in time | High-value pairs first contacted within 24 hours of screening | Pair | MP7 where first_contact_at − screened_at ≤ 24 hours |
| MP9 | Untouched pairs | High-value pairs no recruiter has contacted | Pair | MP6 where contact_attempts = 0 |
| MP10 | Contact attempts | Recruiter contact activity on high-value pairs | Attempt | Sum of contact_attempts over MP6. Activity only; never used as coverage |
| MP11 | Coverage | Share of high-value pairs followed up at all | % | MP7 ÷ MP6 |
| MP12 | In-time rate | Share of high-value pairs followed up within 24 hours | % | MP8 ÷ MP6 |
| MP13 | Time to first contact | How long followed-up pairs waited | Hours | Median of (first_contact_at − screened_at) over MP7 |
| MP14 | Days waiting | How long an untouched pair has gone without contact | Days | As-of time − screened_at, for each MP9 pair |
| MP15 | Job orders with zero follow-up | Job orders whose high-value pairs were all left untouched | Job order | Distinct jo_id with MP6 ≥ 1 and MP7 = 0 |
The job order balance always ties out: MP1a + MP2 − MP2a − MP2b = MP1, for every period.
Why MP10 is kept apart from MP11: 10 job orders with 17 contacts says little. If 3 pairs were contacted 3 times and 2 were contacted 4 times, only 5 pairs were reached. With 10 high-value pairs, coverage is 50%.
4. Business rules
These rules turn the raw record into the measuring points. Items marked Open or Assumption need a decision before live data is connected.
| Rule | Definition | Status |
|---|---|---|
| High-value, Automated | match_score ≥ 90 and screening_score ≥ 90 | Confirmed |
| High-value, Manual | A manual match has no match score, so there is no high-value match. A recruiter-matched pair qualifies when the recruiter's screening result is Passed (yes/no, no score). | Confirmed (Oct 16) |
| Manual match event | A manual match exists only when a recruiter links a candidate to a job order, marked by the Bullhorn status Internal Submission | Confirmed (Oct 16) |
| Follow-up | At least one recruiter contact on the pair after screening | Confirmed |
| In time | First contact within 24 hours of screened_at | Confirmed |
| Not yet due | A high-value pair with no contact, screened less than 24 hours before the as-of time. Excluded from MP6 and all rates until the window closes | Proposed |
| As-of time | The latest timestamp in the data; live, it is the time of the refresh | Proposed |
| Period membership | A pair belongs to a period by matched_at; a new job order (MP2) by jo_opened_at | Proposed |
| Periods offered | Last 7, 14 and 30 days and the full data range, ending on the latest day in the data; plus each calendar month (a partial month is labelled) | Built |
| Coverage target | 100% of high-value pairs followed up | Proposed |
| In-time target | 100% of high-value pairs within 24 hours | Confirmed (Oct 16) |
| Job order status path | Open → Internal Submit → External Submit → Placed. From any status a job order can instead drop out sideways (Closed, Wash, Cancelled, Filled elsewhere) and is gone from the active list | Confirmed (Oct 16); exit status names to confirm |
| Opening balance | The data assumes job orders were already active before the first day of match data; the Job Orders box starts from that balance | Confirmed (Oct 16) |
| Who did the work | Each step is marked human (recruiter) or machine (AI). Follow-up is always human | Confirmed |
5. Data requirements
One record per job order × candidate pair, with the same 16 columns for both lanes. The grain is the pair, not the contact: contact activity is rolled up onto the pair as a count plus first and last times.
| Field | Type | Required | Definition | Likely source | Example |
|---|---|---|---|---|---|
pair_id | Text | Yes | Unique key for the pair: jo_id + / + candidate_id | Derived | JO-1047/C-01182 |
jo_id | Text | Yes | Job order identifier | Bullhorn job order | JO-1047 |
jo_opened_at | Date-time | Yes | When the job order was opened | Bullhorn job order | 2026-08-12T09:40 |
recruiter | Text | Yes | Recruiter who owns the job order | Bullhorn job order owner | Recruiter D |
candidate_id | Text | Yes | Candidate identifier | Bullhorn candidate | C-01182 |
lane | Manual | Automated | Yes | Which path created the match | Derived from matched_by | Automated |
matched_by | Recruiter | Magic Matching (AI) | Yes | Human or machine that made the match | Bullhorn status change (Manual); Magic Matching output (Automated) | Magic Matching (AI) |
matched_at | Date-time | Yes | When the match was made | Same as matched_by | 2026-08-12T14:05 |
match_score | 0–100 | Automated only | Magic Matching score; blank for Manual | Magic Matching | 94 |
screened_by | Recruiter | Digital worker (AI) | blank | Yes if screened | Human or machine that screened | Recruiter activity; digital-worker output | Digital worker (AI) |
screened_at | Date-time | Yes if screened | When the screening result was recorded | Same as screened_by | 2026-08-12T17:20 |
screening_score | 0–100 | Yes if screened | Screening result | Same as screened_by | 92 |
screening_result | Passed | Not passed | blank | Manual, if screened | Recruiter screening outcome; Manual pairs have no screening score | Recruiter activity | Passed |
contact_attempts | Integer ≥ 0 | Yes | Recruiter contacts on this pair after screening (calls, emails, texts) | Bullhorn activity / notes | 3 |
first_contact_at | Date-time | If attempts ≥ 1 | First recruiter contact after screening | Bullhorn activity | 2026-08-13T10:15 |
last_contact_at | Date-time | If attempts ≥ 1 | Most recent recruiter contact | Bullhorn activity | 2026-08-15T16:30 |
Job order status record
A second file holds one row per job order with the date it reached each status. It feeds the Job Orders box: the opening balance, what was created, placed or gone, and where active job orders sit.
| Field | Type | Required | Definition | Example |
|---|---|---|---|---|
jo_id | Text | Yes | Job order identifier; joins to the pair record | JO-4210 |
recruiter | Text | Yes | Recruiter who owns the job order | Recruiter D |
opened_at | Date-time | Yes | Status Open | 2026-08-12T09:40 |
internal_submit_at | Date-time | If reached | Status Internal Submit | 2026-08-14T11:05 |
external_submit_at | Date-time | If reached | Status External Submit | 2026-08-20T15:30 |
placed_at | Date-time | If placed | Status Placed (filled by our recruiters) | 2026-09-08T10:00 |
ended_at | Date-time | If ended | When it left the active list; blank while active | 2026-09-08T10:00 |
end_status | Placed | Closed | Wash | Cancelled | Filled elsewhere | blank | If ended | How it left the active list | Filled elsewhere |
Data quality checks to run on each load:
pair_idis unique; no candidate appears twice on one job order.match_scoreis present for every Automated row and blank for every Manual row.first_contact_at≥screened_at≥matched_at≥jo_opened_at.contact_attempts= 0 exactly whenfirst_contact_atis blank.- All times share one time zone (Eastern).
6. Dashboard components
The page reads top to bottom, from the overall flow, through what needs attention, to the list of people to call. Every component follows the period picker.
| # | Component | What it shows | Measuring points | Interaction |
|---|---|---|---|---|
| C1 | Header | Title, a one-sentence takeaway for the period, definitions, the date range shown | MP6, MP11, MP12 by lane | Period picker (rolling windows and months) |
| C2 | Flow: job order to follow-up | Job Orders box (active balance rolled forward: at start + created − placed − gone = at end; active by status; gone by status reached; job orders with matches) branching into the Manual and Automated lanes, five steps each in a straight line, with counts, share kept from the previous step, and a bar against the lane's first step. The follow-up steps are dashed; the bigger of their two losses is red | MP1, MP2, MP15, MP3–MP8 | Click any number in the Job Orders box for those job orders; click a step for its pairs |
| C3 | Human / AI markers | A 12px muted icon in each box's top-right corner: a person outline for a recruiter, a sparkle for AI. A two-item key sits under the card's note | — | Hover for the label |
| C4 | Headline figures | High-value pairs due, coverage, in-time rate, untouched pairs, and contact attempts vs pairs reached; each with an AI and Manual split | MP6, MP7, MP9–MP12 | Click a figure, or its AI / Manual split, for the pairs |
| C5 | What needs attention | Plain-language findings with a severity tag and a recommended action | Section 7 | — |
| C6 | Activity isn't coverage | High-value pairs by number of attempts (never, 1, 2, 3, 4+), per lane; "never" in red | MP7, MP9, MP10 | Click a row for those pairs |
| C7 | Coverage by week | Coverage per week matched, one line per lane | MP11 | Hover a week; click for its pairs |
| C8 | Within 24 hours by week | In-time rate per week, one line per lane, with a 100% target line | MP12 | Hover a week; click for its pairs |
| C9 | Follow-through by recruiter | High-value pairs, from AI, followed up, untouched, attempts, coverage, in-time rate | MP6–MP12 | Sort by any column; click a row for that recruiter's pairs |
| C10 | Call list | Every untouched high-value pair in the period: job order, candidate, lane, recruiter, scores, matched date, days waiting (over 7 in red), longest first | MP9, MP14 | Search by job order, candidate or recruiter; click a row to open the job order |
Drill-down
Every count opens a side panel listing the records behind it, for the selected period. Lists sort by any column and download as CSV. Any job order in a list opens its own view: its status path (Opened → Internal Submit → External Submit → Placed, or the status it left from) and every candidate matched to it in both lanes. Back returns to the previous list.
7. Diagnostician rules
The diagnostician runs these rules for the selected period and lists each finding that fires, in this order. It separates a coverage problem (never touched, which points to prioritization) from a speed problem (touched too late, which points to capacity or alerting).
| # | Finding | Fires when | Severity | Recommended action shown |
|---|---|---|---|---|
| R1 | Coverage leak | Automated untouched pairs > 0 | Red if AI coverage < 80%, else amber | Work the call list, AI lane first |
| R2 | Hand-off gap | Manual coverage exceeds AI coverage by 10 points or more | Red | Route 90+/90+ results into the recruiter's queue with an owner and a due time |
| R3 | Activity ≠ coverage | AI contact attempts > AI pairs followed up | Amber | Measure recruiters on unique pairs reached, not calls; cap repeat attempts until the untouched list is cleared |
| R4 | Speed | Overall in-time rate < 100% | Red if < 80%, else amber | Check recruiter capacity and whether a 90+ result triggers an alert |
| R5 | Job orders at risk | MP15 > 0 | Red | Flag the clients getting no candidates from an AI-qualified pool |
| R6 | Hotspot | A recruiter has untouched pairs; the one with the most is named | Amber | Rebalance job orders or add a second recruiter to the AI lane |
| R7 | Trend | Both halves of the period have at least 8 AI high-value pairs; compares AI coverage first half vs second half | Green if holding, red if falling | If falling: recruiter capacity is not keeping up with AI volume |
| R8 | Going cold | Any untouched pair; the longest-waiting one is named | Red if waiting > 7 days, else amber | Strong candidates are usually off the market within one to two weeks |
If no high-value pairs are due in the period, a single "Quiet" finding replaces the list.
8. Assumptions, open items and later phases
The prototype runs on sample data generated for this design: 3,022 candidate pairs on 130 job orders, 8 recruiters, Jul 17 – Oct 15, 2026; plus a job order status file of 365 job orders, 187 of them already active on Jul 17 (the opening balance) and 178 created during the window. A follow-up gap was seeded on purpose. None of its numbers describe real performance.
Open items
- Decide the Manual high-value rule: a screening score of 90+, or a yes/no "passed screening"
- Confirm which Bullhorn status change marks a manual match: Internal Submission
- Confirm the in-time target (100%) and the coverage target (100%)
- Confirm the exact Bullhorn job order status names, especially the sideways exits (Closed, Wash, Cancelled, Filled elsewhere are placeholders)
- Confirm which Bullhorn activity types count as a contact attempt
- Confirm where Magic Matching and digital-worker scores are stored and how to join them to Bullhorn job orders
- Connect Bullhorn so the dashboard reads live data instead of the sample file (on hold: staying on sample data for now; there is no Bullhorn connector, so this will need an export or a warehouse copy)
Later phases
- Phase 2: extend each lane past follow-up to internal submittal to sales, client submittal, client interview, offer and placement, each with conversion rate and time in stage.
- Phase 3: compare the two lanes end to end, from job order to placement, to measure what the AI tools change.