Hiring AI Promised Objectivity. It Delivered a Single Point of Failure.

· AI Evaluation & Safety · 9 min read

One vendor's score becomes everyone's verdict.

The analysis

The bias debate around hiring AI has been framed as a question about individual systems: is this model fair, was this training set representative, does this vendor audit for disparate impact. Those are the right questions to ask of one tool. They are the wrong questions to ask of a labour market.

When a small number of screening vendors serve a large share of employers, the failure mode changes character. A human recruiter with an idiosyncratic preference disadvantages a candidate at one firm; the candidate applies elsewhere and the idiosyncrasy washes out. A shared model with the same preference disadvantages that candidate everywhere at once, silently and consistently. The harm is not that any single decision is worse — it may well be better on average — but that the decisions stop being independent. Monoculture removes the diversity of error that made the old system survivable for people who fell outside the median profile.

This reframes the procurement question. Concentrating on one screening vendor looks like a cost and consistency win on the business case and is a correlated-risk decision in reality — legally, reputationally and in terms of the talent you never see.

For a people function the practical response is unglamorous: keep at least one non-model path into the pipeline, audit rejection patterns rather than only hire patterns, and treat vendor consolidation in screening as a risk committee topic rather than a purely commercial one.

Full essay on Substack: Hiring AI Promised Objectivity. It Delivered a Single Point of Failure..

More on AI Evaluation & Safety

Discuss this with Prasanna: pw@prima-partners.com · book a 30-minute call.