The Vendor Conversation Most HR Leaders Are Skipping

Has anyone in HR approved the AI feature currently screening your candidates? Chances are, it showed up in a product update from a vendor you signed two years ago. One line in a changelog nobody read, and by the time anyone noticed, it had already scored a few hundred resumes.

80% of candidates say employer AI policies are vague, rare, or completely absent, according to Greenhouse Research. Some of that is on HR. A lot of it is because implementing AI wasn't a decision HR was ever included in.

I've written about the AI Wedge disrupting the org chart and about the 4Cs AI planning framework, the four checkpoints to run before handing any workflow to an agent we build ourselves. Before that, most are missing another step: the vendor conversation. Most HR leaders vet a new HRIS or ATS on price and implementation timeline. Almost nobody vets the AI already running inside it with the same rigor they'd apply to an agent they built themselves.

According to Deloitte, companies are putting 93% of their AI budget into the technology and  7% into the people  expected to use it. Vendor vetting lives in that 7%, and it's usually the first thing cut when budget is approved, and the project plan calls for closing it by the end of the quarter. SHRM provides a great ten-category AI Tool Evaluation Checklist for exactly this reason, and most sales cycles never touch it. We run our own version of that discipline before any workflow goes to an agent we've built in-house: four checkpoints, Context, Constraints, Connections, Control. Here's what each checkpoint looks like when the agent belongs to a vendor instead of your internal team.

Context: What the Model Knows About Your Business

Context is the checkpoint vendors skip past the fastest in a demo.

Can the tool be grounded in your job descriptions, your leveling framework, your interview rubrics, or does it score every candidate against a generic model trained on whatever data the vendor could license?

A screening tool with no context of your role definitions can't tell the difference between a strong hire for you and a strong hire for the last company that used it. Ask to see example outputs for a role like yours before you ever sign. A vendor who can't show you what the tool actually produces is asking you to buy a black box and hope it fits.

Constraints: What It's Trained On, and What It's Bound By

Constraints are split into two question sets to ask of every vendor before signing anything. The first has to do with training and data security.

You should ask what is the model trained on?

And also, does your proprietary employee data become part of the vendor's public training set or stay walled off as yours? Most vendors can cleanly answer the first question, but the second is usually more murky.

The second question set has to do with the constraint most vendors don't volunteer: how do you test for bias in your algorithm? Testing for bias means someone actually ran a test. Only 21% of recruiters are "very confident" that their own screening system isn't filtering out qualified candidates. Ariana Moon, the VP of Talent Planning and Acquisition at Greenhouse, put words to what that question is really asking in their recent report: "We need to evolve beyond the black box experience of recruiting and fully embrace a glass box mindset, one that's about transparency." If a vendor can't give you the glass box, the confidence problem lands on you, too.

And, the constraints remain a moving target. As of publishing this article, Texas has required employers to document how their AI systems work since January 1st, and opened its complaint portal this past month. Illinois also required AI-use disclosure to employees and candidates, but then repealed its own proposed rules in June without saying when they're coming back. Colorado repealed its original AI law and replaced it with a narrower one that doesn't bind anyone until January 2027. The EU pushed its high-risk employment deadline out by more than a year, according to a recent regulatory roundup from the National Law Review. None of this is settled, and my point is that any vendor's compliance claim from six months ago may already be wrong. Put the burden of keeping current on the vendor. Don't carry it alone. They should be at the ready with the latest news and disclosures.  

Connections: What It Touches, and What That Costs

Buy one AI-enabled tool, and the risk stays with that vendor. Buy three, and the risk moves to the boundary between them. For example, the compensation data an AI recruiting tool pulls from your HRIS to benchmark a proposed offer. Or, the performance history that an AI copilot references to draft a review. And what about the resume data that flows from your ATS into a screening model built by a company you've never had a call with? Every one of those connections needs a disclosure and an accountable party attached to it. Ask the vendor what data specifically crosses 3rd-party boundaries, who is accountable for its security if so, and if there’s functionality to opt out of it. "The vendor's default settings" is not an answer any HR leader should accept.

That unnamed connection has a cost, whether or not it ever shows up on an invoice. Every integration nobody can fully explain is a liability already sitting on your books. The fix is knowing exactly what each connection does, what data it moves, and who signed off on it before it went live.

Closing the 'AI control gap' requires HR leaders to vet vendor features with the same rigor applied to in-house systems before candidate data ever enters the loop.

Control: Who's Accountable, and What Your People Are Told

Control is the checkpoint that protects the person on the other side of the screen as much as it protects your company, and it's also where most HR leaders are already exposed without knowing it. A recent IBM study found two-thirds of CIOs and CTOs say they're held accountable for AI systems they don't fully control, an "AI control gap" that widens every time deployment outruns governance. HR walks into that same gap whenever a vendor ships a new AI feature that nobody in HR approved.

Three things belong in every vendor contract to close it: a human override on any AI-driven decision, an audit trail you can actually produce if someone asks how a decision was made, and a named contact at the vendor who owns what happens when something breaks.

None of that is optional once the tool touches a real candidate or employee.

Disclosure belongs here, too. Candidates and employees are the ones actually affected by what a vendor's model decides, and they're usually the last to find out how it decided anything. A vendor contract with no disclosure language asks the people it affects to trust a process that nobody agreed to describe to them.

Want the scored version of everything in this post? Download the 4Cs Vendor Evaluation Worksheet.

Want to put these questions in front of a vendor in writing? Get the RFP Template.

Want to run your own team through this? Register for our AI Fluency for HR masterclass in October or November.

Let's Orchestrate.