Journal/Strategy/On choosing your first AI partner.

On choosing your first AI partner.

Six questions to ask before you sign anything. Half of them have nothing to do with models, frameworks, or benchmarks.

Published
Mar 19, 2026
Reading time
6 minutes
Category
Strategy

If this is your first time hiring an AI partner, the temptation is to evaluate them the way you evaluate a software vendor: ask about the stack, the certifications, the case studies, the pricing tiers. This is not wrong — but it will not protect you from the most common failure mode, which is hiring someone who is technically competent and operationally hostile to your business.

Here are the six questions that, in our experience, separate partners who deliver from partners who explain.

01. "What did the last engagement look like — week by week?"

A good answer here is unglamorous. Week one was interviews. Week two was a workflow map. Week three was a feasibility note nobody asked for but everyone needed. The bad answer is a sizzle reel.

Ask for a real timeline of a real project. If the partner cannot describe their last engagement in calendar weeks with deliverables attached to each, they do not have a method. They have a brand.

02. "When was the last time you told a client not to build something?"

This question separates consultants who get paid by the hour from partners who care about the outcome. The right answer involves a specific story and an awkward conversation. The wrong answer is "we always find something to build."

Most operator engagements have a build/buy/redesign decision in the first month. A partner who never says "buy" or "redesign first" is selling builds, not solutions.

The most useful sentence we say in a discovery call is often: "this isn't an AI problem." — a working principle

03. "Who runs the system after you leave?"

This is the question that ends a surprising number of bad engagements before they start. If the answer is "we will, on a retainer," you are renting a system, not buying one. There are good reasons to rent — but you should know that is what you are signing.

The better answer names someone on your team, lays out a handoff plan, and budgets for documentation that survives the partner leaving. The best partners want to be unnecessary by month six.

04. "What does good look like, and how will we measure it?"

If the partner cannot, in fifteen minutes on a discovery call, propose a measurable target — and the harness to measure it — they will not know whether the project worked when it ships.

This is closely related to the eval-first habit we wrote about in the month-three piece. A partner who does not lead with measurement is a partner who will not be able to defend the project when it is questioned later.

05. "Where does our data live during the engagement?"

The answer here should be specific to the level of paranoia your industry requires. For a regulated firm: in your tenant, never copied. For a less regulated one: the partner's environment, with named retention policies. The wrong answer is hand-waving.

This is not just a compliance question. It is a tell about how the partner thinks. Partners who are casual about data are casual about a lot of other things you will only discover later.

06. "Show me a postmortem you wrote for a project that didn't go well."

The answer to this question is the single best signal in the room. A partner who cannot produce one either has never had a project go sideways — unlikely — or does not write postmortems. Either way, they are not a partner who will be honest with you when something goes wrong on your engagement.

The half that isn't about models: questions 01, 02, 03, and 06 do not mention models, frameworks, or benchmarks. They are about how the partner runs themselves. That is what determines whether your project ships.

A short closing

The model is the easy part of any AI engagement. The hard parts are scoping, measurement, ownership, and honesty. Hire for those, and the model usually takes care of itself.


Filed under: STRATEGY · METHOD
First published: Mar 19, 2026