How to evaluate an AI ERP vendor — the questions that actually work

Every vendor claims AI-powered, enterprise-grade, secure and fast to implement. Those words no longer sort anything, which is precisely why everyone uses them.

A useful question has one property: different vendors give meaningfully different answers. Here are the ones that pass that test. Some are uncomfortable for us, and we have flagged those.

On the AI itself

"Show me the AI creating something that does not exist yet."

The single best question, and most demos quietly avoid it.

Watch what happens. If the AI summarises data, answers questions about records, or spots an anomaly — that is AI reading an existing system. Useful, but it is a feature on top of a fixed data model.

Ask them to build a module you invent on the spot. Something specific to you that nobody preconfigured. What happens next tells you which category of product you are looking at, in about ninety seconds. The full distinction is in what is an AI-native ERP.

"What happens when I need something you did not anticipate?"

Every growing business lands here. The answers vary enormously: a change request and a quote, a professional-services engagement, a roadmap item, or "you describe it".

There is no wrong answer — but the cost and timeline attached to it will dominate your total spend, and it is the thing least likely to be in the proposal.

"What does your AI not do well?"

A vendor without a straight answer either has not thought about it or is managing you. Both are informative.

We publish ours: what AI still cannot do in ERP. You should expect something comparable, or at least an unrehearsed answer.

On correctness

"When the AI is wrong, how do I find out?"

If the answer is "you would catch it at month-end", their control loop is weeks long.

Ask specifically: what is flagged, to whom, how fast, and what does the system do when it is uncertain? A system that never says "I am not sure" is not confident — it is not measuring.

"Can you show me the provenance of an extracted value?"

Point at a figure the AI pulled from a document. Where did it come from — which document, which region, which line? A system that cannot answer is unauditable regardless of its accuracy.

"What is in the audit trail when the AI takes an action?"

"The system did it" is not an answer for a regulator. You want: which component, on what input, at what time, approved by whom. More on why in AI hallucinations in financial data.

"Do you track whether human reviewers actually catch errors?"

Almost nobody measures this, and it is the question that separates a real control from documentation of a control. If a person approves four hundred items a day, they are clicking, not reviewing — and you have less oversight than before while believing you have more.

On data

"Is my data used to train models? Show me the contract term."

Non-negotiable, easy to answer, and a vendor who cannot answer immediately has not read their own supply chain.

"What exactly gets sent to the model provider on a given operation?"

Tests whether they designed the boundary or just wired an API in. A good vendor answers concretely — "the invoice image and nothing else". A vague answer means nobody drew the line.

"Where is it processed, geographically?"

Matters if you have residency obligations. Check whether you actually do rather than assuming — both over- and under-estimating this is expensive. Fuller version: where does your business data go.

On the parts vendors undersell

"What happens to my data during migration — and who decides which duplicate customer is real?"

The correct answer is "you do, and we should budget time for it". A confident, fast answer means they either have not done this or intend to move your mess verbatim. Both produce a system nobody trusts. See migrating to an AI-native ERP.

"Who can create things, and who approves?"

Ask any AI-native vendor, us included. A hand-wavy answer means you are buying future sprawl — the new shadow IT is the failure mode, and it is real.

"What does a change cost after go-live?"

Get a worked example, not a policy. For a traditional vendor this is the rigidity tax and it is where the money goes.

"What happens if you go out of business? Can I export the data and the structure?"

Data export is table stakes and everyone says yes. Structure is the real question — can you get the schema, the workflow definitions, the rules? Otherwise you have a CSV and a rebuild.

Fair to ask of any smaller vendor. It should be asked of us.

On the money

"What does it cost, roughly, before we book a discovery call?"

A vendor who will not indicate cost before a call is preserving room to price you individually. That is legal, common, and tells you what kind of relationship you are entering.

Our numbers are on the pricing page precisely so you can have that conversation early, with real figures.

"What is the total for year one, including implementation, migration and the first three change requests?"

The subscription is the visible part. This question surfaces the rest, and the reluctance to answer it is itself the answer.

The questions to ask yourself

More decisive than anything you ask a vendor, and mostly skipped.

"Can two people in this department describe this process the same way?" If not, you have an organisational disagreement, and no software resolves it. You will pay for it either way — better to find out before you buy.

"How many users actually need to be in the system?" Not headcount. Most businesses overcount badly, and this single number moves cost more than any negotiation.

"What have we never had time to do?" If the list is long, this is an opportunity. If it is empty, be honest that you are making a headcount decision — see what finance teams actually do.

"When our business changes, do we want the software to resist or to follow?" The whole decision compressed into one question. Most people know their answer instantly. The buyer's guide unpacks it.

The meta-test

Watch how a vendor handles a question they cannot answer well.

The good ones say "we are not strong there" or "that is a real limitation" or "you probably want someone else for that". The bad ones reframe, or answer a different question, or promise a roadmap item.

You are not buying a feature list. You are buying a relationship with people who will be handling your business records for years. How they behave when the answer is inconvenient is the most predictive data you will get — and you only get it by asking something inconvenient.

Which, to be fair, includes asking us.

Common questions

What is the single best question to ask an AI ERP vendor?

"Show me the AI creating something that does not exist yet." Most demos quietly avoid it. If the AI summarises data, answers questions about records or spots an anomaly, that is AI reading an existing system — useful, but a feature sitting on top of a fixed data model. Ask them to build a module you invent on the spot, something nobody preconfigured, and what happens next tells you which category of product you are looking at.

How do I compare vendor prices fairly?

Ask for the total for year one, including implementation, migration and the first three change requests. The subscription is the visible part; this question surfaces the rest, and reluctance to answer it is itself the answer. Ask separately what a change costs after go-live, and insist on a worked example rather than a policy. A vendor who will not indicate cost before a discovery call is preserving room to price you individually.

What should I ask about how a vendor handles my data?

Three things, all of which should get immediate answers. Is my data used to train models, and can you show me the contract term? What exactly gets sent to the model provider on a given operation — a good vendor answers concretely, a vague answer means nobody drew the boundary. And where is it processed geographically, which matters if you have residency obligations, so check whether you actually do rather than assuming.

What should I work out before I talk to any vendor?

Whether two people in the same department can describe the same process the same way. If they cannot, you have an organisational disagreement that no software resolves, and you will pay for it either way — better to find out before you buy. Also settle how many users genuinely need to be in the system, because most businesses overcount badly and that single number moves cost more than any negotiation.

Related: AI-native ERP vs traditional ERP, or take the fit check — it is built to tell you when the answer is no.