Questions to ask an AI accounting vendor about the accounting

The commercial and technical questions — data ownership, security, pricing structure, exit terms — are covered in how to evaluate an AI ERP vendor.

These are different. They are about whether the product understands accounting, and they are grouped by the processes you actually run. Each has a tell.

On document capture

"What happens to a document it cannot read confidently?"

Good: it is held, flagged with the reason, and shows what it did extract alongside the original for a person to complete. Bad: anything implying it always reads them, or that low-quality documents are rare.

"Do you need a template per supplier?"

Good: no, and here is a supplier layout we have never seen being processed. Bad: configuration during onboarding. That is the older technology under a newer name.

On coding

"How does it handle capital versus revenue?"

Good: it cannot reliably infer intent from a document, so above your capitalisation threshold it routes to a person, or inherits the classification from the purchase order. Bad: it learns from your history. It does, and that is not sufficient here — this is the most consequential coding error available and it depends on information the invoice does not contain.

"What happens when a supplier changes what they sell?"

Good: it would keep coding to the historical pattern, which is why sampling and periodic review matter — here is how you would spot it. Bad: it adapts automatically. It does eventually, and in the meantime it is confidently wrong, which is the honest answer.

"Can I force a treatment that overrides the model?"

Good: yes, as a hard rule the model cannot override. Bad: you can correct it and it learns. That is not the same thing, and for mandated treatments the difference matters.

On reconciliation

"Show me a payment settling nine invoices, less a credit note, less a deduction."

Good: proposes the combination, shows the arithmetic, flags the unexplained deduction separately. Bad: reports it as unmatched.

"What gets written off automatically?"

Good: nothing, or only below a threshold you set, with every write-off recorded and reportable. Bad: small differences are cleared automatically to keep reconciliations clean. That means the reconciliation always agrees and can never tell you anything is wrong.

"What is your matching rate, and how is it calculated?"

Good: a specific figure with the denominator explained, and an acknowledgement that the remainder is genuine ambiguity requiring a person. Bad: 100%, or a figure with no stated basis. See how accurate is AI in accounting.

On the close

"What does the system do about period cut-off?"

Good: it flags candidates — deliveries near period end, invoices dated just after — and a person decides. Bad: it is handled automatically. Cut-off depends on facts about events, and this is the most-tested area in an audit.

"How are recurring accruals reviewed?"

Good: each carries a recorded basis, an owner and a review date, and the system compares the accrual against the actuals that eventually arrive. Bad: they are scheduled and post automatically. Scheduling is the easy half; drift is the failure mode — see accruals and prepayments under automation.

On exceptions and review

"Show me the exception queue with real data."

Good: grouped by reason, plain-language explanations, proposed treatment with confidence, source document one click away, ageing, routable to people outside finance. Bad: a chronological list of unmatched items.

"What determines whether something is reviewed?"

Good: a confidence threshold you set, which can vary by category and amount. Bad: a single global setting, or no expressed uncertainty at all.

On the record

"Are explanations recorded at the time or generated when I ask?"

Good: recorded. Here is the stored decision record. Bad: the system can explain any transaction. That describes generation, which is a plausible reconstruction rather than evidence.

"How do I find every transaction processed under a particular rule version?"

Good: a query, demonstrated. Bad: an unclear answer. This determines whether correcting a systematic error takes hours or weeks — see what happens when the AI is wrong.

"What does the audit trail record, and who can edit it?"

Good: the six elements in evidence an auditor will accept, and edits are themselves logged. Bad: "full audit trail" with no detail.

The one worth ending on

"What do your customers most often complain about?"

A vendor with real customers knows the answer and will usually tell you. One who says there are no complaints is either new, not listening, or managing you.

The answer is also genuinely useful: whatever it is will probably be your experience too, and you can decide in advance whether it matters.

One worth adding to that list: ask what the assistant can actually explain about a number, and what it can't — see what an AI assistant can and cannot tell you about a number.

Common questions

What should I ask an AI accounting vendor about coding?

How the system handles capital versus revenue, what happens when a supplier changes what they sell, and whether you can enforce a treatment as a hard rule the model cannot override. The first is the most consequential coding error and depends on information the invoice does not contain, so a good answer involves routing to a person or inheriting the classification from a purchase order rather than relying on learned patterns.

What reconciliation questions are most revealing?

Ask what gets written off automatically. If small differences are cleared to keep reconciliations tidy, the reconciliation always agrees and has stopped functioning as a control. Also ask for a demonstration of a payment settling multiple invoices less a credit note and an unexplained deduction, which distinguishes a real matching engine from exact-reference matching.

How do I test whether the audit trail is real?

Ask whether explanations are recorded at the time of the decision or generated when requested, ask how you would identify every transaction processed under a particular rule version, and ask who can edit the audit log and whether those edits are themselves recorded. A generated explanation reads identically to a recorded one while being a reconstruction rather than evidence.

What is a good closing question for a vendor?

What their customers most often complain about. A vendor with real customers knows and will usually say, and whatever they name will probably be your experience too, which lets you decide in advance whether it matters. A claim that there are no complaints indicates a vendor who is new, not listening, or managing you.

Related: how to evaluate AI accounting software · red flags in an AI accounting demo · what to check before signing