Red flags in an AI accounting demo

A demo is designed to succeed. The data is chosen, the path is rehearsed, and the awkward cases are absent. That is not dishonest; it is what a demo is.

Which means what you learn from watching one is limited, and what you learn from interrupting one is considerable.

Eight red flags, in rough order of how much they should worry you.

1. You cannot use your own data

The single biggest signal. If a vendor will not process your documents during or shortly after the demo, ask why.

Reasonable reasons exist — a trial environment needs setting up, your data needs anonymising. What should not happen is deflection. A product confident in its extraction lets you try a difficult invoice.

Escalate this one: if you get to the end of an evaluation having never processed your own data, you have bought on a demonstration.

2. The exception queue is not shown

Or is shown briefly, with three tidy items in it.

The exception queue is where you find out what daily life is like — see the exception queue as a control. A vendor skipping past it is skipping past the part of the product you will spend most time in.

3. Confidence is nowhere in the interface

If nothing anywhere expresses how sure the system is, ask directly what determines whether a transaction is reviewed.

Rule-based systems cannot express uncertainty — a rule fires or does not. The absence of any confidence indicator is a reliable signal about what is underneath, whatever the marketing says.

4. "It's 100% accurate" or "fully automated"

Nothing in accounting is fully automated, because exceptions exist by definition. A vendor claiming otherwise is either not counting exceptions or is hiding them in a balancing figure.

The honest version — most transactions process automatically, a minority need a person, here is what that looks like — is more reassuring, not less.

5. Every question is answered yes

A product that does everything does nothing distinctively. Vendors who know their product say "that goes to the exception queue" or "we don't do that well" or "you'd handle that manually".

Test it deliberately: ask about something genuinely difficult — how they handle period cut-off, or a supplier who changed what they sell. An unhesitating yes is a signal, not a reassurance.

6. Explanations appear generated rather than recorded

If the system produces a fluent explanation of why a transaction was treated a certain way, ask whether that is retrieved from a decision record stored at the time, or produced when you asked.

They read identically. Only one is evidence — see explainability in accounting automation.

7. No answer on identifying transactions by rule version

Ask how you would find every transaction processed under a particular rule between two dates. Vendors who have handled a customer incident answer immediately. A vague answer means nobody has needed to, which means you will be the first.

8. The demo never touches period end

Everything shown is a normal mid-month transaction. But accounting difficulty concentrates at period boundaries — cut-off, accruals, reversals, the close.

Ask to see period end specifically. It is the least rehearsed part of most demos.

The request that reveals the most

More useful than any question:

"Take this invoice, and show me what happens when it goes wrong."

Hand over a genuinely awkward document — a duplicate, one from an unknown supplier in an ambiguous category, one where the amount does not match the purchase order.

You are watching two things. The product's failure behaviour: does it stop clearly, explain itself, show what a reviewer would see? And the vendor's response: comfortable and specific, or defensive?

A vendor who enjoys this question has built something they trust. One who steers back to the prepared path has told you what happens when you raise a problem after signing.

What is not a red flag

Some things get held against vendors unfairly:

An unpolished interface. Irrelevant to whether the accounting is right.

Admitting a case is handled manually. That is the correct answer for genuine judgement, and a vendor who says so about cut-off is displaying competence.

A small customer base. Worth knowing, not disqualifying. Ask to speak to two of them.

Not having an integration you need. Ask what the export looks like instead — a scheduled file often does the job.

Common questions

What are the warning signs in an AI accounting demo?

Being unable to use your own data, the exception queue not being shown or shown with only a few tidy items, no confidence indicator anywhere in the interface, claims of 100% accuracy or full automation, every question answered yes, explanations that appear generated rather than recorded, no clear answer on identifying transactions by rule version, and a demo that never touches period end.

What should I ask to see in a demo?

Your own difficult documents processed, the exception queue populated with real data, what happens at period cut-off, and specifically what the product does when something goes wrong. Handing over a genuinely awkward document — a duplicate, or an invoice from an unknown supplier in an ambiguous category — reveals both the product's failure behaviour and how the vendor responds to a problem.

Is it a bad sign if a vendor says something is handled manually?

No, it is usually a good one. Period cut-off, estimates and provisions genuinely require judgement, and a vendor who says so is displaying competence rather than weakness. The concerning answer is an unhesitating yes to every capability question, because a product that does everything does nothing distinctively.

What should not count against a vendor?

An unpolished interface, which has no bearing on whether the accounting is correct; admitting that certain judgement-dependent cases are handled by a person; a small customer base, which is worth investigating by speaking to two of them rather than treating as disqualifying; and a missing integration, since a scheduled file export frequently does the same job.

Related: questions to ask an AI accounting vendor · how to evaluate AI accounting software · what to check before signing