Rules or machine learning? What is really under the bonnet of accounting automation

Two products both say "AI-powered reconciliation". One matches 60% of your transactions and gets stuck. The other matches nearly all of them and keeps improving. The difference is not marketing budget — it is what is actually running.

There are three mechanisms in circulation. They are not interchangeable.

Mechanism one: rules

A person writes conditions. If the description contains "GRAB", code it to travel. If the amount matches an open invoice exactly, match them.

Strengths. Completely predictable. Auditable in the plainest sense — you can read the rule. Fast, cheap, and correct on the cases it was written for. For genuinely fixed processes, rules are not a compromise; they are the right answer.

Weakness. They only know what someone anticipated. Every new supplier, every changed format, every unusual case needs a new rule, and the rulebook grows until nobody can safely change it. Most finance teams have a spreadsheet somewhere that is really a rulebook nobody dares touch.

How it fails: silently and stubbornly. It does not match, and it cannot tell you why beyond "no rule fired".

Mechanism two: classical machine learning

Instead of writing conditions, you show the system examples. It learns the statistical relationship between the features of a transaction — supplier, amount, description, timing — and the treatment you chose.

Strengths. Generalises. A supplier it has never seen gets coded sensibly because it resembles suppliers it has seen. It improves from corrections without anyone writing a rule, and it can report how confident it is, which is the feature that makes automation safe.

Weakness. Needs history. On day one with no data it is guessing. It also learns whatever you did before, including your bad habits — if two people have been coding the same cost differently for years, it learns the inconsistency.

How it fails: confidently, on cases that look like something they are not. This is why the confidence score matters more than the accuracy figure.

Mechanism three: language models

The newest layer. Rather than working from structured features, it reads the document — or your question — as language.

This is what lets a system extract from an invoice layout it has never encountered, understand that "Inv 4471 less agreed rebate" refers to a specific invoice, or answer "why is margin down" in a sentence.

Strengths. Handles the unstructured mess that defeats both other approaches. Requires no template and no training set for basic competence. Explains itself in language a person can read.

Weakness. It is a language system, not an arithmetic one. Left unconstrained it will produce a fluent, plausible, entirely wrong figure — and it will not sound any less certain when it does. That failure mode is genuinely dangerous in finance, and we have written about it separately in AI hallucinations in financial data.

How it fails: fluently. Which is the worst way for a financial system to fail.

Why the answer is all three

A serious accounting system uses each where it belongs:

• Language models to read documents and to answer questions in plain language
• Machine learning to propose treatment and score its own confidence
• Rules to enforce the things that must never vary — approval limits, segregation of duties, the treatments your accountant has mandated

The rules layer is the one buyers overlook, and it is the reason the whole arrangement is safe. You do not want a model deciding whether a RM 90,000 payment needs a second approver. You want a rule, and you want it to be unbreakable.

The correct architecture is: models propose, rules constrain, humans decide the exceptions.

How to tell what you are being sold

Four questions that cut through the demo:

"What happens with a supplier the system has never seen?" Rules-only answers involve configuration. Learning systems make a proposal and tell you how sure they are.

"Can it tell me how confident it is?" Rules cannot — a rule either fires or does not. If nothing in the interface expresses uncertainty, there is probably no model underneath.

"Does it improve if I correct it?" And specifically: does it improve without someone editing configuration? That distinguishes learning from a rulebook with a nicer front end.

"What stops it doing something it should not?" If the answer is "the model wouldn't", walk away. The answer you want describes hard constraints that exist independently of the model.

The practical consequence

If you are buying rules dressed as AI, expect a good first month and a plateau — the configured cases work, the long tail never does, and someone ends up maintaining the rulebook forever.

If you are buying a model with no constraint layer, expect impressive demos and an audit conversation you will not enjoy.

If you are buying all three, expect a slower start while it learns your history, then a system that gets quietly better and hands you a short list of things that genuinely need you.

Common questions

Is rule-based automation the same as AI?

No. Rules execute conditions a person wrote in advance and cannot handle anything nobody anticipated, while machine learning infers treatment from examples and generalises to new cases. A great deal of software marketed as AI accounting is rule-based automation, which is not dishonest about being useful but does mean it will plateau once the configured cases are covered.

Which is better for accounting, rules or machine learning?

Neither alone. Machine learning is better at proposing treatment across a long tail of cases nobody configured; rules are better at enforcing constraints that must never vary, such as approval limits and mandated treatments. A sound system uses models to propose, rules to constrain, and routes anything uncertain to a person.

Why do language models get financial figures wrong?

Because they are trained to produce plausible language rather than to compute, so when asked for a number they can generate one that reads correctly but was never calculated from the underlying records. The mitigation is architectural: the model should retrieve and present figures that the system computed, with every number traceable back to source records, rather than producing figures from its own reasoning.

How can I tell if a product actually learns?

Ask whether correcting it changes its future behaviour without anyone editing configuration, and ask to see a confidence indicator. A rulebook cannot express uncertainty and does not improve from corrections, so the absence of both is a reliable signal that there is no model underneath.

Related: what AI actually does in accounting · why confidence scores matter in finance · AI in accounting