How to tell whether an AI system can be trusted with client numbers

We have built an AI system that does real client work. It picks up what was decided on calls between firms and their clients, keeps track of it, and checks it against the client’s actual books and bank. This piece is what we learned doing that: where the AI belongs, where it does not, and where our own system got things wrong.

The rule that shaped most of it came from one of our design partners, a practising CPA, who kept coming back to the same point while we were building: better to fail than to pretend to work. If the system cannot tell where something belongs, it should say so. It should never quietly put it somewhere wrong and carry on.

It is not a guide to building anything. It is a guide to judging. If you are a partner, a CPA or a fractional CFO deciding whether to let an AI system near client numbers, you probably cannot see what goes on inside one, and most vendors will not show you. By the end you will have four questions you can put to any of them, and a sense of what good answers sound like.

The books are a fossil record

By the time something becomes a journal entry, the decision behind it happened earlier, often weeks earlier. The ledger keeps the amount, the date and the account. It does not keep the conversation where the owner said they were going to delay the hire, or why.

That matters because a controller is paid for the forward half of the job. Not just what happened, but what was decided, what was expected to follow, and whether it did. A system that only reads the books can tell you a lot about the past. It cannot do that forward half, because the forward half was never written into the books in the first place.

So the first design choice was to catch decisions earlier, when they are made, usually on a call between the firm and the client. At that point there is more to work with than a line item: what was said, what was intended, and when it was supposed to happen.

A captured decision is worthless unless something checks it

It is tempting to stop there. Pick up every decision from every call, store it neatly, done.

But a stored decision that nothing ever looks at again is just a prettier audit trail. The value is in checking it later. The owner said they would cut ad spend from next quarter: did spend actually fall? The client expected a large payment by the 15th: did it arrive?

Checking what the system said would happen against what actually happened is the point. Storage is just the cost of getting there. (We covered the record a firm needs to hold for this in how to track client commitments.)

The rule that falls out: do not capture what you cannot check

Once checking is the point, it works backwards and decides what a captured decision has to contain. To check that something happened, you need to know how much, by when, in which part of the business, and with whom. Amount, date, category, counterparty.

That is the shape a decision has to have. If something said on a call cannot be put into that shape, the system does not capture it. No shape, no capture.

If a required detail is missing, the system flags it for the firm to confirm before saving the commitment. A complete-looking record is not enough; the details must come from the conversation or the firm’s confirmation.

This sounds restrictive. It is meant to be. Anything captured without that shape can never be checked, which means it can never be shown to be wrong, which means it has no business in front of your client.

Why this matters more when AI is doing the listening

A person taking notes on a client call uses judgement about what counts as a decision. They know the difference between “we’re pushing the hire to March” and an owner thinking out loud.

A language model does not know that difference by default. Ask it to pull decisions out of a call and it will find them, whether they are there or not. It will turn an offhand remark into a tidy, well-formed decision. And if the shape demands a date, it will supply one, because producing a complete answer is what it is built to do.

The required shape is what lets the system catch that. Take “marketing feels expensive, I might cut it.” That is a mood, not a decision. There is no amount and no date. Rather than invent them, the system declines to capture it.

For you, this is the difference between a system that produces confident nonsense from small talk and one that only brings you things it can stand behind.

Three questions every piece of work has to answer

Once decisions are in a checkable shape, the rest of the system comes down to sending each piece of work to the right place. We ended up with three questions.

Is this a language problem? Then the model handles it. Working out what was decided from what was said on a call is genuinely a language problem. So is answering a client’s question in plain English. But the model only answers over facts that have already been settled elsewhere. It explains numbers. It does not produce them.

Is this an arithmetic problem? Then plain code handles it, with no AI in it, so it gives the same answer every time. Did the money move? Did it match the invoice? Was the claim right? No language model goes anywhere near these questions. This is most of the system.

Is this a judgement call about this particular client, and has the system already learned the answer? If it is a genuine judgement call and the answer is not yet known, it goes to the firm, and the answer is kept.

The shape a decision has to have is what connects the three. Nothing moves from the model to the plain code until it has that shape. That is what lets language work hand off to arithmetic at all, and it is why the numbers do not drift between runs.

How a claim actually gets checked

Once something is captured, the system has to close it out: confirm it happened, or confirm it did not. There are three routes, tried in order.

First, translate the claim into something observable. “We’re delaying the hire” is not something you can see in a bank feed, but it shows up in a payroll category. Watch that.

Payroll is evidence, not the whole story. If one employee leaves as another joins, total payroll may barely change. The numbers alone cannot settle what happened to the hiring decision.

Second, watch the bank where there is a direct signal. A claim tied to a specific invoice can be checked against the money arriving.

Third, where neither works, hold the claim open and bring it to the next check-in call. More on that below, because it turned out to matter more than we expected.

When the bank closes it by itself

The simplest case needs nobody. Every claim about money owed maps to a specific invoice, and the invoice’s details are fingerprinted. When a bank transaction arrives, it is matched against that fingerprint. This is ordinary reconciliation, rebuilt as plain code. No AI, no person, no judgement.

When the matching got it wrong

Then it broke, in a way anyone who has done a reconciliation will recognise.

A recurring customer is invoiced monthly on 30-day terms. They fall behind, then pay two invoices on the same day, or a day apart. Now there are two open claims for the same customer for the same amount. Our matching could close the September claim with the August money.

Nothing exotic happened. The model did not hallucinate. The model was not involved at all. Reconciliation was simply ambiguous, the same way it is for a person.

What made it fixable is that plain code is not infallible, but it is predictable. It makes the same mistake the same way every time, so you can find it and fix it. The fix was to recognise when two candidates are too close to tell apart. When that happens, the system does not pick. It either declines to match or puts the item in front of a person for review.

It is the same rule as before, pointed at a different problem. Do not record what you cannot check. Do not match what you cannot tell apart.

The claim only a person can close

Some claims cannot be checked from data at all.

A cash forecast can only see what is in the books and the bank. If a business is expecting half a million in investment next week, or plans to draw on a credit line, neither exists anywhere in the data until the money lands. The forecast reads low, and nobody is wrong. It simply cannot know.

So the system picks it up from the call instead. Then there are two ways to check it. If the bank is connected, watch for the money to arrive. If not, the only route left is the next check-in: you said money was coming in, what happened?

The harder version is a claim with nothing behind it yet. “I’m expecting money from this customer,” with no invoice raised. There is nothing to fingerprint and nothing to match. The system can watch for an invoice to appear, but if none shows up within a week, that is not evidence either way.

This is not a gap waiting to be engineered away. Some claims are structurally out of reach of the data, and the honest answer is to hold them open and ask. The regular call is not a workaround. It is how everything the data cannot reach gets checked. A system that claimed to settle these from the numbers alone would be the one to worry about.

The one thing it has to ask you

Most AI products present human review as a safety feature, and imply that more of it is always safer. In practice, a person sprinkled through every step becomes a second job, and people stop doing second jobs.

The person belongs in exactly one place: where the model’s output cannot be checked by the plain code.

Here is the example that taught us this. A client says they are “reducing ad spend by 10% from next quarter.” That is a well-formed decision, with amount, date and direction all present. But “ads” does not match anything in this client’s chart of accounts. The model cannot resolve it, and the plain code cannot either, because it is a question about this particular set of books. Only someone who knows the client can answer it. One tap: ads means marketing.

The test of a step like this is how little it asks. Hand someone a form and they stop using it. Ask them one specific question they can answer in a second, and they will.

It remembers what you confirm

There is a catch. If the system asks about “ads” every month, that one tap becomes a permanent tax on your time.

Once the firm confirms what “ads” means for that client, the system remembers it. The firm should not have to keep resolving the same issue.

This gives you something you can actually measure. The test is whether the firm keeps having to resolve issues it has already confirmed. Those repeated questions should fall as the system remembers the answers. And anyone asked the same question in month three stops trusting all of it, not just that one item.

Sometimes the right answer is no AI at all

After two years of hearing that AI is the answer to everything, this may be the most useful sentence in the piece: the core of our system is plain code.

That is deliberate. The parts that decide whether money moved, whether it matched and whether a claim came true have no AI in them, because plain code gives the same answer every time. The model does the parts where language is genuinely the problem, and nothing else.

If a vendor tells you their product is AI all the way down, it is worth asking what that means for the numbers.

The questions to take to any vendor

What this design buys is a system whose output can be graded. Not an answer that sounds right, but a claim written down before the outcome and checked after it. That is what lets you say whether last quarter’s output was right, rather than just whether it read well.

You do not need to understand the internals to test for this. Four questions do most of the work.

1. Which parts of this are the model doing, and which parts are plain code? A good answer is specific. A vague answer, or “it’s all AI”, tells you the numbers may not come out the same way twice.
2. What does it do when it cannot tell? You want to hear that it stops and says so. A system that always produces an answer is guessing some of the time, and you will not know which time.
3. Does it remember the corrections we make? Ask how a confirmed answer is carried into later work, so the firm is not repeatedly resolving the same issue.
4. How would I know if it had been wrong? If there is no record of what it said would happen, there is nothing to check it against, and the honest answer is that you would not.

Those four are for a firm judging a system it will run. An owner being sold an “AI CFO” has a different question to ask, and it is who owns the decision once the analysis is done.

The examples in this piece came from building AgentLink, an AI controller layer that picks up decisions from client calls in a checkable shape, checks them against what actually happens, and asks the firm only what the data cannot settle. It is sold through CPA and fractional CFO firms. The firm keeps the client, the pricing and the judgement.

Frequently asked questions

Can an AI system be trusted with client financial numbers?

It depends on which parts the AI is doing. A system where a language model handles language, such as what was said on a call or a plain-English answer, and plain code handles arithmetic, such as matching and checking outcomes, can be trusted with numbers in a way that a system using a model for everything cannot. Plain code gives the same answer every time.

What is the difference between an AI mistake and a reconciliation mistake?

An AI mistake is often an invention: a confident, well-formed answer with no basis, such as a made-up date. A reconciliation mistake, like matching a payment to the wrong one of two identical invoices, is genuine ambiguity. Plain code makes it the same way every time, which is what makes it findable and fixable.

Should an AI system ever refuse to give an answer?

Yes. A system that declines when it cannot tell is more trustworthy than one that always produces an answer. Refusing to capture a vague remark, or to match a payment it cannot tell apart from another, is a safety property, not a shortcoming.

How much should a person have to review in an AI finance system?

As little as possible, placed exactly where the system’s output cannot be checked automatically. Once the firm confirms an answer, the system should remember it and carry it into later work, so the firm is not repeatedly resolving the same issue.

Can an AI cash forecast see expected investment or new financing?

Not from the data alone. Money that has been promised but not yet invoiced or received does not exist in the books or the bank. The honest approach is to pick it up from the conversation, hold it open, and check it at the next call.

About the author

Nilanjan Raychaudhuri is the founder of AgentLink (Tublian LLC, Columbus, Ohio), which builds controller-layer software sold through CPA and fractional CFO firms. He has spent the past year interviewing practitioners at fractional CPA and CFO practices about how advisory work is actually delivered. Team page

The practitioner observations in this guide come from design-partner conversations and are used with permission where attributed. Corrections to nilanjan@agentlink.finance.