When an AI says it did something it did not

The failure mode that matters most in this category: an AI telling a customer something is done when it is not. How it is detected, and why an absence of evidence is reported as a warning rather than a lie.

Of everything that can go wrong on an automated call, most is recoverable. A caller told nothing is booked will ring back. A caller told they are booked, who is not, turns up to nothing — and you hear about it from them rather than from your software.

How it is detected

Every spoken commitment on a call is paired against what the tools actually returned at that moment. The comparison is mechanical: it does not ask a model whether the receptionist was honest, it checks whether a claim of success has a successful result behind it.

Critical and warning are not degrees of the same thing

A sentence claiming success with a contradicting failure behind it is critical — the system said something its own record disproves. A sentence with no relevant record at all is a warning, and is explicitly not reported as a lie: a gap in what the system can see is not evidence of dishonesty. Telling an owner their receptionist lied when it did not is how they learn to ignore the alerts, and then they miss the critical one.

Why this is a product feature rather than a QA process

A review process finds these after the customer has. This runs on every call, and the result lands in the same place the owner already looks.

The three sentence shapes that get checked

Detection is not sentiment analysis on the transcript. It looks for the specific commitments that can be checked against a tool result: a claim that something is booked, a claim that a confirmation was sent, and a claim that a change was made. Each is paired with what the corresponding tool returned at that timestamp. A claim with a contradicting result is critical; a claim with no corresponding call at all is a warning. Everything else the receptionist said is out of scope, because a mechanism that tried to judge every sentence would be a model marking its own homework.

What an owner is shown, and in what order

Critical findings appear above everything else on the call, with the sentence that was said and the tool result that contradicts it, side by side and timestamped. Warnings appear lower and are worded as gaps rather than accusations. The ordering matters as much as the detection: a system that surfaces twenty warnings above one critical finding has technically reported it and practically buried it.

What this page will not claim

Detection depends on Balaay having a record of the action. Something done entirely outside Balaay cannot be paired against anything, and that gap is reported as a warning rather than assumed to be fine.

Related pages

• What is an AI receptionist?
• What an AI receptionist does
• AI phone receptionist
• AI answering service
• After-hours answering
• AI appointment booking
• How this category is priced
• vs an answering service
• vs a human receptionist
• vs voicemail
• vs a virtual receptionist
• AI front desk
• AI call handling
• Call Truth
• Action receipts
• Verified actions
• Business Twin
• Website to receptionist
• Pre-flight testing
• Reliability
• All industries
• Pricing

Explore Balaay

In summary

When an AI says it did something it did not — The failure mode that matters most in this category: an AI telling a customer something is done when it is not. How it is detected, and why an absence of evidence is reported as a warning rather than a lie. Balaay is an AI receptionist for businesses that run on calls. It learns the business from its website, answers the phone, takes the next step, and shows what it did.

Loading interactive experience…