How do I audit what my AI agents have told customers?
Last updated
From a record written at the moment each promise was made, not from logs read afterwards. A log tells you what was sent; an audit needs what it relied on, who owned that, and which rules were in force when it was said.
Reconstructing those afterwards fails in a specific way. The rules have changed since, the person who owned a price has moved team, and the product sheet has been edited. The answer you rebuild is about today, not about the day the sentence went out.
What each entry has to carry
| field | why it cannot be worked out later |
| what was said, and to whom | the thread may be gone, or in a system you no longer have |
| which agent, and who was behind it | keys get rotated, agents get renamed |
| the line it relied on, as it read then | the line has been edited since |
| who owned that line, then | owners change; the CRO approved this means nothing if it resolves to today's CRO |
| the rules it ran under, and their version | the rules have changed since, so they cannot be rebuilt |
| whether it was allowed, flagged or sent for approval | the answer depends on all of the above, on that day |
Written once and never edited. A record anybody can amend is a draft.
Three questions to run
- Which promises are still open? A promise with nothing closing it is the one a customer will come back about.
- Which promises rested on a line that was overdue or unowned when it was used? Those are the ones nobody could have stood behind.
- What did each agent tell this account, in order? That is the question a renewal call or a dispute starts with.
What it needs from you
A list of what your company says is true, with owners — the audit is only as good as what each promise can point at. And each agent connected, so its checks are written down as they happen rather than assembled from its provider's logs.
What this does not do
It does not read your mailboxes or your CRM to find promises made outside a check. What it records is what your agents asked before they acted, and anything you import. Where a promise went out without a check, the record cannot see it, and the audit should say so rather than read as complete.
/prove-what-an-ai-agent-said.md covers showing the record to a customer. /agent-promise-register.md is what each promise points at.