How do I stop an AI agent making up product features?
Last updated
Give it one list of what your product does, with an owner on every line, and a rule that it may only repeat lines that are current. When a buyer asks about something not on the list, the agent says it will check and hands over, which is an answer rather than an invention.
Telling the agent only say true things in its prompt does not work, because it has no way to know which things are true. It has your website, an old product sheet and whatever the last call transcript said, and all three read equally confident.
Why it happens
An agent fills a gap with the most plausible sentence. A buyer asks does it integrate with X, and yes is more plausible than I don't know, so it says yes. Nothing about the sentence looks wrong: it is polite, on topic, and passes any check on wording.
The fix is the gap, not the agent. If every capability the agent may mention is a line somebody owns, then not on the list becomes a fact the agent can act on.
What the list needs
- One line per thing you say, not one per document. A datasheet goes out of date as a whole and nobody notices which sentence went wrong.
- A named owner on each line. A line nobody owns is one no agent may use. That turns an empty owner field from a chore into a gap somebody wants to fill.
- A date it was last checked, and how often it should be. Past that date the line reads as overdue and stops being usable until somebody confirms it.
What the agent does with a missing or unusable line
It says it is checking, tells the buyer so, and the question goes to the line's owner — or, where there is no line, to whoever owns that area. The open question is written down, so it is answered rather than forgotten. Answering anyway is not an option anybody can switch on.
What this does not do
It cannot stop a model generating a sentence. It answers the agent's question before it sends, refuses to let it cite a line that is not usable, and records what was said either way.
/keep-agent-product-claims-up-to-date.md covers keeping the list current. /stop-an-ai-sdr-quoting-the-wrong-price.md is the same problem for prices.