Trust what AI tells you only as far as you can trace it. A line that repeats something from the material you gave it can be checked in seconds. A figure or conclusion it worked out needs checking against that material before anyone relies on it. And anything it states that is not in what you gave it, however reasonable it sounds, has nothing behind it: it comes out, or someone who knows confirms it.
Why AI mistakes are hard to see
An AI model writes a wrong sentence in exactly the same voice as a right one. There is no hesitation, no change of tone and usually no warning. Anthropic’s documentation calls this hallucination: text that is factually incorrect or inconsistent with the context it was given. It also says plainly that the techniques for reducing it do not eliminate it, and that critical information should always be validated.
At work, the mistakes that cause trouble tend to fall into three shapes:
- A number that was never in the file. A percentage, a date or a total that looks precise and is not in anything you supplied.
- A dropped clause. A condition, exception or deadline that was in the source and quietly disappeared from the summary.
- A promise nobody approved. A reply or memo that commits you to something, such as a discount, a date or extra work, that no one has agreed to.
An example: a contract summary
You manage facilities at a sample insurance office. Your cleaning contract is up for renewal, and you paste the vendor’s renewal terms into an AI assistant and ask for a short summary for your manager. It comes back clear and well organized, and it contains these three lines:
Annual price increases are capped at 3%.
The contract renews automatically for another year.
The vendor has agreed to add weekend coverage at no extra cost.
Read against the terms you pasted, each line is a different problem. The terms say prices may be adjusted annually and give no cap at all: the 3% is a number the model supplied because contracts like this often have one. The automatic renewal is real, but the terms also say you can cancel with 60 days’ written notice, and the summary dropped that, which is the one detail your manager most needed. And weekend coverage was something you mentioned wanting in an earlier message. The vendor never agreed to it. If this summary went out, your manager would think you had negotiated something you have not.
None of these would be caught by reading the summary on its own. They are caught only by reading it next to the source.
Match the check to where each line came from
| Where the line came from | What to do |
|---|---|
| Repeated from what you gave it | Find it in the source and confirm nothing was softened or dropped. Usually quick. |
| Worked out from what you gave it (a total, a comparison, a conclusion) | Check that part against the source before it goes anywhere. Pick a row at random and redo it yourself. |
| Not in anything you gave it | There is nothing to check it against. Take it out, or ask someone who actually knows. |
| About the outside world (a law, a price, a policy) | Verify it on the official source, or leave it out. |
The last row matters because a well-written, reasonable explanation is not evidence. If nothing in front of you supports a claim, how convincing it sounds tells you nothing about whether it is true.
Tip: Ask for the answer to show its working. “Quote the line in the terms that supports each point” turns a summary you have to trust into one you can audit. Anthropic suggests the same thing: have the model cite a supporting quote for each claim, and retract any claim it cannot support.
One good run does not prove it works
The same prompt, sent again word for word, can come back different: other wording, another order, an item included one time and missing the next. That matters most when you are about to rely on a prompt again and again, for example a weekly summary or a set of labels a spreadsheet will group by. One good answer shows what the prompt can produce, not what it will produce every time.
A practical test is to run it two or three times and compare. Anthropic’s documentation lists this as a check in its own right: run the same prompt several times, and treat inconsistencies between the answers as a warning sign. Answers that agree across runs are consistent, not necessarily right: check what matters against a source. Whatever moved, such as the labels, the order or what counted as urgent, needs pinning down in the prompt: give it the list of labels to use, say what goes first, say what to leave out.
What to cut
Checking is not only about errors. AI drafts often run long with lines that are true and still do not belong: a restatement of the question, a paragraph of caveats, a closing offer to help further, background the reader already knows. Ask of each line whether the reader needs it to do what they are going to do. If not, cut it. A shorter draft is also easier to check.
When to stop revising
Every revision can fix one thing and break another. A tighter rewrite may drop the deadline that was right in the last version, or soften a condition while improving the tone. So when you ask for changes, compare the new draft with the old one and ask two questions: what did it fix, and what did it break?
Stop when the draft meets what you said counted as done, every claim traces to a source, and nothing a reader needs is missing. Past that point, more rounds mostly move words around and give new mistakes another chance to appear. If you cannot tell whether it is done, the brief never said, and that is the thing to fix (see how to write a good prompt).