Checking AI answers before they go out
You check AI answers with three moves. First, open the cited source yourself, because language models also invent references. Second, ask the counter-question: what speaks against this answer? Third, never adopt figures unchecked; every figure that leaves the business needs a primary source. Confirmation by the model itself checks nothing. The three moves cost minutes and catch the frequent and expensive mistakes, though not all of them.
This lesson is station 5 of the AI basics course and the third of three lessons; two tools sit in between. It covers the move that decides the value of all the others: checking before anything leaves the business.
Why is common sense not enough?
Because the usual warning signal is missing. With people, uncertainty shows in the tone. A language model can sound certain and be wrong – its tone tells you nothing. Invention and fact arrive in the same confident packaging. Whoever checks by feel is checking the packaging.
Move one: open the source yourself
Language models also invent source references, formally correct-looking ones with plausible titles. A reference is therefore not a check but an invitation to one. Open it, see whether the source exists and whether the statement is really in it.
The order matters: read the source first, then use the claim. The other way round, the invented figure travels into the proposal and the check never happens.
Move two: ask the counter-question
Ask the model for the opposing view: “What speaks against this answer? What would have to be true for it to be wrong?” That is not a real check, but it is a good error detector. Strikingly often the counter-question names exactly the weak spot.
What checks nothing, by contrast: asking the model to confirm its own answer. It does so far too often, even when the answer is wrong.
This lesson in two sentences:
You check AI answers with three moves: open the cited source yourself, ask the counter-question, and never adopt figures unchecked. Confirmation by the model itself is not a check, and the convincing tone of an answer says nothing about whether it is right.
Move three: never adopt figures unchecked
The strictest rule applies to figures, because they err at the highest cost: prices, quantities, deadlines, legal provisions. Every figure that leaves the business has a primary source, or it stays out.
One limit we state openly: these three moves do not find every mistake. They find the frequent and the expensive ones, and together they cost a few minutes. More certainty only comes with expertise in the field concerned.
Exercise: fall for it once, on purpose
Ask the tool for a statistic about your industry, including a source. Switch off the tool's web search for this if it has one – with web search you are testing the web, not the model. Open the source. Check both: does it exist, and is the figure in it?
If both hold, you chose well. If not, you caught an invented source before it ended up in a quote. After that you are ready for the knowledge check, the final station of the course.
