What AI can do, and what it only seems to do
Language models predict the most plausible next word and therefore optimise for plausibility, not truth. They are strong where language is shaped and the material comes from you: summarising, rephrasing, translating, drafting. They are weak where the answer itself is the knowledge: figures, names, sources. Invented answers sound just as convincing as correct ones. For a business this means: run tasks that come with material, check tasks that do not, and never judge an answer by its tone.
This lesson is station 1 of the AI basics course. It needs no prior knowledge and about ten minutes. At the end you will know why the same tool summarises brilliantly and invents sources at the same time.
How does a language model work?
Building on vast amounts of text, a language model predicts the most plausible next word. It does not understand the way a person does, and it does not look things up in a database. It works with probability, not with truth.
That sounds like a technical footnote, but it explains almost everything in daily use. Plausible and true often coincide, which is why the answers are so often usable. They do not always coincide, and the model itself does not notice the difference.
This lesson in three sentences:
A language model predicts the most plausible next word. It optimises for plausibility, not for truth. That is why right and wrong answers sound equally convincing, and why a human check remains part of any serious use.
What is it strong at?
The tool is strong wherever language needs shaping while the knowledge is already there: summarising, rephrasing, translating, sorting, drafting. You supply the material, the model supplies the form.
Little goes wrong with these tasks, because the truth is already in the material. The model does not have to know the minutes you paste in, only condense them.
Where does it err confidently?
It is weak wherever the answer itself is the knowledge: figures, names, dates, legal questions, sources. If the model lacks the information, a fluent answer appears anyway, because some next word is always plausible.
This inventing has a name: hallucination. It is not a rare defect but the flip side of the same mechanics that make summarising so good. How often it happens depends on model, task and wording. A universal rate does not exist. Measured figures only ever hold for one specific test, never for your case.
What does that mean for a business?
Two consequences are enough to start with. First: tasks with material work, tasks without material need checking. Second: the more convincing an answer sounds, the less its tone says about whether it is right.
Exercise: five minutes on your own case
Ask the tool of your choice two questions. First one with material: paste a longer email and have it summarised in three sentences. Then one without material: ask for three specialist firms in your trade in your district, with addresses. Switch off the tool's web search for this exercise if it has one – with web search you are testing the web, not the model.
Compare the results with reality. The first answer is usually good, the second often partly invented. That is exactly the difference to keep in mind from now on. Continue with lesson 2: the six building blocks of a good instruction.
