How does AI write an answer, one word at a time?

Ask a chatbot a question and watch the answer arrive. It doesn't appear all at once. Words spill onto the screen a few at a time, as if someone on the other end were typing. That's more than a design touch. The answer is being made while you watch, and the way it's made explains a lot about artificial intelligence (AI) chatbots, the good parts and the frustrating ones.

Underneath, a chatbot runs a short loop, over and over.

1. 1.It reads everything so far: your question, plus any words it has already written.
2. 2.It works out which small piece of text is most likely to come next, usually a word or part of one.
3. 3.It adds that piece to the end.

Then it goes back to the top and does it again, hundreds of times, until the answer is finished. The whole answer isn't written out ahead of time.¹²

Because every guess reads everything before it, each new word leans on the ones already written. An answer that opens with "Yes" heads down one road. One that opens with "It depends" heads down another. As it writes, the model keeps building forward from what it has already said, and it doesn't go back to rewrite its first line to fit where it ended up. That's also why an answer that sets off down the wrong road can stay on it for a whole paragraph.

Each step picks something that fits the words before it. Do that hundreds of times and you get sentences that flow, grammar that holds together, and a tone that matches the question. The model learned what fits from an enormous amount of text during training. So even a rough answer usually reads smoothly. Smooth is the default.¹

But fitting the words before it is a different job from checking the facts: a sentence can sound exactly right and still be wrong, because "sounds like what comes next" is the test each word passed. Researchers at OpenAI have written that the way models are trained and graded tends to reward a confident guess over saying "I'm not sure." That's one reason a mistake can arrive in the same smooth voice as the truth.³

One word at a time doesn't mean no thought for what comes later. In 2025, researchers at Anthropic looked inside one of their models while it wrote a rhyming poem. Before it wrote the second line, it had already settled on a rhyming word to end it with, then wrote a line to land there. It still writes one piece at a time. But the pieces can be aimed at something, in that case a rhyme.⁴

So when an answer starts strong, keep reading. A good first line doesn't promise a good last paragraph, because every word after it was a fresh guess. The next time you watch a reply type itself out, ask for the same thing again in a new chat, and see whether the second version takes a different road.