How do I check an AI answer?

You ask an artificial intelligence (AI) chatbot how long you have to return a jacket to a store. Sixty days, it says, clearly and politely. That's useful, if it's right. If it's wrong, you find out at the counter on day forty-five. Before you plan around an answer like that, it helps to have a quick way to check it. The check is easier than it sounds.

Most answers don't need checking. If a chatbot suggests a name for your book club, nothing is riding on it. The ones worth a minute are the claims you'll act on: a date, a price, a dose, a rule. Those are the lines that cost you something if they're off. So ask yourself one question. What would it cost me if this line were wrong? If the cost is small, move on. If it isn't, pick the one or two claims the answer depends on, and check those.

Many chatbots can show where an answer came from, and you can always ask. But a link is a starting point. A cited page may not say what the answer claims, and sometimes the source doesn't exist. OpenAI, the company behind ChatGPT, lists made-up citations and references among the mistakes its own chatbot can make. So open the link, and find the sentence that backs the claim. It usually takes seconds.²

The best source is the one in charge of the fact. For a return window, that's the store's own returns page, or the receipt in your hand. For a flight, it's the airline. For a law, it's the government site that publishes it. Any summary, even a good one, is a step away from that. And if the chatbot and the owner's page disagree? Go with the page.

One quick test needs nothing but the chat. Ask the same question again, in a fresh conversation or in different words. A fact that holds steady is a better bet. A fact that changes, sixty days and then thirty, is a sign the model is guessing. That's the one to look up.

For health, money, and legal questions, a chatbot can help you understand the topic and work out what to ask. It shouldn't be the last word. Bring the chatbot's answer along if it helps you ask better questions. A pharmacist, a doctor, an accountant, or a lawyer knows your situation and answers for the advice they give. Organizations are told the same thing in their own terms. The National Institute of Standards and Technology (NIST), a United States government agency, tells them to review and verify the sources and citations in what their AI produces.¹

It sounds like a lot. In practice it's quick, because most days there's one claim to check and one page to open. The first few times take longer. After that, it mostly comes down to one question: where would this be written down? Next time a chatbot gives you a number you plan to act on, find where that number lives, and look at it there.²