How to Fact-Check an AI Answer in Under Two Minutes

The dangerous thing about a wrong AI answer isn't that it's wrong. It's that it looks exactly like a right one.

When a person doesn't know something, you can usually tell. They hedge. They slow down. They say "I think" or "don't quote me on this." AI tools don't do any of that. They deliver a wrong date, a misremembered rule, or a study that was never published in the same calm, tidy, well-organized paragraphs they use for the things they have dead right. There's no tell. The confidence is a constant, and it's the same whether the answer came from something solid or from nothing at all.

So the skill that actually matters with these tools isn't prompting. It's checking. And checking doesn't mean redoing the work by hand — if it did, the tool would be pointless. It means knowing which two or three claims in an answer are actually holding the thing up, and spending ninety seconds on those.

First, understand what kind of wrong you're dealing with

It helps to know why these tools fail, because it tells you exactly where to look.

A chatbot isn't looking anything up, unless you've specifically asked it to search the web. Left to itself, it's producing the text that most plausibly follows your question. Most of the time the most plausible text is also the true text, which is why this works at all. But when the model doesn't have something solid to draw on, it doesn't stop and tell you. It produces plausible text anyway.

The practical consequence is the useful part: risk is not spread evenly across an answer. The paragraph explaining how compound interest works is almost certainly fine. The sentence containing a specific percentage, a court case name, or a link is where the trouble lives. Specific-sounding details are precisely what these models are best at generating and worst at getting right.

So don't check the whole answer. Check the specifics.

Which parts of an answer to distrust

This is roughly how I triage anything an AI hands me. It's not a scientific scale — it's a working habit.

What the AI is telling you	Risk	What to do about it
Explaining a general concept	Low	Skim it. Errors at this level are usually obvious once you read carefully.
Summarizing a document you gave it	Low	Spot-check any names and numbers against the original. It has the text, so it's mostly honest here.
Step-by-step instructions for software	Medium	Follow along as you read. Menu names and button labels drift constantly.
Specific numbers, dates, percentages	High	Verify every single one at an original source. Do not pass these along unchecked.
Names of people, products, laws, or rules	High	Search the name on its own and confirm it exists and means what it says.
Citations, studies, cases, or URLs	Highest	Assume fabricated until you have personally opened it. This is the classic failure.
A quote attributed to someone	Highest	Search the exact phrase in quotation marks. Invented quotes are common and embarrassing.
Legal, medical, or financial specifics	Highest	Verify it, and then talk to an actual professional anyway. The stakes don't match the tool.

Run your eye down an answer with that table in mind and the job usually shrinks to two or three sentences worth checking. That's the whole trick.

I asked ChatGPT for a statistic and a source. Here's what I found when I clicked.

ChatGPT giving a specific statistic with a named BLS source — Asking for the exact figure and the exact source, with web search off. ChatGPT, September 2026.

This is what a citation looks like whether it's real or invented. I opened it: the BLS article exists, with that title, and it does say the survival rate fell 20.4 percentage points in the first year. So the answer checked out — this time. The point is that I couldn't have known that from the screen, and neither can you.

For anything that leans on a statistic or a rule, go to the publisher rather than a summary of it. US government statistics live on sites like the Bureau of Labor Statistics and data.census.gov, research papers are searchable on Google Scholar, and a page that has since changed or vanished can usually be found in the Wayback Machine.

Four checks that actually work

1. Ask for the source, then actually open it

"Where does that number come from?" is the single most useful follow-up question there is. Not because the answer is trustworthy, but because of what happens next. Sometimes you get a real, checkable source. Sometimes you get a vague gesture at "industry reports." And sometimes you get a beautifully formatted citation for a paper that does not exist.

The critical part is the clicking. A fabricated citation looks completely normal — plausible authors, a real journal, a sensible year, a URL with the right shape. The only way to tell is to open it. If a claim can't survive one click, it doesn't go in your work.

2. Make it argue against itself

Paste the answer back and ask: "What's the strongest case that this is wrong?" or "What would someone who disagrees with this say?"

This works better than you'd expect, and it works for a specific reason: it stops rewarding agreement. Ask a chatbot "are you sure?" and it will very often just fold and apologize, whether or not it was wrong — you've signalled displeasure and it's obliging you. Asking for the counter-argument gives it a real job instead. A well-grounded answer survives it with minor caveats. A shaky one visibly falls apart, and you can watch it happen.

3. Ask again in a brand-new chat

Open a fresh conversation and ask the same question, worded differently. If you get the same answer, that's mildly reassuring. If you get a meaningfully different one, you've learned something important: it was generating, not recalling.

Be honest about the limits of this one. Consistency is not proof of truth — a model can be confidently and consistently wrong about the same thing all day. Divergence is strong evidence of a problem; agreement is only weak evidence of correctness. Use it to catch errors, not to certify facts.

4. Give it the document instead of trusting its memory

This is the big one, and it changes the odds more than any clever prompt will.

When you paste in the actual contract, report, or policy and ask questions about that text, you've moved the tool from remembering to reading. Its accuracy goes up enormously, and better still, you can check its work — every claim it makes should be findable in the document in front of you. If it says something you can't locate in the source, that's your answer about whether to trust it.

The same logic applies to summarizing a long PDF or pulling insights out of a spreadsheet: give it the material rather than asking what it remembers about the material. And if you want fewer vague answers in the first place, most of that comes down to how you ask the question.

Where this method breaks down

URLs that look perfect and go nowhere. A made-up link has the right domain, a sensible path, and a plausible slug. It just 404s. Always click.

"Studies show" with no study attached. Treat this phrase as an unsupported claim wearing a lab coat. Ask which study.

Confident answers about very recent events. These tools have a cutoff date for what they learned. Ask about last week's news without web search turned on and you may get a fluent answer assembled from nothing.

Arithmetic buried in prose. Numbers inside a paragraph get less careful treatment than you'd think. Ask it to show the steps, or do the math yourself — it takes ten seconds.

Anything about you, your company, or your industry's specifics. It does not know your business, and it will not say so. It will guess, in your own vocabulary, and it will sound right.

Asking it to check itself in the same conversation. It's working from the same context that produced the error, and it's inclined to agree with you either way. A fresh chat or an outside source is worth far more than a self-review.

One thing that doesn't work

You cannot spot a wrong answer by how it's written. There's a persistent belief that bad AI output reads a certain way — a bit stiff, a bit listy, too many em-dashes. Whatever truth that has is about identifying that text was AI-generated, which is a completely different question from whether it's correct. The best-written paragraph in an answer is just as likely to be the fabricated one. Sometimes more likely, because invented material has no messy real-world details to trip over.

Judge the claims, not the prose. And if what you're evaluating arrived unsolicited — an email, a voice message, a video — the stakes shift from accuracy to security, which is a different problem worth knowing about.

Two minutes, every time

None of this is an argument against using AI tools. I use them every day and they save me real time. It's an argument against the specific, avoidable mistake of treating fluency as evidence — of assuming that because an answer is well-organized and confident, somebody checked it.

Nobody checked it. That part is your job, and it's a small one if you aim it properly: find the two or three specifics the answer actually rests on, and confirm those at the source. Everything else you can read the way you'd read a knowledgeable colleague thinking out loud — useful, worth listening to, and not the final word.

Do that consistently and these tools get durably useful.

Search This Blog