All articles
Tips2 min read

How I check an AI answer

A working source link and a neat table do not prove a claim. Check the facts that determine what you will do with the answer.

An answer reads smoothly, includes a table and cites sources. It looks convincing. I would still break it down: what exactly is being claimed, and what evidence would verify it?

Start with the consequences of an error

A mistake in a list of project names is usually minor. A customer commitment, financial calculation or decision about an employee is different. The level of checking should match the intended use.

The NIST AI RMF Playbook offers guidance on assessing risks and evaluating AI systems. The approach below is a practical method for checking one answer, not an official NIST checklist.

Extract the important claims

Start with names, dates, amounts and firm conclusions. Separate information from the input from assumptions added by the model. Compare the claims with the original document or relevant source.

A working link alone is not enough. Read the page and check whether it supports the claim. Check its date and applicability. Asking the same AI whether it is correct is not an independent check.

Recalculate an example

If an answer says a task takes twenty hours per month, find the number of actions, time per action and period used. Put them into a simple calculation. You may discover minutes and hours have been confused.

If inputs are missing, leave the conclusion open. A precise number based on an unknown assumption is not an accurate result.

Record what remains uncertain

When sharing the answer, include any assumptions that still need confirmation. For recurring tasks, a short checklist may help.

The aim is to stop readable text quietly becoming an established fact on which others act.

Sources and further reading

Explanation and commentary based on the sources below. Examples are illustrative.