Reading and verifying AI output

Every lesson in this module has been about sending better input to an AI. This one is about the other half: what you do with what comes back. This is not a minor detail. The habit of verification is what keeps you in control when AI is producing content, code, or claims. Without it, you are trusting a system that is, by its nature, occasionally and unpredictably wrong — and that does not know when it is wrong.

The goal is not to verify everything with the same thoroughness. That would make AI useless as a productivity tool. The goal is to verify appropriately — more when the cost of an error is high, less when you can correct mistakes cheaply.

The "trust but verify" mindset

"Trust but verify" captures the right posture: assume the AI is competent and useful, and also assume you need to check it. These are not in tension. You trust a calculator to give you an answer, and you still glance at the result to see if it makes sense. You trust a colleague's code review and still run the tests.

With AI, the concrete practice looks like this:

• Read before acting. Do not run code you have not read. Do not publish a summary you have not read. Do not send an email drafted by AI that you have not read. It takes seconds and it is your name on the output.
• Understand what it did, not just what it produced. If you read code and cannot explain what each part does, you do not actually know if it is correct — you just know it looks plausible. Ask the AI to explain any part you do not follow, then verify the explanation.
• Test with your own inputs. An AI response may be correct for the example it implicitly had in mind and wrong for your actual data. Running the code with your real inputs is the check.

Evaluating code specifically

Code has a clear ground truth: does it run, and does it do the right thing? That makes code the most straightforward output to verify. A structured approach:

Step 1: Read it. Walk through the logic line by line. What is each line doing? Where does data flow? Are there any calls to functions or methods you do not recognise? (Unrecognised library methods are a common hallucination vector — the model may invent a method name that does not exist.)

Step 2: Check the seams. Where does this code call other parts of your system, or use a library? Verify that those calls are real: look up the method in the documentation, or at minimum run a quick test call. AI-generated code frequently calls real libraries with invented method signatures.

Step 3: Run it. In an isolated environment if possible — not directly in production. Observe the actual output.

Step 4: Test edge cases. The model probably wrote code that handles the obvious case. What about an empty input? A very large input? An input with special characters? A null or missing value? Edge cases are where AI-generated code most often has silent bugs.

Evaluating factual claims

Facts have a different verification path. You cannot run them. Instead:

• Treat them as hypotheses, not conclusions. An AI-stated fact is a claim that should be checked against a source you trust — documentation, an authoritative reference, your own domain knowledge.
• Check specifics, not just plausibility. A date, a library version, a command-line flag, a function signature — these are easy to verify and easy to get wrong. If a claim is specific, check it specifically.
• Be more sceptical of obscure claims. The model is more reliable on high-frequency topics (common library APIs, well-known algorithms) and less reliable on niche topics, recent events, or specific version details. Calibrate your verification effort accordingly.

Evaluating explanations

Explanations are the subtlest output to evaluate. An explanation can be fluent, well-structured, and entirely misleading.

The key question: is the AI actually answering your specific question, or is it producing a fluent, generic response on a related topic?

A model given a question it does not know how to answer precisely will often produce a confident response on the most closely related topic it has strong patterns for. You asked about a specific library version's behaviour; you got a general explanation of the library. The response feels helpful but answers a slightly different question.

To catch this: after reading an explanation, ask yourself "did this answer the specific thing I asked, or did it answer an easier nearby question?" If you are not sure, ask the AI to address your specific case explicitly.

Warning signs in any output type

A few patterns that should raise your alertness regardless of output type:

• Specific-sounding details you cannot easily check. A specific statistic, a citation, a precise date or version number stated without a source — these are exactly the kinds of things AI hallucinates. If you cannot check it, note that you are accepting something unverified.
• Confident answers to questions that should be uncertain. If you ask about a recent event, an edge case in an obscure library, or the "best" approach to a genuinely debated topic, confident certainty is a warning sign, not a reassurance.
• Code that "just solves it" without explaining the trade-offs. Real solutions to non-trivial problems have trade-offs. An answer that presents one approach as obviously correct and ignores alternatives may be skipping the part where your constraints actually matter.
• Plausible-sounding but vague answers. "This depends on your specific requirements" repeated several times, answers that use your words back at you without actually addressing the question — these are signs the model is hedging because it doesn't have the specific answer.

Check your understanding

Where to go next

You have now covered the full conversation loop: the mechanics, how to prompt, how to give context, and how to evaluate output. The module closes with a lab where you will put all of this together — using AI to help you write, read, and test a small Python program from start to finish.