Lab: Exploring AI outputs

The last three lessons gave you a conceptual picture of what LLMs are and where they succeed and fail. Now you will see it directly. Each step below is a small experiment. Take notes — even rough ones — so you can compare what you expected against what you actually got.

Step 1: Ask the same question different ways

Pick a topic you know something about. Ask the AI these three versions of essentially the same question, one at a time, and observe how the answers differ:

1. "Tell me about [topic]."
2. "Explain [topic] in three bullet points for a complete beginner."
3. "I have five years of experience with [topic]. What are the subtleties most people miss?"

What to notice:

• Does the content change, or just the format?
• Which response is most useful to you right now?
• Does the third version introduce anything the first two didn't?

The point is not that one phrasing is "correct" — it is that the model is responsive to framing. You are not sending a database query; you are having a conversation where context shapes the output. This is something you can control.

Step 2: Make a factual claim and verify it

Ask the AI a question with a specific, checkable answer — something like:

• "When was Python first released?"
• "What is the boiling point of water at high altitude?"
• "Who wrote the TCP/IP protocol specification?"

The AI will answer confidently. Your job is to verify that answer using a source you trust: an encyclopedia, a specification document, an official project page.

What to notice:

• Was the answer correct?
• Was any part of it slightly off (a year wrong, a name slightly misspelled)?
• Did the confidence in the AI's tone match the accuracy of the answer?

Do this with at least two questions. You are building calibration: a sense of how much to trust confident AI responses on factual matters.

Step 3: Ask AI to explain something, then evaluate it

Pick a concept you understand reasonably well — something from your coding work, a scientific concept, anything with real depth. Ask the AI to explain it.

Read the explanation as if you were a teacher marking a student's answer:

• Is it accurate?
• Is it complete, or does it leave out something important?
• Is it actually answering the concept, or giving a plausible-sounding but shallow treatment?
• Would a beginner come away with a correct mental model?

Then ask: "What are the common misconceptions about [concept]?"

What to notice:

• Does the second answer match what you would say are the real misconceptions?
• Does it introduce any misconceptions while trying to correct them?

Step 4: Ask it to write a small Python function

Give the AI a clear, specific task. Something like:

"Write a Python function called count_words that takes a string and returns the number of words in it. Words are separated by whitespace."

When it gives you code:

1. Read it before running it. What does each line do? Is the logic correct for the obvious case? What happens with an empty string? What about a string with multiple spaces between words?

2. Run it in a Python environment (your terminal, an online Python runner, or a code block below) with at least three different inputs:

   • A normal sentence
   • An empty string ""
   • A string with extra spaces " hello world "

3. Compare actual output to what you expected. If it differs, try to understand why before asking the AI to fix it.

What to notice:

• Did the code handle all three cases correctly without you asking for that?
• If it didn't, was the failure obvious on reading, or only visible on testing?

This exercise is the whole "read before you run, test with edge cases" habit compressed into five minutes. It is the habit you will use every time you accept code from an AI.

Step 5: Ask something it cannot reliably know

Ask the AI about something that requires information it genuinely does not have:

• A news event from the past few months
• The current price of a specific product
• A prediction about something that hasn't happened yet
• The outcome of a recent sports event

What to notice:

• Does the AI refuse to answer, or does it answer anyway?
• If it answers, is it clearly hedging ("as of my training cutoff..."), or does it sound as confident as when it answered historical questions?
• Can you tell from the answer alone that the information might be unreliable?

Reflect

Before moving to the next module, take a moment to note:

1. One thing the AI did in this lab that you found genuinely useful.
2. One thing it did that surprised you — in either direction.
3. One habit you want to take forward into your AI use based on what you saw.

The goal of this track is not to make you sceptical of AI to the point of never using it — it is to calibrate that scepticism precisely, so you use AI where it helps and verify where it fails. This lab is the calibration session.

Where to go next

You have now seen the strengths and limits firsthand. The next module, Your First AI Conversation, goes deeper on the mechanics of the conversation itself: context windows, message history, and how to structure prompts that get you useful answers consistently.