Some Examples of the “Track It Down” Move
Source: Mike Caulfield Substack
Author: Mike Caulfield
Published: 2025-11-25
Source type: guide
Original source: https://mikecaulfield.substack.com/p/some-examples-of-the-track-it-down
Private backup: the full article text is archived in the private repository at archives/articles/mikecaulfield-substack-com-some-examples-of-the-track-it-down.source.md. It is not published on the public Quartz site.
Summary
Mike Caulfield demonstrates the “Track It Down” move by using NotebookLM to index roughly fifty walkthrough videos and surface examples where AI-generated statements needed verification. Many outputs were broadly right, but the process revealed errors, missing context, and better sources. The article shows that AI outputs can be useful starting points only when learners trace claims back to original or authoritative evidence.
Pull quotes
Track the claim down
“In writing it, I have been trying an experiment — I uploaded the 50 or so “Get it in, track it down, follow up” walkthrough videos I have created so far into NotebookLM, had it search the seven hours or so of walkthrough content, and pull eight examples illustrating the usefulness of the “Track It Down” move.”
Often right, sometimes wrong
“Most of them are just examples of confirming that the LLM was correct (which will usually be the case), though in two of eight cases (“Florida coastline” and “Eiffel Tower”) tracking down the claims shows the LLM was wrong.”
Better sources matter
“The initial AI response provided a link which was half-decent, but better sources were necessary for this claim involving Pope Francis (a known Chicago White Sox fan) receiving a Cubs jersey.”
Big ideas
- Students need to check AI answers against real evidence
- Students need to bring the purpose; AI should not supply it for them
Claims
- AI-assisted inquiry should ground claims in evidence
- Research prompts can support inquiry without taking over student judgment
Key evidence and examples
- A Florida coastline example found that AI wrongly said a 1982 CBS report omitted a 25% flooding figure, while original footage showed the figure was mentioned with caveats.
- An Eiffel Tower example showed AI repeating a common 15 cm expansion claim, while the official site clarified the measurement differently.
- A Pope Francis and Cubs jersey claim became stronger when traced to Vatican News.
- An Emma Goldman quote was traced to Jack Frager’s 1973 T-shirt wording and a richer first-person account.
- Examples involving Mr. Rogers, lightning and rainbow photography, Einstein misattribution, and “hoedown” etymology show that source-tracing can confirm, correct, or deepen AI answers.
Education relevance
This is a strong classroom fit for AI literacy, lateral reading, media literacy, citation practice, and teaching students to move from generated answers to evidence.
Durability note
The examples are time-bound walkthroughs, but the durable value is the repeated move: use AI output as a starting point, then track claims down to stronger evidence.