L2-10 · Build an evidence record
A claim is only as strong as the evidence behind it. An evidence record writes down the claim, where it came from, the calculation, what was assumed, how uncertain it is, and who the data covered. It keeps what was tested apart from what was only assumed.
Level 2 · Grades 7–9 · about 45 minutes
Big idea. A claim is only as strong as the evidence behind it. An evidence record writes down the claim, where it came from, the calculation, what was assumed, how uncertain it is, and who the data covered. It keeps what was tested apart from what was only assumed.
Before this lesson. L2-03, L2-05, L2-06, L2-08, L2-09
Grown-ups. The guide for adults is at the end of this page.
What you will learn¶
I can fill in an evidence record with a claim, source, calculation, assumptions, uncertainty, and scope.
I can point to the part of a claim that was assumed rather than tested.
I can rewrite a claim so it fits the data it came from.
Story: The juice poster¶
Leo spots a poster for Brand X juice. It is made up for this lesson, but it looks like a real advert: “Students who drink Brand X get better grades!” Maya grins. “Let’s build an evidence record and see what holds up.”
They imagine a survey with made-up numbers: 80 students at one imagined school, over one term. Of 30 juice drinkers, 18 out of 30 had mostly A and B grades. That is 60 percent. Of the 50 other students, 20 out of 50 did. That is 40 percent. The gap is 20 points.
Next, Maya draws a causal map. “What if students who sleep well and eat breakfast also drink more juice? Sleep could raise grades on its own.” The survey cannot rule that out. So “no common cause” is an assumption, not something the data tested.
Leo adds two more worries. “Only 80 students is a small sample. Another school could show a smaller gap, or none.” And the scope is narrow: one imagined school, one term. The poster talks about all students everywhere.
They finish the record. Tested: a gap of 20 points in made-up data. Assumed: no common cause. Not tested: whether juice causes better grades. “To test that,” says Maya, “a coin would decide who gets the juice, like the app study in Lesson 8.”
Scattered facts about a made-up juice claim become an evidence record; weak links glow and the scope shrinks.
What the animation shows. Facts about the made-up juice claim float around the screen. Then a card appears with rows for claim, source, calculation, assumption, uncertainty, and scope, and each fact flies into its row. The calculation reads 18 out of 30 vs 20 out of 50, then 60 percent vs 40 percent: 20 points. The assumption row, no common cause, glows amber with the note assumed, not tested. The uncertainty row, only 80 students, glows amber with the note small sample. The scope shrinks from all students everywhere to 80 students, one imagined school, one term.
Hands-on: Write your own evidence record¶
About 20 minutes.
You need:
A printed or hand-drawn template with six rows: claim, source, calculation, assumptions, uncertainty, scope
A pencil
Your notes from Lessons 3, 5, 6, and 8
A calculator (optional)
Steps:
Choose a claim. Use one from Lessons 3, 5, 6, or 8, or a headline from a news site or an advert. Copy it word for word.
Source: write where the claim came from and who collected the data. Say whether the numbers are real or made up.
Calculation: copy or redo the key number, such as a rate, a conditional probability, or a gap. Write each denominator.
Draw a causal map for the claim. Include at least one possible common cause.
Assumptions: list what the claim needs to be true that the data did not test.
Uncertainty: write how many records there were. Could chance or a small sample explain the result?
Scope: write who, where, and when the data covered. Then rewrite the claim so it stays inside that scope.
Finish with one line: what experiment would test the claim, or why one is not possible.
Practice together¶
Rewrite the Brand X poster claim so it fits the made-up data and its scope.
Check your answer
This made-up survey covered 80 students at one imagined school over one term. In it, 60 percent of Brand X drinkers had mostly A and B grades, compared with 40 percent of other students. The survey does not show that the juice caused the difference.
Try it yourself¶
1. In Lesson 8, a coin decided who got the study app. Which assumption from the juice record does a coin make much safer? What still limits the scope?
Check your answer
The coin makes “no common cause” much safer, because motivation cannot decide who gets the app. The scope is still limited: the numbers were made up and came from one imagined school and one simple model.
2. An imagined headline says “Dog owners live longer.” Write one assumption, one uncertainty question, and one scope question for it.
Check your answer
Assumption: nothing like fitness or income affects both owning a dog and health. Uncertainty: how many people were studied, and could chance explain the gap? Scope: which country, which ages, and which years did the data cover?
Talk about it¶
Why do adverts often skip the scope and the assumptions?
Which row of the evidence record is hardest to fill in, and why?
Can a claim be useful even when it is not tested by an experiment? Give an example.
Check yourself¶
1. In the juice record, which item was assumed rather than tested?
A. That 18 out of 30 drinkers had mostly A and B grades
B. That no common cause, such as sleep, affects both juice and grades
C. That 80 students took part
Show the answer
B. That no common cause, such as sleep, affects both juice and grades
The counts came from the made-up survey. Whether a common cause is at work was never checked, so it is an assumption.
2. Which scope statement fits the made-up juice data best?
A. All students everywhere
B. Every student who has ever tried Brand X
C. 80 students, one imagined school, one term
Show the answer
C. 80 students, one imagined school, one term
A scope should name exactly who, where, and when the data covered, and no more.
3. Remember Lesson 6. In made-up data, program A did better than B in easy courses and in hard courses, but worse overall. What usually causes a flip like this? (Review of L2-06)
A. Someone made an adding mistake.
B. The groups had different sizes, because A took mostly hard courses.
C. A is really worse in every course.
Show the answer
B. The groups had different sizes, because A took mostly hard courses.
When most of A’s students took hard courses, A’s combined rate is pulled down. The split shows A did better in each group.
Words to know¶
- evidence record
- A short write-up of a claim with its source, calculation, assumptions, uncertainty, and scope.
- assumption
- Something a claim needs to be true but that the data did not test.
- scope
- Who, where, and when the data covered. A claim should not reach past its scope.
For grown-ups: about this lesson
Why it matters. This capstone lesson practices two NGSS science and engineering practices: constructing explanations and engaging in argument from evidence NGSS Lead States, 2013. The assumptions row draws on causal inference: an association in observational data has a causal reading only under assumptions such as no unmeasured common cause, which a randomized experiment makes far more plausible Hernan & Robins, 2020. Pearl stresses writing causal assumptions down explicitly, for example as a graph, so others can examine them Pearl, 2009. The juice survey is entirely made up, and its numbers (18 of 30 against 20 of 50) are stored with the animation so the text and video match.
Watch for these mix-ups.
A big gap in a table proves the claim. (A gap shows an association in that data; a cause needs more, such as a randomized trial or a well-argued causal map.)
Assumptions are weaknesses to hide. (Every claim rests on some; writing them down lets others check them.)
A result from one school applies everywhere. (Keep the claim inside the scope the data covered.)
Uncertainty means the result is useless. (It says how far to trust the result and what extra data would help.)
To make it easier. Fill in only four rows: claim, source, calculation, and scope. Use the juice example and change only the numbers.
To make it harder. Write evidence records for two claims, one from an experiment and one from watched data, and compare their assumptions rows. Then swap with a partner and challenge one assumption each.
Limits. The juice poster and survey are invented. Real headlines can come from strong or weak studies, and a short record cannot replace reading the original study.
On the computer. Grown-ups can see where this idea leads later in Lab: Variational circuits and gradients, written for older learners.
Sources for this guide. NGSS Lead States, 2013Hernan & Robins, 2020Pearl, 2009
Next: L3-01 · Complex numbers and phase
- NGSS Lead States. (2013). Appendix F: Science and Engineering Practices in the NGSS. Next Generation Science Standards. https://www.nextgenscience.org/sites/default/files/Appendix%20F%20%20Science%20and%20Engineering%20Practices%20in%20the%20NGSS%20-%20FINAL%20060513.pdf
- Hernan, M., & Robins, J. (2020). Causal Inference: What If. https://miguelhernan.org/whatifbook
- Pearl, J. (2009). Causal inference in statistics: An overview. Statistics Surveys, 3(none), 96–146. 10.1214/09-ss057