Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

L1-05 · Assign a fair comparison

To test two things fairly, let chance decide who gets which. A coin does not care who is strong or weak. So hidden differences between the groups balance out on average, but not perfectly every time.

Level 1 icon Level 1 · Grades 4–6 · about 40 minutes
Big idea. To test two things fairly, let chance decide who gets which. A coin does not care who is strong or weak. So hidden differences between the groups balance out on average, but not perfectly every time.
Before this lesson. L0-04, L1-03
Grown-ups. The guide for adults is at the end of this page.

What you will learn

Story: Leo’s favorite plane

Leo has folded two paper planes. Plane A has long, thin wings. Plane B has short, wide wings. He wants to know which design flies farther. Sixteen friends will each throw one plane once.

“I will choose who throws which,” says Leo. Maya frowns. “You like Plane A best. What if you give it to all the strong throwers? Then Plane A wins because of the throwers, not the wings.”

To see the problem, a grown-up gives each friend a made-up throwing strength from 1 to 10. Leo sorts the 8 strongest friends to Plane A. Plane A’s group: average 7.6. Plane B’s group: average 3.1. That is not a fair test at all.

So they start over. For each friend, Maya flips a coin. Heads means Plane A, and tails means Plane B. This time the coin gives 8 friends to each plane, but that was luck. Plane A’s group: average 5.5. Plane B’s group: 42 divided by 8 is 5.25, which rounds to average 5.3.

“The averages are close, but not the same,” says Leo. Maya nods. “A coin does not promise equal groups. But it does not take sides. If we did this again and again, neither plane would get the strong throwers more often.”

Coin flips split 16 friends into two groups. Then Leo picks the groups himself. The strengths are made up.

What the animation shows. Sixteen circles stand for friends. Darker circles are stronger throwers, and each shows a made-up strength from 1 to 10. First a coin flip for each friend sends them to Plane A or Plane B, and the labels read average 5.5 and average 5.3. Then Leo picks, and the strong throwers all go to Plane A: average 7.6 against average 3.1.

Hands-on: Coin groups or picked groups?

About 20 minutes.

You need:

Steps:

  1. Ask a grown-up to write a made-up throwing strength from 1 to 10 on one side of each of 20 cards.

  2. Turn the cards number side down so you cannot see the numbers. Write a name or letter on the blank side of each card.

  3. Coin groups: flip the coin once for each card. Heads goes in pile A, and tails goes in pile B.

  4. Turn the cards over. Add up the numbers in each pile, then divide by the number of cards in that pile. Write both averages.

  5. Picked groups: now look at all the numbers. Put the 10 biggest numbers in pile A and the rest in pile B.

  6. Find the two averages again and write them down.

  7. Mix the cards and do the coin groups two more times. Write down the averages each time.

  8. Compare. Were the coin groups close every time? How far apart were the picked groups?

Practice together

A coin sends 4 cards to pile A, with strengths 2, 6, 7 and 5. What is the average strength of pile A?

Check your answer

Add them: 2 + 6 + 7 + 5 = 20. There are 4 cards, so divide by 4. The average is 20 divided by 4, which is 5.

Try it yourself

1. A teacher wants to test a new spelling game. She gives the game to the 10 kids who already spell best. They score higher than the others. Is that a fair test? How could she fix it?

Check your answer

No. The game group were already the best spellers, so they might score higher anyway. She could flip a coin for each kid to decide who plays the game. Then good spellers would most likely end up in both groups.

2. Sam flips a coin for 6 friends. Pile A gets strengths 9, 8 and 7. Pile B gets 2, 3 and 4. Did the coin do something wrong?

Check your answer

No. With only 6 friends, a coin can give lopsided groups by luck. Pile A has an average of 8 and pile B has an average of 3. That happens sometimes. With more friends, the averages are usually closer.

Talk about it

Check yourself

1. Why do scientists use a coin or another chance tool to make groups?

Show the answer

C. So nobody’s choices can pile the strong ones into one group

A coin does not take sides. It balances hidden differences on average. It does not promise exactly equal groups every time.

2. Coin groups give averages of 5.5 and 5.3. Picked groups give 7.6 and 3.1. Which groups make a fairer test?

Show the answer

B. The coin groups

The coin groups are close, so neither plane gets much stronger throwers. The picked groups are far apart, so the strong throwers would decide the winner.

3. Remember Lesson 3. You flip a fair coin 16 times and get 10 heads. What is the relative frequency of heads? (Review of L1-03)

Show the answer

C. 10 out of 16

A relative frequency comes from your own count: 10 heads out of 16 flips. One half is what the fair-coin model predicts, not what you counted.

Words to know

random assignment
Using chance, like a coin flip, to decide who goes in which group.
average
Add up all the numbers, then divide by how many numbers there are.
fair test
A comparison where only the thing you are testing is different between the groups.
For grown-ups: about this lesson

Why it matters. OpenStax explains that random assignment places participants in treatment groups by chance, so that groups are similar in other ways and differences in results can be credited to the treatment; it also names lurking variables as the problem this guards against Barbara Illowsky and Susan Dean, 2023. The NGSS practices for grades 3 to 5 ask children to plan fair tests in which variables are controlled and the number of trials is considered NGSS Lead States, 2013. The animation uses a fixed list of made-up strengths and the course’s seeded random number generator, so the averages in the story, the video and this guide match.

Watch for these mix-ups.

  • Random assignment makes the two groups exactly equal. (It balances them on average over many assignments; one small split can still be lopsided.)

  • Random means careless or messy. (Here random means a planned chance rule that nobody can steer.)

  • Picking groups carefully by hand is fairer than a coin. (A person can steer groups without meaning to; a coin cannot.)

  • A coin must give the same number in each group. (Group sizes can differ by luck; that is fine, and the averages still compare well.)

To make it easier. Use 10 cards with strengths written in plain view, and compare only one coin split with one picked split.

To make it harder. Pool the coin splits from the whole family or class. Plot the gap between the two averages for each split and see that the gaps center near zero. Try 40 cards and see the gaps shrink.

Limits. The strengths are made up so that the hidden difference can be seen. Real experiments with people need consent, and random assignment does not fix other problems, such as measuring the two groups differently.

On the computer. Grown-ups can see where this idea leads later in Lab: Confounding and intervention, written for older learners.

Sources for this guide. Barbara Illowsky and Susan Dean, 2023NGSS Lead States, 2013

Next: L1-06 · Look for a common cause

References
  1. Barbara Illowsky and Susan Dean. (2023). 1.4 Experimental Design and Ethics. OpenStax, Introductory Statistics 2e. https://openstax.org/books/introductory-statistics-2e/pages/1-4-experimental-design-and-ethics
  2. NGSS Lead States. (2013). Appendix F: Science and Engineering Practices in the NGSS. Next Generation Science Standards. https://www.nextgenscience.org/sites/default/files/Appendix%20F%20%20Science%20and%20Engineering%20Practices%20in%20the%20NGSS%20-%20FINAL%20060513.pdf