How to measure reading comprehension without a percent-of-questions trap

By the IEP Atlas editorial teamLast updated November 1, 2026

Short answer

To measure reading comprehension for an IEP goal, use unpracticed passages at one fixed level, ask the same number and type of items every probe, and pick a scoring method that fits the skill: a rubric for retell, summary and inference, percent accuracy for main idea or literal questions, and opportunities for strategy use. Take 3 baseline probes and graph every probe after that.

Comprehension is harder to measure than fluency. There is no single number like words correct per minute that captures it. Many IEPs fall back on "80% of comprehension questions," and that number tells the team very little.

This guide shows how to build a comprehension probe that gives a stable, honest score, with one fictional student's data from baseline to the first progress report. For what to teach, see the pillar on reading comprehension strategies.

The percent-of-questions trap

"Answers comprehension questions with 80% accuracy" sounds measurable. In practice it hides four problems.

The passage changes. A story about soccer and a passage about volcanoes are not the same task. Background knowledge can swing the score more than skill.
The questions change. Three literal questions are easier than three inference questions. A student can jump from 40% to 80% because this week's questions were easier.
The denominator is tiny. With 5 questions, each one is worth 20 percentage points. One lucky guess looks like a month of growth.
Some questions need no passage. "How do you think the boy felt when his dog was lost?" can be answered from life. That item measures general knowledge, not reading.

None of this means questions are useless. It means the probe needs fixed rules. The steps below set them.

What you need before you start

The goal's skill, named precisely: main idea, retell, inference, monitoring, or another sub-skill.
A set of 8 to 12 unpracticed passages at one level, similar in length and type.
A scoring guide: an answer key, or a rubric with points defined.
A data sheet you will use all year, such as the rubric scoring data sheet or the accuracy data sheet.

Step 1: Name the exact skill

"Reading comprehension" is a group of skills. Measure one. A goal about main idea should be probed with main idea items only. A goal about inference needs inference items only. Mixing them makes the score move for reasons that have nothing to do with the goal.

Write down what counts as correct. For main idea, that might be: "names the topic and the most important point about it, in own words, without a minor detail." For inference: "states an idea not written in the text and quotes or points to a clue."

Step 2: Fix the passage level and type

Choose one level and one genre for the whole year, matched to the goal's condition. If the goal says "unpracticed grade 4 informational passage of 250 to 350 words," every probe passage must fit that.

Never reuse a passage that was taught. A student who discussed a passage on Tuesday is not reading it cold on Friday. Keep the probe set separate from the lesson materials.

Decide on access. If the student has a read-aloud or text-to-speech accommodation for comprehension goals, give it on every probe and say so in the goal. Switching between student-read and adult-read text changes the task.

Step 3: Match the scoring method to the skill

Skill	Method	Why
Literal questions, main idea per paragraph, text structure	Percent accuracy	Each item is right or wrong
Retell, summary	Rubric scoring	Shows which parts are present or missing
Inference with evidence	Rubric (inference plus clue)	Separates a guess from reasoning
Monitoring, fix-up and questioning strategies	Opportunities	Counts use at planned stops
General comprehension screening	Maze or retell curriculum-based measurement	Short, repeatable, standard rules

A rubric for retell might give 2 points each for character, setting, problem, events and solution, plus 2 for order: 12 points in all. Write the rubric once and use it for every probe.

Maze is a timed silent-reading task where the student picks the right word at regular gaps in a passage. Standard published maze and retell measures have their own scoring rules and, for some, norms. Use them as the publisher directs. The National Center on Intensive Intervention reviews progress monitoring tools, including comprehension measures, on its tools chart.^[2]

Step 4: Set the number of items

Use enough items that one answer does not swing the score. Aim for at least 8 to 10 scored items per probe for accuracy measures. For a rubric, use one passage with a rubric of 8 points or more, or two short passages.

Keep the item count the same every time. Write it into the goal: "with 80% accuracy on 10 paragraphs per probe." Now the denominator is visible to everyone who reads the IEP.

Check each question by asking whether a student could answer it without reading. If yes, rewrite or drop it.

Step 5: Take 3 baseline probes and report the median

Give 3 probes on different days under the goal's conditions. Order the scores and report the middle one, with the range. Repeated short measures on a fixed schedule are the base of progress monitoring.^[1] The full method is in how to collect baseline data.

Step 6: Score consistently

Score right after the probe, while the response is fresh. For rubric items, keep 2 or 3 scored samples as anchors: one low, one middle, one high. Compare each new response to the anchors.

Once a quarter, have a second adult score the same response without seeing your score. If you disagree by more than 1 or 2 rubric points, tighten the rubric wording together.

Step 7: Graph and decide

Plot each probe on the same graph as the baseline and draw a goal line from the baseline to the target. Look at 6 to 8 points before deciding whether instruction is working. Comprehension scores bounce more than fluency scores, so a single low probe is not a signal.

Worked example: one student from baseline to the first report

A fictional 6th grader, Maya, reads grade 5 text at 97% accuracy. Her decoding is not the barrier. In class she retells only the last event of a story and leaves out the problem.

Skill: narrative retell. Passages: unpracticed grade 5 narratives of 300 to 400 words, read silently. Scoring: a 12-point story grammar rubric (2 points each for character, setting, problem, key events, solution, and correct order).

Baseline probes scored 4, 6 and 5 points. The median is 5 of 12 and the range is 4 to 6. The team writes a goal from the story grammar retell template: "After silently reading an unpracticed grade 5 narrative, Maya will retell the story scoring at least 10 of 12 points on a story grammar rubric on 3 consecutive biweekly probes."

Probe	Date	Character	Setting	Problem	Events	Solution	Order	Total
Baseline 1	Sep 9	2	1	0	1	0	0	4
Baseline 2	Sep 11	2	2	0	1	1	0	6
Baseline 3	Sep 13	2	1	0	1	1	0	5
Probe 4	Sep 27	2	2	1	1	1	1	8
Probe 5	Oct 11	2	1	1	1	1	1	7
Probe 6	Oct 25	2	2	1	2	1	1	9

All numbers are illustrations. The total rose from a median of 5 to 9 in 6 weeks. The row-level data says more. The problem column moved from 0 to 1, but never to 2. Maya now mentions the problem but does not explain it. The next lessons target stating the problem in a full sentence with "because."

The first progress report can say: "Maya's retell score rose from a baseline median of 5 to 9 of 12 points. She now includes the problem and the order of events. Next focus: explaining the problem." That sentence is far more useful to a parent than "Maya is at 75%."

Common mistakes

Weak: "Will answer comprehension questions with 80% accuracy." Better: "Will state the main idea of each paragraph with 80% accuracy on 10 paragraphs per probe, using unpracticed grade 4 informational passages."
Weak: Probing with the story taught that week. Better: Keep a separate set of unpracticed probe passages.
Weak: Mixing literal and inference questions in one score. Better: Score only the item type the goal targets.
Weak: Five questions per probe. Better: At least 8 to 10 items, or a rubric with 8 or more points.
Weak: Reporting a grade equivalent from a reading test. Better: Report the probe score in the goal's own unit, with the date and conditions.
Weak: Changing passage level when the student struggles. Better: Keep the level fixed for the goal year and change the instruction.

Templates and tools

The comprehension goals page has goals for every sub-skill, each with a matching measure, and the reading comprehension strategies goal hub groups goals by strategy. For 16 worked goals with baselines, see IEP goals for reading comprehension.

The IEP must say how progress on each goal will be measured and when it will be reported.^[3] Write the method, the passage level and the item count into the goal, then paste it into the goal checker to catch anything missing.

Frequently asked questions

What is the best way to measure reading comprehension for an IEP goal?

Use unpracticed passages at one fixed level, the same number of items each time, and a scoring method that fits the skill. Rubrics work for retell, summary and inference. Percent accuracy works for main idea and literal questions. Opportunities work for strategy use.

How many questions should a comprehension probe have?

At least 8 to 10 scored items for accuracy measures, so one answer does not move the score too far. With only 5 questions, each one is worth 20 percentage points. For retell and summary, use a rubric with 8 or more points instead.

Can I use a reading program's unit tests to measure a comprehension goal?

Usually not. Unit tests cover taught material and change in difficulty from unit to unit. Use unpracticed passages at a fixed level instead, so each probe is comparable to the baseline. Unit tests can still inform instruction.

How often should reading comprehension be probed?

Every 1 to 2 weeks is typical. Comprehension scores vary more than fluency scores, so collect 6 to 8 points before deciding whether to change instruction, and look at which rubric rows or item types are improving.

Use it in IEP Atlas

Sources

Progress Monitoring: Data & Goal Setting, National Center on Intensive Intervention
Intervention & Assessment Tools Charts, National Center on Intensive Intervention
34 CFR §300.320: Definition of individualized education program, Legal Information Institute, Cornell Law School

Written and checked by the IEP Atlas editorial team under our editorial policy. This guide describes general practice and federal rules. It is not legal advice; your state may add requirements, and the IEP team makes the decisions for each student.