Louise Spear-Swerling

Assessment of Reading Comprehension

September 2006

Good reading comprehension is the ultimate goal of reading instruction at all grade levels and for all children, including those with learning disabilities. Accurate assessment of reading comprehension is necessary to know if this goal is being met, to identify children who need remediation, and to help plan future instruction. However, many scientific investigators of reading agree that further work on measures of reading comprehension is essential, including development of comprehensive systems of assessment that pinpoint key strengths and weaknesses in individual youngsters.

Existing tests of reading comprehension

Measures of reading comprehension can be individually administered, such as tests given in typical psychoeducational evaluations to determine eligibility for special education services, or group administered, such as state-mandated assessments of reading, in which children with learning disabilities typically participate.

Reading comprehension tests can vary along many other important dimensions besides mode of administration, such as the type of text children are expected to read on the test (e.g., narrative, informational, or poetic material); time constraints and pressure for speed; whether or not children can refer back to the text in answering comprehension questions; and response format, or how children are expected to demonstrate comprehension of what they have read.

Three response formats are especially common: cloze, question-answering, and retellings. Cloze format tests present sentences or passages with blanks in them (e.g., "The fish were swimming in the ____"); the child is expected to read the text and provide an appropriate word to go in the blank (for the previous example, a word such as water, lake, or pond).

In tests with a question-answering format, the child reads passages and answers questions about them; the questions may involve multiple-choice or open-ended items and may be answered orally or in writing.

Retellings require a child to read a text and then orally tell an examiner about what was just read, usually with some sort of coding system for scoring the quality of the retelling.

Concerns about existing tests

In general, different measures of reading comprehension correlate significantly, and quite substantially, with each other. That is, students who score highly on one measure of reading comprehension also tend to score highly on other measures, whereas those who do poorly on one test tend to have difficulty on other measures as well.

However, there is evidence that different tests may tap the abilities that underlie reading comprehension – such as word decoding, vocabulary, listening comprehension, and speed of reading – to different extents, such that scores sometimes can vary substantially in individual cases.

For instance, although all reading comprehension measures tap word decoding, cloze format tests may tap word decoding skills relatively more heavily than do question-answering tests, perhaps because children can rely on the gist of a passage or background knowledge in answering many typical comprehension questions.

Similarly, reading comprehension assessments that require students to write answers to open-ended questions – as do some state-mandated assessments – may be tapping components of writing as well as reading. And a test with stringent time limits clearly will tap speed of reading more than does an untimed test.

Therefore, if only one measure of reading comprehension is given, as is often true, the results can potentially be misleading in certain cases.

In addition, tests of reading comprehension are broad measures that, by themselves, do not usually help teachers pinpoint difficulties in individual students. Two children might obtain the same score on a measure of reading comprehension but might arrive at that score in very different ways. If one child has a strong vocabulary and strong oral comprehension skills coupled with weak decoding, and the other decodes well but has an impoverished vocabulary, then instruction for those two youngsters will need to differ in some important respects. Assessment of key component abilities, such as those mentioned above, is essential in order to interpret reading comprehension performance and facilitate instructional planning.

Finally, current measures of reading comprehension are not geared toward distinguishing specific comprehension processes that might underlie poor comprehension in both listening and reading. Measures identifying such processes could be enormously helpful in diagnosing and remediating comprehension problems. Developing these kinds of measures is currently an area of much interest in the scientific community.

What can practitioners do?

In noting these concerns about existing comprehension tests, I do not mean to suggest that these tests have no utility. Current reading comprehension measures certainly can be informative in many ways. Here are a few specific suggestions for practitioners interested in assessment of reading comprehension:

Look for and use tests that are technically adequate.
Technical adequacy includes reliability (i.e., accuracy and consistency of measurement) and validity (i.e., the test measures what it is intended to measure). Information about the technical adequacy of published tests can be found in the test manuals as well as in many textbooks on assessment.
If possible, use more than one test to assess reading comprehension performance.
An average of scores across two or three tests may give a more accurate indicator of current reading comprehension performance in some cases. Differences in performance across tests, and possible reasons for those differences, should also be considered. For instance, a youngster who performs markedly better on an untimed than a timed comprehension test may have difficulties with reading speed.
Assess key component skills, and use the results of those assessments to interpret reading comprehension performance and plan instruction.
Important component skills in reading include out-of-context identification of real words, decoding of pseudowords (nonsense words), oral vocabulary, listening comprehension, and speed of reading. Component writing skills (e.g., handwriting and spelling) may also be relevant if the comprehension measure requires significant amounts of writing.
Take everyday classroom performance into account.
Observations of everyday classroom performance may provide valuable insights into comprehension strengths and weaknesses. For example, individual students may have a strong interest in or knowledge base about a particular topic that may enable them to read more difficult texts about that topic as compared to other types of content.
Consider both current and upcoming grade expectations.
The comprehension demands of texts, as well as the sheer volume of reading expected of students, escalate dramatically in the middle and upper elementary grades. Thus, a component weakness that appears to have little impact on comprehension in one grade – such as mild vocabulary deficits or slow reading – may have a much greater impact in subsequent grades. Early identification and remediation of these component weaknesses may prevent or ameliorate later reading comprehension problems.

Examples of sources

Peer-reviewed journal articles

Aaron, P. G., Joshi, R. M., & Williams, K. A. (1999). Not all reading disabilities are alike. Journal of Learning Disabilities, 32, 120-137.

Jenkins, J. R., Johnson, E., & Hileman, J. (2004). When is reading also writing: Sources of individual differences on the new reading performance assessments. Scientific Studies of Reading, 8, 125-152.

Nation, K., & Snowling, M. (1997). Assessing reading difficulties: The validity and utility of current measures of reading skill. British Journal of Educational Psychology, 67, 359-370.

Rupley, W. H., Willson, V. L., & Nichols, W. D. (1998). Exploration of the developmental components contributing to elementary school children's reading comprehension. Scientific Studies of Reading, 2, 143-158.

Shepard, L. (2000). The role of assessment in a learning culture. Educational Researcher, 29, 4-14.

Shuy, T. R., McCardle, P., & Albro, E., guest editors. (2006). Reading comprehension assessment (special issue). Scientific Studies of Reading, 10, 221-330.

Spear-Swerling, L. (2004). Fourth-graders' performance on a state-mandated assessment involving two different measures of reading comprehension. Reading Psychology, 25, 121-148.

Other helpful sources

Nation, K. (2005). Children's reading comprehension difficulties. In M. Snowling & C. Hulme (Eds.), The science of reading: A handbook (pp. 248-266). Boston, MA: Blackwell Synergy.

Pearson, P. D., & Hamm, D. N. (2005). The assessment of reading comprehension: A review of practices---Past, present, and future. In S. G. Paris & S. A. Stahl (Eds.), Children's reading comprehension and assessment (pp. 13-69). Mahwah, NJ: Erlbaum.

RAND Reading Study Group. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Arlington, VA: RAND.

Salvia, J., & Ysseldyke, J. E. (2004). Assessment in special and inclusive education (9 th edition). Boston, MA; Houghton Mifflin.

Sweet, A., & Snow, C. (2003). Rethinking reading comprehension. New York: Guilford.