The National Joint Committee on Learning Disabilities (NJCLD) recognizes the role that state and district-wide assessments of learning outcomes have towards achieving high academic standards for students and documenting educational accountability for the public. For students with learning disabilities such assessments often present both needed opportunities and serious challenges. It is imperative that students with learning disabilities are provided the opportunity to access, participate, and demonstrate knowledge and skills in state and district-wide assessments.
Throughout the United States over the last several years, states have been involved in educational reform. A major focus has been to make public schools accountable for the education of all students, including students with learning disabilities.
This includes setting high academic standards, raising graduation requirements, and creating highstakes state assessments that may affect grade promotion, graduation, scholarship eligibility, and the accreditation of individual schools. With this paper the NJCLD highlights issues and offers guidance on sound implementation of state and district-wide assessments to ensure equal access by students with learning disabilities. Federal regulations require access and accountability for students with learning disabilities. Section 504 of the Rehabilitation Act of 1973 [PL 93-112], as amended, requires that individuals with disabilities, including students with learning disabilities, be given equal opportunity to participate in and benefit from the policies and procedures customarily granted to all individuals. The Individuals with Disabilities Education Act of 1997 [IDEA], [PL 105-17], regulations require that all students with disabilities participate in a state’s accountability system. Additionally, the No Child Left Behind Act of 2001 [NCLB], [PL 107-110], requires that at least 95% of students with disabilities participate in assessments that measure adequate yearly progress of schools, school districts, and states.
Thus, State Boards of Education and Departments of Education must understand that the participation of students with learning disabilities in state and district-wide assessments is not participation just for the sake of participation. Instead, participation in these assessments must lead to informed teaching, improved learning, and the acquisition of needed literacy skills, learning strategies, and social skills that allow students with learning disabilities to access the general education curriculum. Furthermore, these assessments must be related to outcomes that go beyond the schoolhouse door (e.g., employment, technical education, post-secondary education).
The National Center on Educational Outcomes has noted that as of 2000-2001 all states have state assessments and that 22 states were using these assessments as a condition for graduation from high school (Thurlow, Wiley, & Bielinski, 2002). However, there is the potential for harm if high-stakes assessment programs are implemented with insufficient resources or with tests that lack the needed reliability and validity for their intended purposes. As a result:
- Students may be placed at an increased risk of academic failure and dropping out of school; Curriculum and classroom instruction may be severely distorted if the goal of instruction becomes achieving high test scores instead of learning;
- The public and policy makers may be misled by an increase in test scores that are not related to actual educational improvement; Teachers may be required to spend excessive time reviewing and preparing for an assessment instead of providing the needed instruction; and
- Teachers may be blamed or punished for the lack of student achievement that may be the result of inadequate resources for which they have no control.
Conditions for Sound Implementation
The U.S. Department of Education’s Office of Civil Rights is addressing a wide range of discrimination complaints surrounding this issue of testing in the K-12 setting with implications for high standards learning. Therefore, when policies and decisions are made at the state level by Boards of Education and Departments of Education, the following set of conditions, which are adapted from the position statement of the American Educational Research Association (July 2002) on High-Stakes Testing in PreK-12 Education with which the NJCLD agrees, are essential to sound implementation of highstakes assessment programs for all students, including students with learning disabilities.
- Protection Against High-Stakes Decisions Based on a Single Test
Decisions that affect individual students’ life chances or educational opportunities should not be made on the basis of test scores alone. Other relevant information should be taken into account to enhance the overall validity of high-stakes decisions for individual students such as promotion to the next grade or high school graduation. Students must be afforded sufficient instruction and multiple opportunities and ways to demonstrate proficiency. - Adequate Resources and Opportunity to Learn
When academic standards and associated tests are introduced as a reform to change current practice, opportunities for educators to access appropriate materials and professional development consistent with the intended changes should be provided before schools, teachers, or students are sanctioned for failing to meet the new standards. In particular, when testing is used for individual student accountability, students must have an opportunity to learn the tested content and skills. Thus, it must be shown that the tested content and skills have been incorporated into the curriculum, materials, and instruction before highstakes consequences are imposed. - Validation for Each Intended Use
Tests valid for one use may not be valid for another. Each separate use of a highstakes test, for student accountability, school accountability, curricular improvement, increasing student motivation, or other uses requires a separate evaluation of the strengths and limitations of both the testing program and the test itself. Additionally, the manner in which test results are compiled and reported must also be consistent with their intended use. For instance, educational performance of a school with a high mobility rate would not be fairly assessed by a composite score of all students enrolled — many of whom were educated elsewhere. - Sufficient Reliability for Each Intended Use
Reliability refers to the accuracy or precision of test scores. It must be shown that scores reported for individuals or schools are sufficiently accurate to support each intended interpretation. High reliability is essential when high-stakes assessments contribute heavily to decisions about individual students. - Full Disclosure of Negative Consequences of High-Stakes Testing Programs
Where credible scientific evidence suggests that a given type of testing program is likely to have negative consequences, test developers and users should make a serious effort to explain these possible effects to policy makers. - Ongoing Evaluation of Intended and Unintended Effects of High-Stakes Testing
With any high-stakes testing program, ongoing evaluation of both intended and unintended consequences is essential. In most cases, the governmental body that mandates the test should also provide resources for a continuing program of research and dissemination of research findings concerning both the positive and the negative effects of the testing program. For example, unintended consequences of high-stakes assessment when used for school accountability may include:- A narrowing of the curriculum to the exclusion of other areas (e.g., the arts and humanities) that prepare students for life after high school;
- The use of inappropriate “quick fix” approaches to learning;
- Finding ways to exclude students with learning and other disabilities from participation; and
- Creating a culture that focuses on “teaching-to-the-test” at the exclusion of instruction in important content areas.
- Alignment Between the Test and the Curriculum
Both the content of the test and the processes needed in taking the test should adequately represent the curriculum. High-stakes tests should not be limited to that portion of the curriculum that is easiest to measure. The test should be aligned with the curriculum as set forth in state and district standards. Multiple test forms should be used or new test forms should be introduced on a regular basis to avoid a narrowing of the curriculum toward just the content sampled on a particular form and a distorting of instruction. - Validity of Passing Scores and Achievement Levels
When testing programs use specific scores to determine “passing” or define reporting categories like “proficient,” the validity of these specific scores must be established. The purpose and meaning of passing scores or achievement levels must be clearly stated. There is often confusion, for example, between minimum competency levels (traditionally required for grade-to-grade promotion), grade level (traditionally defined as a range of scores around the national average on standardized tests), and “world-class” standards (set at the top of the distribution, anywhere from the 70th to the 99th percentile). Once the purpose is clearly identified, validity studies should be conducted to establish passing scores or proficiency levels consistent with the stated purpose. - Opportunities for Meaningful Remediation for Students Who Fail High-Stakes Tests
Students who do not pass a high-stakes test should be provided meaningful opportunities for remediation. This remediation should focus on the knowledge and skills the test is intended to address, not just the test performance itself. For students with learning disabilities this includes the literacy skills needed to take the assessment. There should be sufficient time before retaking the test to assure that students have time to remedy any areas of concern. - Appropriate Attention to Language Differences
If a student lacks mastery of the language in which a test is given, then that test becomes, in part, a test of language proficiency. Unless a primary purpose of a test is to evaluate language proficiency, it should not be used with students who cannot understand the instructions or the language of the test itself. If English language learners are tested in English, their performance should be interpreted in the light of their language proficiency. Special accommodations for English language learners may be necessary to obtain valid scores. Also, it should be recognized that English language learners might have a learning disability and may need accommodations and modifications. - Appropriate Attention to Students With Disabilities
In testing individuals with disabilities, steps should be taken to ensure that the test score inferences accurately reflect the intended construct rather than the students’ disabilities. Additionally, the tests must be developed to allow accessibility by students with disabilities, including learning disabilities. - Strict Adherence to Rules for Determining Which Students Are to be Tested
When schools, districts, or other administrative units are compared to one another or when changes in scores are tracked over time, there must be explicit policies specifying which students are to be tested and under what circumstances students may be exempted from testing. Such policies must be uniformly enforced to assure the validity of score comparisons. In addition, reporting of test score results should accurately portray the percentage of exempt students. However, there must never be a policy that exempts a student from participation solely based upon the student’s disability.
Universal Design
Additionally, the NJCLD believes that state and district-wide assessments, including high-stakes and web-based assessments, must be developed using the principles of universal design. Universal design means that the assessments must be designed and developed from the beginning to be accessible and valid for the widest range of students, including students with learning disabilities. A review of the research by the National Center on Educational Outcomes (Thompson, Johnstone, & Thurlow, 2002) has identified seven elements of universal design that apply to assessments (see Appendix A for an explanation of each). They are
- Inclusive Assessment Population
- Precisely Defined Concepts
- Accessible, Non-Biased Items
- Amenable to Accommodations
- Simple, Clear, and Intuitive Instructions and Procedures
- Maximum Readability and Comprehensibility
- Maximum Legibility
Legal and Policy Issues
When State Boards of Education and Departments of Education make policies and decisions, they need to understand that the requirements for including all children in assessments, including students with learning disabilities, are based on federal laws and regulations. State and district-wide assessment is an integral aspect of educational accountability systems that provide valuable information that benefits individual students by measuring individual progress against standards. Because of the benefits that accrue as the result of assessment, exclusion from assessments solely on the basis of a learning disability would violate Section 504 and the Americans with Disabilities Act [ADA] of 1990 [PL 101-336]. IDEA provides parents and educators with tools to “promote improved educational results for children with disabilities through early intervention, preschool, and educational experiences that prepare them for later educational challenges and employment.”
The new focus is intended to produce attention to the accommodations and adjustments necessary for children with disabilities to access the general education curriculum and the special services, which may be necessary for appropriate participation’
Children with disabilities must be included in state and district-wide assessments of student progress with individual modifications and accommodations as needed. Thus, the bill requires that the IEP include a statement of any individual modifications in the administration of state and district-wide assessments’
The committee reaffirms the existing Federal Law requirement that children with disabilities participate in state and district-wide assessments.’” (Committee on Labor and Human Resources Report of May 9, 1997)
Making Accommodation and Modification Decisions
A student with a learning disability may be eligible under IDEA or qualified under Section 504. Thus, the student’s individualized education program (IEP) team or 504 committee needs to determine any needed accommodations or modifications in order for the student to participate in state and district-wide assessment programs. As the IEP team or 504 committee makes these decisions they need to base their decisions on the full understanding of the consequences for reporting and accountability. It is important that the IEP team or 504 committee makes sure that the accommodations and modifications do not breach test security nor invalidate the purpose of assessment. For example, a passage that is used to measure reading comprehension should not be read to the student, however, subsequent questions regarding the passage may be.
Accommodations and modifications may be in the areas of timing/scheduling, setting, presentation, and response mode. A list of examples of accommodations and modifications in each of these areas can be found in Appendix B.
One of the most critical aspects of effective high-stakes assessment of students with learning disabilities is the process used to determine which accommodations and modifications the student with a learning disability will need to access and participate in state and district-wide assessment programs. The IEP team or 504 committee through a process that ensures parents’ active participation makes this decision. The IEP team or 504 committee must understand how their decisions regarding accommodations and modifications will affect the use of the scores as well as the student’s ability to be promoted to the next grade or graduate from school with a standard diploma.
The accommodations and modifications the IEP team or 504 committee identify to be used during state and district-wide assessments should be chosen from those the student needs and uses during classroom instruction and assessment and are listed in the student’s IEP or 504 plan. The use of unfamiliar accommodations or modifications on state or district-wide assessments may have a negative impact on the student’s performance. Additionally, it is important that the accommodations and modifications are in no way based on the setting in which a student receives services, the student’s disability, the number of classes the student attends in the general education curriculum, or based solely on the potential the accommodations and modifications have to enhance performance beyond providing equal access.
Questions that the IEP team or 504 committee should consider in determining which accommodations and modifications the student will use in instruction and assessment should be considered relative to state and district-wide assessment programs (Elliot, Thurlow, Ysseldyke, & Erickson, 1997). These include but are not limited to:
Setting/Timing
- Can the student focus on his or her work with 25 to 30 other students in a quiet setting?
- Does the student display behaviors that are distracting to him or her or to other students?
- Can the student take a test in the same way as it is administered to other students?
- Can the student work continuously for the entire length of a test?
- Does the student use accommodations that require more time?
Scheduling
- Does the student take medication that dissipates over time, so that optimal performance might occur at a certain time of the day?
- Does the student’s anxiety level increase dramatically when working in certain content areas; so that these should be administered after all other content areas are assessed?
Presentation
- Can the student listen and follow oral directions given by an adult or an audio tape?
- Can the student read?
Response
- Can the student write? Can the student track from a test booklet to a test response form?
Concluding Comments
The NJCLD strongly believes that the use of assessment accommodations and modifications must be provided to ensure that students with learning disabilities have an equal opportunity to demonstrate what they know. The fact that students with learning disabilities use an accommodation or modification during general instruction or assessment, however, does not automatically mean that they need to use it during state and district-wide assessments. These decisions must be made for each student, taking into consideration the ramifications of such decisions. Schools must be accountable for not only the development of literacy skills, learning strategies, and social skills, but also the academic outcomes of students with learning disabilities. State and district-wide assessments are an important part of demonstrating this accountability. Students with learning disabilities must have access and participate in these assessments. Additionally, the administration of high-stakes tests with appropriated accommodations and modifications will enhance student outcomes, reduce dropout rates, and lead to graduation with proficiency.
Appendix A
| Element | Explanation |
|---|---|
| Inclusive Assessment Population | Tests designed for state, district, or school accountability must include every student except those in the alternate assessment, and this is reflected in assessment design and field testing procedures. |
| Precisely Defined Concepts | The specific constructs tested must be clearly defined so that all construct irrelevant cognitive, sensory, emotional, and physical barriers can be removed. |
| Accessible, Non-Biased Items | Accessibility is built into items from the beginning, and bias review procedures ensure that quality is retained in all items. |
| Amenable to Accommodations | The test design facilitates the use of needed accommodations (e.g., all items can be Brailled) |
| Simple, Clear, and Intuitive Instructions and Procedures | All instructions and procedures are simple, clear, and presented in understandable language. |
| Maximum Readability and Comprehensibility | A variety of readability and plain language guidelines are followed (e.g., sentence length and number of difficult words are kept to a minimum) to produce readable and comprehensible text. |
| Maximum Legibility | Characteristics that ensure easy decipherability are applied to text, tables, figures, illustrations, and response formats. |
Based on Thompson, Johnstone, and Thurlow (2002). From Universally Designed Assessments: Better Tests for Everyone! Prepared by Sandra Thompson and Martha Thurlow National Center on Educational Outcomes Policy Directions Number 14 / June 2002 |
|
Inclusive Assessment Population
When tests are first conceptualized, they need to be thought of in the context of who will be tested. If the test is designed for state, district, or school accountability purposes, the target population must include every student except those who will participate in accountability through the alternate assessment. Assessments need to be responsive to growing demands—increased diversity, increased inclusion of all types of students in the general curriculum, and increased emphasis and commitment to accountability for all students
Precisely Defined Constructs
An important function of well-designed assessments is that they actually measure what they are intended to measure. Test developers need to carefully examine what is to be tested and design items that offer the greatest opportunity for success within those constructs. Just as universally designed architecture removes physical, sensory, and cognitive barriers to all types of people in public and private structures, universally designed assessments must remove all non-construct-oriented cognitive, sensory, emotional, and physical barriers.
Accessible, Non-Biased Items
Items are reviewed through bias review or sensitivity review procedures to ensure that they do not create barriers because of lack of sensitivity to disability, cultural, or other subgroups. But, perhaps more important, items are developed by individuals who understand the varied characteristics of students, and the characteristics of items that might create difficulties for any group of students. Accessibility is incorporated as a primary dimension of test specifications, so that accessibility is woven into the fabric of the test rather than being added after the fact.
Amenable to Accommodations
Even though items on universally designed assessments will be accessible for most students, there will still be some students who continue to need accommodations. Thus, another essential element of any universally designed assessment is that it is compatible with accommodations and a variety of widely used adaptive equipment and assistive technology.
Simple, Clear, and Intuitive Instructions and Procedures
Assessment instructions should be easy to understand, regardless of a student’s experience, knowledge, language skills, or current concentration level. Directions and questions need to be in simple, clear, and understandable language. Knowledge questions that are posed in complex language certainly invalidate the test if students cannot understand how they are expected to respond to a question.
Maximum Readability and Comprehensibility
A variety of guidelines exist to ensure that text is maximally readable and comprehensible. These features go beyond what is measured by readability formulas. Readability and comprehensibility are affected by many characteristics, including student background, sentence difficulty, organization of text, and others. All of these features need to be considered in developing the text of assessments. Plain language is a concept now being highlighted in research on assessments.
Plain language has been defined as language that is straightforward and concise. Strategies for editing text to produce plain language have been identified (see Table 1).
| Strategy | Description |
|---|---|
| Reduce excessive length | Reduce wordiness and remove irrelevant material. |
| Use common words | Eliminate unusual or low frequency words and replace with common words (e.g., replace “utilize” with “use”). |
| Avoid ambiguous words | For example, “crane” should be avoided because it could be a bird or a piece of heavy machinery. |
| Avoid irregularly spelled words |
Examples of irregularly spelled words are “trough” and “feign.” |
| Avoid proper names | Replace proper names with simple common names such as first names. |
| Avoid inconsistent naming and graphic conventions |
Avoid multiple names for the same concept. Be consistent in the use of typeface. |
| Avoid unclear signals about how to direct attention |
Well-designed heading and graphic arrangement can convey information about the relative importance of information and order in which it should be considered. |
| Mark all questions | Give an obvious graphic signal (e.g., bullet, letter, number) to indicate separate questions. |
Source: Brown (1999). |
|
Maximum Legibility
Legibility is the physical appearance of text, the way the shapes of letters and numbers enable people to read text easily. As delineated by Schriver, a leading document designer, text that is legible can be read “quickly, effortlessly, and with understanding” (see Resources—not there). Despite a great deal of research on what the characteristics of maximum legibility are, the personal opinions of editors about how they want text to look often prevail.
Bias results when tests contain physical features that interfere with a student’s focus on or understanding of the constructs that test items are intended to assess. Dimensions can include contrast, type size, spacing, typeface, leading, justification, line length/width, blank space, graphs and tables, illustrations, and response formats (see Table 2).
| Dimension | Maximum Legibility Characteristics |
|---|---|
| Contrast | Black type on matte pastel or off-white paper is most favorable for both legibility and eye strain. |
| Type Size | Large type sizes are most effective for young students who are learning to read, students with visual difficulties, and individuals with eye fatigue issues. The legal size for large print text is 14 point. |
| Spacing | The amount of space between each character can affect legibility. Spacing needs to be wide between both letters and words. Fixed-space fonts seem to be more legible for some readers than proportional-spaced fonts. |
| Leading | Leading, the amount of vertical space between lines of type, must be enough to avoid type that looks blurry and has a muddy look. The amount needed varies with type size (for example, 14-point type needs 3-6 points of leading). |
| Typeface | Standard typeface, using upper and lower case, is more readable than italic, slanted, small caps, or all caps. |
| Justification | Unjustified text (with staggered right margin) is easier to see and scan than justified text especially for poor readers. |
| Line Length | Optimal length is about 4 inches or 8 to 10 words per line. This length avoids reader fatigue and difficulty locating the beginning of the next line, which causes readers to lose their place. |
| Blank Space | A general rule is to allow text to occupy only about half of a page. Blank space anchors text on the paper and increases legibility. |
| Graphs and Tables |
Symbols used on graphs need to be highly discriminable. Labels should be placed directly next to plot lines so that information can be found quickly and not require short-term memory. |
| Illustrations | When used, an illustration should be directly next to the question for which it is needed. Because illustrations create numerous visual and distraction challenges, and may interfere with the use of some accommodations (such as magnifiers), they should be used only when they contain information being assessed. |
| Response Formats |
Response options should include larger circles (for bubble response tests), as well as multiple other forms of response. |
Based on Thompson, Johnstone, and Thurlow (2002). |
|
Appendix B
Examples of Accommodations and Modifications
Timing/Scheduling
- Time of day
- Breaks during test
- Multiple test sessions
- Order of test administration
- Extend the time to complete the test
- Administer the test over several days
Setting
- Preferential seating (e.g., at the front of the room or in a study carrel)
- Small group testing
- Individual testing (one-on-one)
- Special lighting
- Adaptive or special furniture
- Test administration in locations with minimal distractions
- Noise buffers
- Auditory trainers
- Hospital/home
Presentation
- Braille
- Large print
- Enlarging the answer sheet
- Reading directions to students
- Simplifying directions
- Interpreting/transliteration directions (e.g., sign language, cued speech)
- Written directions to accompany oral directions
- Clarifying directions
- Computer
- Increased spacing between items or (fewer?)items per page
- Reading test questions
- Interpreting/transliteration test items (e.g., sign language, cued speech)
- Audio-tape version of test items
- Amplifying equipment
- Magnifying glass
- Templates
- Mask or markers to maintain place
- Highlight key word or phrases in directions
- Provide cues (e.g., arrows and stop signs) on answer form
- Secure papers to work areas with tape/magnets
- Short-segment testing booklets
Response Mode
- Student marks booklet
- Student responds verbally to scribe
- Student points to response
- Abacus
- Brailler
- Calculators
- Pencil grip
- Large diameter/special grip pencil
- Word processor/computer/typewriter
- Answer recorded on audiotape
- Augmentative or alternative communication devices
- Spell check
- Dictation to a scribe
- Use sign language
- Use template for recording
Additional Resources
Heubert, J. P. and Hauser, M., Eds. (1999). High stakes testing for tracking, promotion, and graduation. National Research Council. Washington, DC: National Academy Press.
McDonnell, Lorraine M., McLaughlin, Margaret J. , and Morison, Patricia, Eds. (1997). Educating one and all: Students with disabilities and standards-based reform. National Research Council. Washington, DC: National Academy Press.
Thompson, S. J., Thurlow, M. L., Quenemoen, R. F., & Lehr, C. A. (2002). Access to computer-based testing for students with disabilities (Synthesis Report 45). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved January 15, 2003, from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Synthesis45.html
U.S. Department of Education —Office of Special Education and Rehabilitative Services. (2000, August). Questions and answers about provisions in the individuals with Disabilities Educational Act of 1997 related to students with disabilities and state and district-wide assessments. (Memorandum OSEP 00-24). Washington, D.C.: Author.
U.S. Department of Education — Office of Special Education and Rehabilitative Services. (2001, January). Clarification of the role of the IEP team in selecting individual accommodations, modifications in administration, and alternate assessments for state and district-wide assessments of student achievement. (Memorandum). Washington, D.C.: Author.
U.S. Department of Education — Office of Special Education and Rehabilitative Services. (2001, January). Guidance on including students with disabilities in assessment programs (Memorandum OSEP 01-06). Washington, D.C.: Author.
U.S. Department of Education, Office of Civil Rights. (2000, December). The use of tests as part of high-stakes decision—making for students: A resource guide for educators and policy-makers. Washington, D.C.: Author.
Additional information and resources relative to students with disabilities on the topics of accommodations, accountability, graduation requirements, standards, universal design, out-of-level testing, and participation can be found on the National Center on Education Outcomes Web site: http://education.umn.edu/NCEO/
Topic:
Evaluation / LD Testing