The Proficiency IllusionReport as inadecuate

The Proficiency Illusion - Download this document for free, or read online. Document in PDF available to download.

Thomas B. Fordham Institute

At the heart of the No Child Left Behind Act (NCLB) is the call for all students to be "proficient" in reading and mathematics by 2014. Yet the law expects each state to define proficiency as it sees fit and design its own tests. This study investigated three research questions related to this policy: (1) How consistent are various states' expectations for proficiency in reading and mathematics? In other words, is it harder to pass some states' tests than others? (2) Is there evidence that states' expectations for proficiency have changed since NCLB's enactment? If so, have they become more or less difficult to meet? In other words, is it getting easier or harder to pass state tests? (3) How closely are proficiency standards calibrated across grades? Are the standards for earlier grades equivalent in difficulty to those for later grades (taking into account obvious grade-linked differences in subject content and children's development)? In other words, is a state's bar for achievement set straight, sloping, or uneven? This study used data from schools whose pupils participated both in state testing and in assessment by the Northwest Evaluation Association (NWEA) to estimate proficiency cut scores (the level students need to reach in order to pass the test for NCLB purposes) for assessments in twenty-six states. Among the findings: (1) State tests vary greatly in their difficulty; (2) Most state tests have not changed in difficulty in recent years; (3) Improvements in passing rates on state tests can largely be explained by declines in the difficulty of those tests; (4) Mathematics tests are consistently more difficult to pass than reading tests; and (5) Eighth-grade tests are consistently and dramatically more difficult to pass than those in earlier grades (even after taking into account obvious differences in subject-matter complexity and children's academic development). As a result, students may be performing worse in reading, and worse in elementary school, than is readily apparent by looking at passing rates on state tests. The report presents both national and state findings. The following are appended: (1) Methodology; (2) Summary of Concurrent Validity Studies; (3) Tables A3.1-mathematics and A3.2-reading summarize key information about each of the state alignment studies, showing the year and school term in which the study was conducted, the grades evaluated, and the average number of students in each grade included; (4) Estimated State-Test Proficiency Cut Scores in Reading using MAP (in Percentile Ranks); (5) Estimated State-Test Proficiency Cut Scores in Mathematics using MAP (in Percentile Ranks); (6) Changes in Proficiency Cut Score Estimates and Reported Proficiency Rates on State Assessments--Reading; (7) Changes in Proficiency Cut Score Estimates and Reported Proficiency Rates on State Assessments--Mathematics; and (8) How Consistent Are the Results from This Study and the NCES Mapping 2005 State Proficiency Standards Study? [A foreword by Chester E. Finn, Jr. and Michael J. Petrilli is also included. This report represents a collaboration of the Thomas B. Fordham Institute and the Northwest Evaluation Association.]

Descriptors: Federal Legislation, Mathematics Tests, Test Validity, Reading Tests, Cutting Scores, Mathematics Achievement, Reading Achievement, Elementary Secondary Education, Educational Policy, State Standards, Expectation, Student Evaluation, Instructional Program Divisions, Educational Testing, State Programs, Standard Setting (Scoring), Test Content, Test Results, Achievement Rating, Academic Standards, High Stakes Tests, Testing Problems, Evaluation Problems, Educational Change, Articulation (Education), Data Analysis, Educational Indicators, Educational Principles

Thomas B. Fordham Foundation & Institute. 1701 K Street NW Suite 1000, Washington, DC 20006. Tel: 202-223-5452; Fax: 202-223-9226; e-mail: backtalk[at]; Web site:

Author: Cronin, John; Dahlin, Michael; Adkins, Deborah; Kingsbury, G. Gage


Related documents