COLLEGE PARK, MD (June, 2017) – A new National Academy of Sciences’ report investigated the achievement levels set for assessing student performance in mathematics and reading in the National Assessment of Educational Progress (NAEP), which is often referred to as the Nation’s Report Card. Peter Afflerbach, a professor in the University of Maryland College of Education, co-authored the NAS report.
The Nation’s Report Card is an important way to measure student progress and achievement in the U.S. and informs educators, policymakers, and the general public of students’ understanding and mastery of academic skills. The new National Academy of Sciences’ report also investigated the value of achievement level descriptors for policy makers, educators, and interested citizens.
“By measuring students’ abilities in subjects at various points in their education, these assessments allow educators and policymakers to compare progress and performance across states and age groups. These comparisons are central to decisions about curriculum and instruction in both reading and math. This is why it’s essential that the measurement criteria are useful,” Dr. Afflerbach said.
The Nation’s Report Card first began in 1969 and is used to assess performance on subjects by testing representative samples of students. The way the assessment measures achievement has evolved over the years, moving from simply evaluating performance on a number scale to the inclusion of achievement levels in 1992, which marked students’ performance in a subject as Basic, Proficient, or Advanced based on their score on a numbered scale.
In the report, the committee reviewed and evaluated the history and process of setting the NAEP achievement levels, along with their reliability, validity, interpretations and uses. They also discussed considerations in setting new standards for the Nation’s Report Card.
In many ways, the NAEP for math and reading is aligned with standard practices and the NAS committee did not recommend changes. Yet, the report findings included:
- In addition to creating detailed achievement-level descriptors (ALDs), the 1992 process also set “cut scores” that indicated the minimum score value for each achievement level, e.g. Proficient level is achieved with a score of 75 or higher for a certain subject.
- The report found “considerable variability” in reliability of the cut scores, as evidenced by a lack of consistency in outcomes on different occasions or with different test questions.
- Noting that “the validity of test score interpretations hinges on the appropriateness of the cut score,” the report suggests further investigation is warranted.
- The achievement-level descriptors have evolved over the years in response to new understandings of reading and math afforded by research, but in general the cut scores used to assess the ALDs did not change.
- In 2005, NAEP changed the ALDs and set a new scale and cut scores for achievement levels in 12th grade math, but not for 4th and 8th grade, creating a lack of coherence in math education across the grades.
- Guidance to various audiences on how to interpret and use the NAEP achievement levels is insufficient and piecemeal.
- Rather than setting new cut scores, which would disrupt the NAEP trend line and ability to interpret the data across years, the committee recommended, in the short-term, working to ensure that the ALDs are aligned to the cut scores.
The NAS committee made several other recommendations, which included a call for regular reviews of the ALDs, research on the relationship between NAEP performance and performance on other measures, such as college readiness, and research on the intended and actual interpretations of the NAEP outcomes by various audiences.
Peter Afflerbach, a member of the NAS committee that conducted this report, is a faculty member in the Department of Teaching and Learning, Policy and Leadership; his research focuses on reading assessment, individual differences in reading, reading comprehension and other aspects of literacy education.
View the full report: https://www.nap.edu/catalog/23409/evaluation-of-the-achievement-levels-for-mathematics-and-reading-on-the-national-assessment-of-educational-progress
