Why FCAT Scores Should Be Compared To More Than One Yardstick
Madeline_Mundt / Flickr
Researcher Matthew Di Carlo takes a second look at just how much improvement there was on Florida Comprehensive Assessment Test writing scores this year.
The headlines last week announced that a higher percentage of Florida students met expectations on the state writing exam. But Matthew Di Carlo at the Shanker Blog says Florida parents and educators should look a little deeper into the results.
Di Carlo explains how the points establishing the state’s 1 to 6 scale — known as cut scores — can provide a false impression of progress.
For instance, Di Carlo notes that the percentage of fourth graders scoring 3.5 or above on the test increased by nine percentage points. The Florida Department of Education loved those results.
But the average fourth grade score improved only slightly. The percentage of fourth graders scoring 3.0 or better, the expectation prior to last year, improved by just two percentage points.
Here’s Di Carlo:
In other words, the “significant gains” touted by the FLDOE are heavily concentrated in a single grade, and, more importantly, they basically disappear if one uses the previous benchmark of 3.0 instead of the current 3.5.
This illustrates a very important point about changes in these kinds of cutpoint-based rates, including proficiency rates, one that we’ve discussed here many times before: Trends in rates depend a great deal on where you set the cut score, and where students are “located” vis-à -vis that threshold.
If you use the 3.5 threshold, there was a moderate increase. But the rate using the 3.0 cut score, which was considered “passing” prior to 2012, was basically flat. It’s the same exact data, but with a different line comes a rather different conclusion.
In Florida’s defense, Di Carlo notes that the state does release the scores so that he an others can dig into the data. Some places, such as the District of Columbia, do not release the scores.
